In our previous post, “How secure is object storage?”, we discussed a range of measures you can use to protect yourself against certain data-loss scenarios. Thanks to server redundancy and erasure coding, object storage systems are quite resilient against hardware and site outages. However, the best protection against accidental or malicious deletion, ransomware, or human error is a backup. In this context, object storage systems are no different than NAS- or SAN-based solutions.
With the PoINT Archival Gateway, PoINT Software & Systems offers a highly available tape-based object storage system for securing hard drive-based S3 object storage. Customers can use cross-region replication (CRR) to set up an automatic asynchronous replication process that sends all new objects from their HDD-based object storage system to the PoINT Archival Gateway, which stores data on cost-effective tape.
One of the object storage systems that permits cross-region replication to an external target is Cloudian HyperStore. As with AWS S3, it only replicates new objects. Any objects already stored in the bucket before CRR is enabled will not be copied over. Versioning must be enabled in both the source and target bucket.
A short overview of the required configuration steps follows:
1. A bucket must first be created within the PoINT Archival Gateway to use as a replication target. Important: Versioning must be enabled. A user account, including S3 access key and S3 secret, must also be created.
2. The next step is to register with the Cloudian Management Console.
3. Create a new bucket in the HyperStore system and enable versioning.
4. Next, in the source bucket’s menu, select “Cross Region Replication” and enable replication. Set the PoINT Archival Gateway as the “Destination Bucket”. The PoINT Archival Gateway is given as the endpoint in the form https://FQDN:PORT. Finally, you need to fill out the “Access Key” and “Secret Key” fields. Done!
From now on, new objects will be automatically replicated in the PoINT Archival Gateway’s bucket. Depending on the erasure code configuration, the PoINT Archival Gateway will save the data to one or more tapes. You can also save to multiple tape libraries in separate fire zones.
Replicated objects retain their original format. If an object named document.docx is replicated, for instance, it will keep the name document.docx in the target bucket and can be accessed easily from the PoINT Archival Gateway using an S3 browser.
The most important question now is what happens when objects are deleted from the HyperStore system.
If no particular version is specified, a “delete marker” will be placed within the HyperStore system and replicated to the PoINT Archival Gateway. If one version in particular is specified, it will be deleted from the HyperStore system but not from the PoINT Archival Gateway.
When running this kind of system, you should monitor the replication process using the log file “cloudian-crr-request-info.log”, located in the /var/log/cloudian/ directory. This file saves the status of each object for which the replication process has started. The status may be COMPLETED, FAILED or PENDING. COMPLETED indicates that the object was successfully replicated. FAILED means that a lasting error occurred with the HTTP status code 403 or 404, and the system will not make another attempt to replicate this object. This may be the result of insufficient permissions for the target bucket, for example. Alternatively, the target bucket may have been deleted. In the case of a FAILED status, an alert will be triggered in the Cloudian Management Console. The PENDING status is triggered by an HTTP 5XX status code, which may for example occur if the PoINT Archival Gateway cannot be reached due to a network error. In this case, the system will try again every four hours.
With the help of the PoINT Archival Gateway, Cloudian customers can setup tape-based backups of their saved object data. Cross-region replication can be set up in just a few steps.