Object storage systems are undoubtedly designed for maximum availability and capable of surviving a wide range of failure scenarios – from hard drive defects and failed system nodes to an entire data centre going down.
But despite the high availability of such systems, it is important not to disregard data security. If an object is deleted, potentially including all of its versions, the deletion can take effect across the entire system in very little time.
In cases like this, replicating data on a separate, independent storage system can help mitigate against this scenario. However, a second hard drive-based object storage system would double the costs of storage. Tape libraries represent a much more attractive strategy from a cost perspective. The PoINT Archival Gateway is a software solution offering every advantage of S3 object storage, but holding data on tape instead of hard drives or SSDs.
AWS offers so-called cross-region replication (CRR) to hold data redundantly. This allows asynchronous replication and is configured from the source bucket. While AWS only allows replication within and between AWS regions, some object storage suppliers allow users to write data to external third-party target systems. Most implementations are comparable to CRR in the ways that matter most and differ only in terms of the details, such as the exact system requirements.
With StorageGRID, the object storage system from NetApp, this functionality is called CloudMirror. CloudMirror can be configured to replicate data to the PoINT Archival Gateway in just a few steps. As soon as NetApp StorageGRID receives a new object, the replication process is triggered. The PoINT Archival Gateway receives a copy of the new object via the S3 API and writes the object to tape media. The PoINT Archival Gateway uses erasure coding across multiple tapes to protect data.
A short overview of the required configuration steps follows:
1. A bucket must first be created within the PoINT Archival Gateway to use as a target for data replication. A user account, including S3 access key and S3 secret, must also be created.
2. The next step is to register with the StorageGRID Tenant Manager.
3. Create a new endpoint under the “S3” menu. Enter the PoINT Archival Gateway (PAG) as the target storage system and specify the FQDN and ports. An endpoint has a unique resource name (URN). The “s3” included represents replication. The target bucket will also be given in the URN.
4. Now go to the “Buckets” menu, select a bucket in StorageGRID and click “Configure Replication”.
The configuration file will then be generated in XML format. Enter the previously configured URN as the “Destination”. You can optionally use a prefix to filter data. Done!
You can find more details on CloudMirror in the NetApp Documentation Center.
From now on, all new objects added to the StorageGRID system will be transferred to the PoINT Archival Gateway and replicated on tape.
Replicated objects retain their original format. If an image named picture.jpg is replicated, for instance, it will keep the name picture.jpg in the target bucket and can be accessed easily from the PoINT Archival Gateway using an S3 browser.
If an object is now deleted from NetApp StorageGRID, the copy will remain on the PoINT Archival Gateway. The deletion process is not repeated on the PoINT Archival Gateway. If versioning is enabled on the target bucket, a “Delete” marker will be added, but the object itself will not be deleted.
We appreciate your feedback about the PoINT blog and this blog post. Please contact us at info@point-blog.de.