TDM Transparent Data Movement (TDM) is a patent pending technology developed by RackTop to enable the seamless movement of data between storage within a BrickStor SP to external storage locations including other BrickStor SP nodes, third party NFS capable storage and S3 compliant object storage. TDM is an advanced hierarchical storage management feature of the BrickStor SP operating system that provides security and compliance to data stored in the cloud or on third party storage systems. Users can continue to access data through the same client protocols using the original file path and do not need to change their workflow. File Chunking BrickStor SP’s TDM feature intelligently chunks large files into smaller objects when transferred to an object store. This means when a large file is updated that has been sent to an object store, only the chunks with modifications must be updated in the object store. If the file was stored as one large object, then the entire file would have to be retrieved and rewritten as an object. This unique feature of BrickStor SP saves bandwidth and speed of file access and cost when using a cloud-based object store. Demand Cache BrickStor SP’s intelligent demand cache optimizer reduces cost and improves performance by reducing IO for remote files on the economic tier. When a file is tiered with TDM, it is not actually removed from the primary storage tier, the “demand cache”, until the space needs to be reclaimed by the OS for other files. This means if a user opens a file before it is evicted from the demand cache, the file will be opened from the primary tier’s demand cache. This eliminates any latency from the economic tier and costs that may be imposed from a cloud provider for IO and data retrieval. The version of the file on the economic tier will be updated if there are any modifications made to the file. Local Space Reclamation There are three ways that TDM will handle the local data once it has been uploaded to the remote location: None - Always keep local copy (mirror) Dynamic - Keep local copy until space is needed. Immediate - Remove local copy immediately after upload. In all cases, the data will appear to all clients of TDM-enabled shares to be local. When a request for a file is made, any necessary chunks will be downloaded from the remote location and the requested file will be returned to the client. Configuring TDM TDM can be configured using either a S3 bucket or a NFS server as a remote data location. NFS To enable TDM on a dataset using a NFS server as a remote data location, follow the steps below. Root or Read/Write permissions for the BrickStor system must be configured on the NFS server. Navigate to the Datasets page. Select the desired dataset from the list. This will navigate to the Dataset Settings page of that dataset. Click the More dropdown, located at the top of the page. Select Enable TDM. Select the desired Local Space Reclamation option. Select New NFS for the location. Enter the NFS Server and Path. Enter the confirmation text. Click Enable. S3 To enable TDM on a dataset using a S3 bucket as a remote data location, follow the steps below. Navigate to the Datasets page. Select the desired dataset from the list. This will navigate to the Dataset Settings page of that dataset. Click the More dropdown, located at the top of the page. Select Enable TDM. Select the desired Local Space Reclamation option. Select New S3 for the location. Select the S3 provider. If it is not listed, choose Custom. (Custom) Enter the Endpoint of the S3 provider. (Custom) Enter the Region of the S3 provider. Enter the name of the S3 Bucket. Enter the Access Key and Secret Key. Enter the confirmation text. Click Enable.