The High Throughput Computing Storage (HTC Storage) service has been designed for researchers with a need for large amounts of long term storage for High Throughput Computing (HTC) with cost as a primary consideration.
Example Use Case
I have a large amount of research data that I may want to quickly analyze on Lewis at a later date. Instead of constantly moving my data between sources, it would be nice to be able to have access to cost effective storage.
The storage is only internally accessible on the Lewis cluster compute nodes and externally accessible on the Lewis login nodes and data transfer (DTN) nodes via rsync over ssh or sftp.
Storage is limited to DCL1 and DCL2 research data. Administrative, commercial, and personal data is prohibited.
Storage is allocated in blocks and each storage block has its own quota mount point.
Storage blocks are prepaid in full in 10TB increments for 5 years for. The FY21 rate is $160/TB. Initial investment must be 50TB or greater.
Storage nodes are expanded in large increments (100TB). Depending on available capacity, requests for storage blocks may be delayed until a storage node is expanded.
The storage blocks expire after 5 years. Users must either purchase another storage block and transfer data to the new storage node; transfer data to another system; or do nothing and the data will be destroyed. Users may request a hardship waiver to the CI Council for temporary storage. No automatic data migration services are provided and users are responsible for the data integrity of moved data.
Storage block may be expanded depending on the available capacity of the storage node. The storage block will retain the same expiry date as the original block and cost of the expansion will be pro-rated.
Storage is based on ZFS and provided on a single node and single volume basis with no high availability (HA).
There are no backups of the data. The data storage system is resilient to multiple disk failures (parity) and we do our best to protect the data but are not responsible for any data loss. Snapshots are available and count towards storage block.
Storage is transferable but is not refundable.
Groups located in HTC (/storage/htc/group_name) are considered flat and all users located within a specific group have the same permissions. The PI for the group is the only person who can approve additions and removals of users in groups.
*New Investments must be at least 50TB. Existing investments must be upgraded in multiples of 10TB.