8.7. Create shared file system
User shall have ROLE_ADMIN to mount FS to the Cloud Pipeline. For more information see 13. Permissions.
A shared file system is a data storage based on network file system. It has several advantages over regular data storages and local file system:
- While regular data storage is a great option for a long-term storage, it cannot be used as a shared file system for high-performance computing jobs as it does not support network-like interface.
- A local disk cannot be shared across several nodes.
- A user needs to specify local disk size when scheduling a run. If a user specifies a size that is not enough to finish a job - it will fail. Cloud-based shared file system could be used to workaround this issue.
Create FS mount
- Navigate to a desired folder and click + Create → Storages → Create new FS mount.
Note: For FS mounts - "Add existing" option is not available.
Note: For the correct FS mount creation, at least one mount point shall be registered in the System Preferences for any Cloud Region. If no - the corresponding button of the FS mount creation becomes invisible:
- Specify Storage path and other optional parameters.
Note: Storage path parameter contains FS mount path and the name of the storage to be created.
Note: FS storages are just subdirectories of the mounted FS. One FS mount can have multiple FS storages. - When deleted from GUI, FS storage is unmounted from the Cloud Pipeline.
FS storage features
- For FS storages GUI doesn't display the following features typical for regular data storages:
- STS
- LTS
- Versioning and Backup duration.
- When a user selects Input/Common/Output path parameter for a pipeline run - it is impossible to set FS storage:
- FS storages aren't displayed in the "Browse..." dialog for Input/Common/Output path parameters;
- Value of Input/Common/Output path parameters is validated so that user is not able to specify a path to FS storage manually.
- The content of files stored in FS data storage could be previewed as well as in regular data storages:
- Since FS isn't an object data storage, it isn't possible to add metadata tags to files in the FS storage.
- Use FS storage between cluster nodes. If pipeline Tools contain FS client, FS storage(s) will be mounted automatically.