8. Manage Data Storage

Data storage is a Cloud Pipeline object that represents cloud storage and its content in a folder hierarchy.

Permissions management for a storage is described here.

"Details view" lists content of the storage: files that may be organized into folders. Clicking on the inside folder will open its content in the "Details view".
Note: storage's folders hierarchy will not be represented in the "Hierarchy view" panel.
Note: you can move Storage to a new parent folder using drag and drop approach.

In "Library tree view" and "Details view" data storages are tagged with region flag to visually distinguish storage locations.
CP_ManageDataStorage

Note: if a specific platform deployment has a number of Cloud Providers registered (e.g. AWS+Azure, GCP+Azure) - in that case auxiliary Cloud Provider icons are additionally displayed, e.g.:
CP_ManageDataStorage

Figure 1
CP_ManageDataStorage

Another option for navigation in the storage is to use "breadcrumbs" control at the top of the "Details" view (see picture above, 1):

  • Clicking an item will navigate to that folder.
    CP_ManageDataStorage
  • Editing a path will allow to copy/paste a path and navigate to any custom location.
    CP_ManageDataStorage

Controls

At the top of the "Details" view there are buttons:

Select page

Clicking this control (see picture Figure 1 above, item 2), the whole file and folders on the current page will be selected. It allows to perform bulk operations like deleting.

Show file versions

This feature is available not for all Cloud Providers. Currently, it is supported by AWS and GCP.

Tick this checkbox (see picture Figure 1 above, item 3) and the view of a page will changed: the all file versions will be displayed. You can expand each version's list by clicking "+" in desired line.
Note: the last version will be marked by "(latest)".

Remove all selected

This is a bulk operation control. It is visible, if at least one of the data storage item (folder or file) is selected.
CP_ManageDataStorage

Generate URL

This control helps to generate URLs for a number of files and then download them manually one by one or via scripts. See details here.
Note: the control is available, if only files are selected.

CP_ManageDataStorage

Show attributes/Hide attributes

Allows see or edit a list of key=value attributes of the data storage (see picture Figure 1 above, item 4).
Note: If selected storage has any defined attribute, Attributes pane is shown by default. See 17. CP objects tagging by additional attributes.

"Gear" icon

Allows to edit the path, alias, description of the storage, manage its STS and LTS durations and enable versions control (see picture Figure 1 above, item 5). The delete option is also here (if storage contains only metadata, it will be deleted anyway). See 8.1. Create and edit storage.

Refresh

Allows updating representation of storage's contents (see picture Figure 1 above, item 6).

+ Create

You can also create new folders and files via this button (see picture Figure 1 above, item 7). See 8.3. Create and Edit text files.

Upload

This control allows uploading files to the storage (see picture Figure 1 above, item 8). See 8.2. Upload/Download data.

Each-line controls

CP_ManageDataStorage

Control Description
Download This control calls downloading of selected file.
Edit Helps to rename a file or a folder.
Delete Delete a file or a folder.

View and edit a text file

You can view and edit text files. For more details see here.
CP_ManageDataStorage

CLI Storage options

There are also several options that are only implemented in CLI but not in GUI:

  • To move files and folders from one storage to another or between local file system and storage.
  • To copy files from one storage to another.

See here for more details.