Cloud Pipeline - Manual

• 1. Quick start
• 2. Getting started
• 3. Overview
• 4. Manage Folder
  • 4.1. Create an object in Folder
  • 4.2. Rename folder
  • 4.3. Delete Folder
  • 4.4. Clone a folder
  • 4.5. Lock a folder
• 5. Manage Metadata
  • 5.1. Add/Delete metadata items
  • 5.2. Upload metadata
  • 5.3. Customize view of the entity instance table
  • 5.4. Launch a run configuration on metadata
  • 5.5. Download data from external resources to the cloud data storage
• 6. Manage Pipeline
  • 6.1. Create and configure pipeline
  • 6.1.1. Building WDL pipeline with graphical PipelineBuilder
  • 6.2. Launch a pipeline
  • 6.3. Delete and unregister pipeline
  • 6.4. Work with AWS HealthOmics Workflow
• 7. Manage Detached configuration
  • 7.1. Create and customize Detached configuration
  • 7.2. Launch Detached Configuration
  • 7.3. Expansion Expressions
  • 7.4. Remove Detached configuration
• 8. Manage Data Storage
  • 8.1. Create and edit storage
  • 8.2. Upload/Download data
  • 8.3. Create and Edit text files
  • 8.4. Control File versions
  • 8.5. Delete and unregister Data Storage
  • 8.6. Delete Files and Folders from Storage
  • 8.7. Create shared file system
  • 8.8. Data sharing
  • 8.9. Mapping storages
  • 8.10. Storage lifecycle management
  • 8.11. Sensitive storages
  • 8.12. Cloud Data application
  • 8.13. Versioned storages
  • 8.14. Omics storages
• 9. Manage Cluster nodes
  • 9.1. Manage hot node pools
• 10. Manage Tools
  • 10.1. Add/Edit a Docker registry
  • 10.2. Add/Edit a Tool group
  • 10.3. Add a Tool
  • 10.4. Edit a Tool
  • 10.5. Launch a Tool
  • 10.6. Tool security check
  • 10.7. Tool version menu
  • 10.8. "Symlinked" tools
  • 10.9. Run capabilities
• 11. Manage Runs
  • 11.1. Manage runs lifecycles
  • 11.2. Auto-commit Docker image
  • 11.3. Sharing with other users or groups of users
  • 11.4. Automatic labels and actions for the runs
  • 11.5. Archive completed runs
• 12. Manage Settings
  • 12.1. Add a new system event
  • 12.2. Edit a system event
  • 12.3. Create a new user
  • 12.4. Edit/delete a user
  • 12.5. Create a group
  • 12.6. Edit a group/role
  • 12.7. Delete a group
  • 12.8. Change a set of roles/groups for a user
  • 12.9. Change email notification
  • 12.10. Manage system-level settings
  • 12.11. Advanced features
  • 12.12. System logs
  • 12.13. System dictionaries
  • 12.14. NAT gateway
• 13. Permissions
• 14. Command-line interface (CLI)
  • 14.1. Install and setup CLI
  • 14.2. View and manage Attributes via CLI
  • 14.3. Manage Storage via CLI
  • 14.4. View pipeline definitions via CLI
  • 14.5. Manage pipeline executions via CLI
  • 14.6. View cluster nodes via CLI
  • 14.7. View and manage Permissions via CLI
  • 14.8. View tools definitions via CLI
  • 14.9. User management via CLI
  • 14.10. SSH tunnel to the running compute instance
• 15. Interactive services
  • 15.1. Starting an Interactive application
  • 15.2. Using Terminal access
  • 15.3. Expose node filesystem
  • 15.4. Interactive service examples
• 16. Issues
• 17. CP objects tagging by additional attributes
  • 17.1. Faceted filters search using tags
• 18. Home page
• 19. Global search
• Appendix A. Instance and Docker container lifecycles
• Appendix B. Working with a Project
• Appendix C. Working with autoscaled cluster runs
• Appendix D. Costs management
• Appendix E. Pipeline objects concept
• Appendix F. Сomparison of using different FS storage types

1. Quick start
This chapter will give you a basic knowledge of pipeline running procedure.

2. Getting started
Find Thesaurus, list of supported browsers and authentication details.

3. Overview
Get familiar with a Cloud Pipeline Graphical User Interface (GUI).

4. Manage Folder
Learn how you can create your own hierarchical structured space.

5. Manage Metadata
Learn how you can create a complex analysis environment using custom data entities.

6. Manage Pipeline
Get details on pipeline creation, configuration, and launching. Get descriptions of "Launch pipeline" page. There is also a description of graphical "Pipeline builder" for WDL pipelines.

7. Manage Detached configuration
Learn how to run a cluster of differently configurated instances. "Launch cluster" box vs LaunchofClusterconfiguration.

8. Manage Data Storage
Create a new storage and learn how to manage it: download and upload, copy, move data, delete and unregister storages.

9. Manage Cluster Nodes
Learn about active nodes, how to terminate and refresh them. Get familiar with a related dashboard.

10. Manage Tools
Add and edit Docker registry. Learn how to add, modify and launch Tools.

11. Manage Runs
Learn about Runs space and what information you can get from it.

12. Manage Settings
Here you can find information about: how to install CLI, add a new system event, managing roles.

13. Permissions
Learn how to manage permissions for Cloud Pipeline objects.

14. Command-line interface (CLI)
Get familiar with a Cloud Pipeline Command Line Interface (CLI).

15. Interactive services
Learn how to setup non-batch application in a cloud infrastructure and access it via a web interface.

16. Issues
Learn how to share results with other users or get feedback.

17. CP objects tagging by additional attributes
Learn how to manage custom sets of "key-values" attributes for data storage and files.

18. Home page
Get details about homepage widgets, how to configure homepage view.

19. Global search How to search objects over the Platform.

Appendix A. Instance and Docker container lifecycles
Learn basics about instance and Docker container lifecycle.

Appendix B. Working with a Project
Learn basics about working with a Project.

Appendix C. Working with autoscaled cluster runs
Learn basics about working with autoscaled cluster runs.

Appendix D. Costs management
Get details about costs management concept.

Appendix E. Pipeline objects concept
Get details about the Pipeline objects concept in Cloud Pipeline environment.

Appendix F. Сomparison of using different FS storage types
Get details about the comparison of using different FS storage types (FSx for Lustre / EFS in AWS / BTRFS on EBS / LizardFS on EBS) in Cloud Pipeline environment.