Cloud Pipeline Introduction
Why Cloud Pipeline
Cloud Pipeline solution from EPAM provides an easy and scalable approach to perform a wide range of analysis tasks in the cloud environment.
This solution takes the best of two approaches: classic HPC solutions (based on GridEngine schedulers family) and SaaS cloud solutions.
|Feature||Classic "HPC"||"Cloud Pipeline"||SaaS|
|Pipelines customization||High. Direct scripting - any level of customization||High. Direct scripting - any level of customization||Low. Provide specific pipeline definition languages or even only a graphical editor|
|Backward compatibility||N/A||High. "Classic" HPC scripts and NGS tools can be run without any changes||Low. Scripts have to be rewritten according to the supported languages and storage structures|
|User Interface||Command Line||Graphical Interface for User interaction and a Command Line Interface for automation scripts||Graphical Interface|
|Calculation power scalability||Low. New nodes shall be deployed and supported on-premises. Idle nodes are still consuming resources||High. New nodes are started according to the job request and terminated as soon as they are not needed anymore. Each job can precisely define required CPU/RAM/Disk resources or even select optimal node up to speed up execution (e.g. memory optimized nodes for cellranger pipelines)||High. Scalable as "Cloud Pipeline" but sometimes limits user to predefined nodes setup|
|Deployment and vendor-lock||Deployed on-premises and introduces no vendor-lock||Can be deployed in AWS/GCP/Azure or on-premises, thus introduces no vendor-lock||Consumed as an Internet service (no on-premises deployment available), all processes are tied to this specific vendor|
|Security||High. All data and analysis processes are located in a controlled network.||High. All data and analysis processes are located in a controlled cloud VPC. All security configurations are performed by user's security officers||Low. No direct control over security configuration. SaaS vendor has full access to the data storages.|
The main components of the Cloud Pipeline are shown below: