On August 28, 2024, AWS announced Parallel Computing Service (AWS PCS). Customers can build scientific and engineering models to quickly and easily set up and manage high-performance computing infrastructure to accelerate large-scale research and development. AWS PCS is designed for a variety of traditional and emerging compute- or data-intensive engineering and scientific workloads across disciplines such as computational fluid dynamics, weather modeling, finite element analysis, electronic design automation, and reservoir simulation, using a familiar way to prepare, run, and analyze simulations and calculations.
Marvel Fusion, Maxar, RONIN, and the National Renewable Energy Laboratory are some of the first customers and partners using AWS Parallel Computing Services.
SEATTLE–(BUSINESS WIRE)– Amazon Web ServicesAmazon.com, Inc. (AWS), an Amazon.com, Inc. (NASDAQ: AMZN) company, announced the general availability of AWS Parallel Computing Service. This new managed service helps customers easily set up and manage high performance computing (HPC) clusters, enabling them to run scientific and engineering workloads of virtually any scale on AWS. The service enables systems administrators to easily build clusters using Amazon Elastic Compute Cloud (Amazon EC2) instances, low-latency networking, and storage optimized for HPC workloads. AWS Parallel Computing Service enables scientists and engineers to rapidly scale simulations to validate models and designs. At the same time, systems administrators and integrators can build and maintain HPC clusters on AWS using Slurm, the most widely adopted open source HPC workload manager. The service accelerates innovation in areas such as accelerating drug discovery, discovering genomic insights, building engineering designs, running weather applications, and building scientific and engineering models.
“Managing HPC workloads, especially the most complex and challenging extreme-scale workloads, is extremely challenging. Our goal is that every scientist and engineer using AWS Parallel Computing Service, regardless of the size of their organization, will be equipped with the same best-in-class HPC capabilities as large enterprises to solve the world’s toughest challenges, whenever they need them, at any scale, so they can be the most productive in their field.”
Ian Colle, Director of Advanced Computing and Simulation, AWS
AWS has a history of innovation in supporting HPC workloads, including the release of the open source cluster orchestration toolkit AWS ParallelCluster, the fully managed batch computing service AWS Batch, the low latency network interconnect Elastic Fabric Adapter, Amazon FSx for Lustre high performance storage, and purpose-built AMD, Intel, and Graviton-based HPC compute instances, the latter of which deliver up to 65% better price performance over comparable compute-optimized x86-based instances.
In November 2018, AWS AWS Parallel ClusterAWS ParallelCluster is an AWS-enabled open source cluster management tool that helps deploy and manage HPC clusters in the AWS Cloud. AWS ParallelCluster enables customers to rapidly build and deploy proof-of-concept and production HPC computing environments. AWS ParallelCluster command line interface, API, Python library, and user interface installed from open source Packages, which are responsible for updates such as tearing down and redeploying the cluster.
As thousands of customers across a broad range of industries move their HPC workloads to AWS to accelerate drug discovery, discover genomic information, maximize energy resources, and spin up supercomputers with millions of cores, AWS continues to innovate in HPC by releasing comprehensive, fully managed HPC services that eliminate the undifferentiated heavy lifting of creating and managing HPC clusters.
AWS PCS streamlines the HPC environment managed by AWS, AWS Management ConsoleAWS SDKs, and AWS Command Line Interface (AWS CLI)System administrators can create managed Slurm clusters that use compute and storage configurations, identities, and job allocation settings. AWS PCS uses Slurm, a highly scalable, fault-tolerant job scheduler used by a wide range of HPC customers, to schedule and orchestrate simulations. End users, such as scientists, researchers, and engineers, can log into AWS PCS clusters to run and manage their HPC jobs, use interactive software on their virtual desktops, and access data. You can quickly migrate your workloads to AWS PCS without significant effort to port your code.
Also read: About Amazon Elastic Compute Cloud (EC2)
Get started with AWS Parallel Computing Service
To try AWS PCS, Creation tutorial A Simple Cluster from the AWS documentation. First, I created a Virtual Private Cloud (VPC) using an AWS CloudFormation template. Amazon Elastic File System (Amazon EFS) In your account in the AWS Region where you want to try AWS PCS. For more information, Create a VPC and Create shared storage It’s described in the AWS documentation.
What you need to know
There are a few things to know about this feature:
- Slurm Version – AWS PCS will initially support Slurm 23.11 and provide a mechanism designed to allow customers to upgrade major versions of Slurm as new versions are added. In addition, AWS PCS is designed to automatically update Slurm controllers with patch versions. For more information, see AWS PCS. Slurm Version It’s described in the AWS documentation.
- Capacity Reservation — On-Demand Capacity Reservations allow you to reserve EC2 capacity in a specific Availability Zone for a specific duration, ensuring you have the compute capacity you need, when you need it. For more information, Capacity Reservations It’s described in the AWS documentation.
- Network File Systems – You can connect a network storage volume where data and files can be written and accessed. Amazon FSx for NetApp ONTAP, Amazon FSx for OpenZFSand Amazon File Cache Similarly Amazon EFS and Lustre on Amazon FSxYou can also use self-managed volumes, such as NFS servers. For more information, see Network File Systems It’s described in the AWS documentation.
availability
AWS Parallel Computing Services It is currently available in the US East (N. Virginia), AWS US East (Ohio), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), EU (Frankfurt), EU (Ireland), and EU (Stockholm) regions.
AWS PCS launches all resources in your AWS account. You are charged for these resources. For more information, AWS PCS pricing page.
resource
https://aws.amazon.com/about-aws/whats-new/2024/08/aws-Parallel-computing-service
