Skip to main content

Working With Cloud Snapshots

Cloud snapshots let you make persistent changes to the operating system image running on your cluster nodes. By using an image provided by Parallel Works as a base, you can stage automations, install additional software, or enable additional services.

Creating Cloud Snapshots​

Navigate to your account settings (Username > Account).

Screenshot of user selecting Account from the User dropdown menu.

On the next page, click Cloud Snapshots.

Screenshot of user selecting Cloud Snapshots in their Account settings.

In Cloud Snapshots, click New Snapshot.

Screenshot of user selecting New Snapshot in the Cloud Snapshots tab.

Snapshot Configuration Settings​

There are several configurable parameters for cloud snapshots, which are outlined below.

Screenshot of blank cloud snapshot settings.

Type​

Use this dropdown menu to select whether your snapshot will be built for AWS, Azure, or Google.

Cloud Account​

Use this dropdown menu to select which cloud account will provision your snapshot. For most users, this menu should be left as the default option, unless you're a member of multiple organizations.

Group​

Use this dropdown menu to select the group name that your organization uses to allocate costs. This menu is especially important if your organization uses multiple groups.

If you’re not sure which group to select, you can contact us or your organization’s PW platform administrator.

Snapshot Region​

Use this dropdown menu to select the region your snapshot will be provisioned in.

If you run in multiple regions, you will want to provision the snapshot wherever you run your clusters.

Base Image​

Use this dropdown menu to select the PW image that will be the base for your snapshot. Typically, this menu will be populated with the suggested version by default.

Root Disk Size (GB)​

Use this field to enter the size of the snapshot's root disk. By default, this field is set to 100. Typically, you won't have to change this value unless you run into capacity issues.

Name​

Use this field to enter the name of your snapshot as it will be listed on the PW platform.

Description​

Use this field to enter a description to provide additional details about your snapshot.

Snapshot Build Script​

You can enter a script in Snapshot Build Script, which will run to make additions to the base PW image. You can use this script to install additional packages from yum repositories, clone GitHub repositories, or download files from a URL. An example build script is provided below.

#!/bin/bash

# Get useful environment info
echo "my username is $USER"
echo "my home dir is $HOME"
echo "my workdir is $PWD"

# Clone a public GitHub repository
git clone https://github.com/hpc/ior.git /usr/local/src/ior;

# Install Anaconda and have it initialize on login
wget https://repo.anaconda.com/archive/Anaconda3-2023.03-Linux-x86_64.sh;
sh ./Anaconda3-2023.03-Linux-x86_64.sh -b -p /usr/local/anaconda3;

# Symlink the Conda executable to the main /usr/local/bin directory
ln -s /usr/local/anaconda3/bin/conda /usr/local/bin/conda;

# Install Jupyter
conda install -y jupyter;

# Packages will be installed under the Anaconda root directory, so symlink what you need
ln -s /usr/local/anaconda3/bin/jupyter-lab /usr/local/bin/jupyter-lab;
ln -s /usr/local/anaconda3/bin/jupyter /usr/local/bin/jupyter;

# Install R and R Studio Server
# Specify the R version you want
export R_VERSION=4.1.3;

# Download and install R packages
curl -O https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm;
yum -y install R-${R_VERSION}-1-1.x86_64.rpm; # NOTE: this package installs R to /opt/R/4.1.3/bin/R

# Install R Studio packages
wget https://download2.rstudio.org/server/centos7/x86_64/rstudio-server-rhel-2023.03.0-386-x86_64.rpm;
yum -y install rstudio-server-rhel-2023.03.0-386-x86_64.rpm;

This build script will:

  • print some basic environment information.
  • clone the public IOR repository from GitHub into /usr/local/src/ior.
  • download and run the Conda installer.
  • download and install R and R Studio Server.
  • link executables to a common $PATH location.

Once you’ve provided your build script, click Create Snapshot to save the configuration.

Provisioning Cloud Snapshots​

After clicking Create Snapshot, the page will update with a Provisioning Log and two new buttons: Save Snapshot Config and Provision Snapshot.

Screenshot of cloud snapshot page after clicking Create New Snapshot.

Click Save Snapshot Config when making configuration changes that you want to save, including changes to the main parameters.

Click Provision Snapshot to start building the snapshot image. Doing so will replace the Snapshot Build Script with the Provisioning Log, where you can monitor the snapshot build.

The snapshot build process works by creating a temporary virtual machine (VM) that runs the PW base image. Then, the provided script runs to make additions to the image, which is saved as a separate snapshot.

When the snapshot is done building, the Provisioning Log will show a message identifying the name of the image in the CSP account. If there were errors in the build that prevented it from completing, those will also be visible in the log.

If your build is successful, the Provisioning Log will show a message such as:

Packer build completed successfully
{"snapshot_id":"ami-0a5854d925e6e558b", "snapshot_name":"pw-Demo-hello", "csp":"aws", "region":"us-east-1" }
https://parallel.works/api/v2/machineimages

If your build is unsuccessful, the Provisioning Log will show a message such as:

Build 'amazon-ebs.aws' errored after 1 minute 57 seconds: Script exited with non-zero exit status: 1. Allowed exit codes are: [0]
==> Wait completed after 1 minute 57 seconds
==> Some builds didn't complete successfully and had errors:
--> amazon-ebs.aws: Script exited with non-zero exit status: 1. Allowed exit codes are: [0]
==> Builds finished but no artifacts were created.
Packer build failed
Note

If you reprovision an existing snapshot and the new build has errors, your previously working snapshot will not be overwritten.

Using Cloud Snapshots​

Snapshots you have created for any CSP will automatically be added to your cluster configuration settings. You can see user-created snapshots on a cluster's configuration page by selecting the Image in Controller Settings and the Elastic Image in Partition Settings.

You can also configure a different snapshot for your controller and each of your partitions.

Deleting Cloud Snapshots​

To delete a custom snapshot, navigate to its configuration page and click Deprovision Snapshot.

A dialog box will appear with the message Are you sure you want to deprovision the snapshot test?

Click Deprovision snapshot. The Provisioning Log will then display the deletion process:

Starting delete of custom image...

CLOUD: aws
REGION: us-east-1
PROJECT: ca-cloudmgmt
IMAGE: pw-Demo-hello
AMI:  ami-023b3386461fdfad6
SnapshotId: snap-01d59ca427982c1d8
aws ec2 deregister-image --image-id ami-023b3386461fdfad6
aws ec2 delete-snapshot --snapshot-id snap-01d59ca427982c1d8

Delete completed...

If you use the method above, you will still have your snapshot listed in Cloud Snapshots (Username > Account > Cloud Snapshots), and you will be able to reconfigure and reprovision the snapshot at any time.


Alternatively, you can delete a snapshot from the list in Cloud Snapshots. If you click the delete icon, a dialog box will appear with the message Are you sure you want to delete the snapshot [snapshot name]?

Clicking Delete snapshot will permanently remove the snapshot from your account and from the list in Cloud Snapshots.

About Deletion

If you want to delete a snapshot from the list in Cloud Snapshots, you first need to deprovision the snapshot on its configuration page. If you don't, the snapshot will still exist in your CSP account.

Last updated on by Parallel Works