# Transferring Data with AWS S3

> Source: https://parallelworks.com/docs/storage/transferring-data/aws-s3-buckets

# Transferring Data with AWS S3

This page explains how to transfer data to/from your AWS S3 buckets with a terminal. You can use the methods on this page for all S3 buckets, whether you created them on the ACTIVATE platform or outside the platform.

To transfer data to/from S3 bucket storage, you’ll use the AWS Command-Line Interface (CLI).

The AWS CLI is pre-installed on cloud clusters provisioned by ACTIVATE, so you can enter commands directly into the IDE after [logging in to the controller](/docs/compute/logging-in-controller) of an active AWS cluster.

If you’re transferring data between S3 buckets and your local machine or an on-premises cluster, you’ll likely need to install the AWS CLI first.

## Check for AWS CLI

Open a terminal or command line and navigate to your data’s destination. Enter `which aws`.

If the AWS CLI is installed, you’ll see a message that shows its location, such as `/usr/local/bin/aws`.

If the AWS CLI is not installed, you’ll see a message such as `/usr/bin/which: no aws` or `aws not found`.

## Install AWS CLI

If you need to install the AWS CLI, we recommend following [the AWS installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), which includes OS-specific instructions for Linux, macOS, and Windows as well as troubleshooting tips.

## Export Your AWS Credentials

You can see our page [**Obtaining Credentials**](/docs/storage/transferring-data/obtaining-credentials) for information on finding your AWS credentials.

In your terminal, enter `export AWS_ACCESS_KEY_ID="_____"` with your AWS access key ID in the blank space.

Enter `export AWS_SECRET_ACCESS_KEY="_____"` with your AWS access key ID in the blank space.

:::info Note

Please be sure to include the quotation marks on both ends of your key IDs. There are characters inside AWS key IDs that, without quotation marks, systems will try to read as commands.

:::

## List Files in S3

In your terminal, enter `aws s3 ls s3://bucket_name` to display the files in your bucket. For this guide, we used a small text file named `test.txt`, so our command returned this message:

```bash
demo@pw-user-demo:~/pw/cloud-data-transfer$ aws s3 ls s3://cloud-data-test
2023-02-08 16:37:56         28 test.txt
```

## Transfer a File To/From S3

The AWS CLI mimics the Linux `cp` command for transferring files. To transfer a file, enter `aws s3 SOURCE DESTINATION` in your terminal.

`SOURCE` and `DESTINATION` can be local files or S3 buckets. S3 buckets are formatted as `s3://bucket_name/file_name`.

Below is an example of the `aws s3` command.

In your terminal, enter `aws s3 cp s3://bucket_name/file_name ./` to download a file from S3 storage to your current directory. You’ll see this message:

```bash
demo@pw-user-demo:~/pw/cloud-data-transfer$ aws s3 cp s3://cloud-data-test/test.txt ./
download: s3://cloud-data-test/test.txt to ./test.txt
```

To download a file from S3 storage to a specific directory, enter its absolute or relative path (e.g., `/home/username/` or `./dir_relative_to_current_dir`) in place of `./` with the `aws s3 cp` command. In the example below, we used the `storage` directory, which is located inside the current working directory:

```bash
demo@pw-user-demo:~/pw/cloud-data-transfer$ aws s3 cp s3://cloud-data-test/test.txt ./storage
download: s3://cloud-data-test/test.txt to ./storage
```

To upload, simply reverse the order of `SOURCE` and `DESTINATION` in the `aws s3` command.

## Delete a File From S3

In your terminal, enter `aws s3 rm s3://bucket_name/file_name` to delete a file. You’ll see this message:

```bash
demo@pw-user-demo:~/pw/cloud-data-transfer$ aws s3 rm s3://cloud-data-test/test_upload.txt
delete: s3://cloud-data-test/test_upload.txt
```

## Further Reading

For a comprehensive guide to installing the AWS CLI in different environments, please see [this page](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) of the AWS documentation.

To configure your AWS CLI settings, please see [this page](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html).

For the full list of AWS CLI transfer commands, please see [this reference guide](https://awscli.amazonaws.com/v2/documentation/api/2.0.34/reference/transfer/index.html).