AWS

AWS Command-Line Interface

Article by:

Date Published:	January 14, 2019
Last Modified:	November 21, 2019

Installation

1
2
3
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

S3

Creating A Bucket

1
$ aws s3 mb s3://<bucket_name> --region <region>

Listing Files

To list buckets:

1
$ aws s3 ls

Copying Files

1
$ aws s3 cp <source> <destination>

1
2
3
4
5
# Copy local file to S3
$ aws s3 cp my_file.txt s3://my_bucket/my_prefix

# Copy file from S3 to local
$ aws s3 cp s3://my_bucket/my_prefix my_file.txt

Syncing Files

Syncing can copy a group of files which all share a similar prefix (or part of a prefix), so you can treat the prefixes almost like directories in this case.

1
$ aws s3 sync <source> <destination>

1
2
# Sync all objects with the prefix "my_prefix/" to the current directory 
$ aws s3 sync s3://my_bucket/my_prefix/ ./

Speeding Up Copy And Sync Commands

You can speed up copy and sync transfers (especially when small files are involved) by running the following commands:

1
2
$ aws configure set default.s3.max_concurrent_requests 100
$ aws configure set default.s3.max_queue_size 10000

This modifies the default profile. It will add the below to the .aws/config file:

1
2
3
4
[default]
s3 =
    max_concurrent_requests = 100
    max_queue_size = 10000

Authors

Geoffrey Hunter

Dude making stuff.

Twitter

GitHub

This work is licensed under a Creative Commons Attribution 4.0 International License .