How to Perform Common S3 Tasks Using the AWS CLI

  • by
AWS S3 CLI

If you’re anything like me, you run S3 commands sparsely enough that you find yourself parsing through AWS’ documentation every couple months to figure out how to complete your task. This page should speed up that process by providing an easy to reference guide anytime you have something you need to do with S3.

Note: This guide does require you to have the AWS CLI installed and configured.

How to list AWS S3 buckets

To list buckets available in S3 run the follow command.

$ aws s3 ls
2020-07-05 15:36:03 bucket-1
2021-02-21 17:15:33 bucket-2
2021-07-15 10:27:00 bucket-3

How to list objects in S3 buckets

To list the objects within a bucket we just need to add the --recursive flag and specify the bucket we’d like to see the objects for.

$ aws s3 ls --recursive bucket-2
2021-02-15 16:23:54        156  dir1/test1.txt
2021-02-15 14:11:34        156  dir1/test2.txt
2021-02-21 17:28:20       1390 dir2/test3.txt
2021-02-21 17:22:05        157  dir2/test4.txt

How to download a file from an S3 bucket

To download an S3 object we can use the S3 cp argument and first specify the path to the bucket and then the path where we’d like to download the file locally.

$ aws s3 cp s3://bucket-1/dir1/test1.txt /tmp/
download: s3://bucket-1/dir1/test1.txt to /tmp/test1.txt

How to download all files in an S3 bucket

If we want more than just one file we can add the --recursive flag to get everything in the bucket. As you run this command you will want to specify a new directory (i.e. /tmp/bucket-1) so that your files get dumped to the same top level directory.

$ aws s3 cp --recursive s3://bucket-1 /tmp/bucket-1
download: s3://bucket-1/dir1/test1.txt to /tmp/bucket-1/dir1/test1.txt
download: s3://bucket-1/dir1/test2.txt to /tmp/bucket-1/dir1/test2.txt
download: s3://bucket-1/dir2/test3.txt to /tmp/bucket-1/dir2/test3.txt
download: s3://bucket-1/dir2/test4.txt to /tmp/bucket-1/dir2/test4.txt

How to upload a file to an S3 bucket

To upload a single file to S3 we use the cp argument and specify first the local file we want to upload followed by the S3 bucket. If you want to upload your file to a specific directory in an S3 bucket simply add the path after the bucket’s name.

$ aws s3 cp test5.txt s3://bucket-2
upload: ./test5.txt to s3://bucket2/test5.txt

$ aws s3 cp test6.txt s3://bucket-2/dir1/
upload: ./test6.txt to s3://bucket2/dir1/test6.txt

How to upload a directory to an S3 bucket

To upload an entire local directory to an S3 bucket we need to add the --recursive flag and specify the path to the local directory we’d like to upload followed by the S3 bucket.

$ aws s3 cp --recursive /tmp/dir_test1 s3://bucket-3
upload: dir_test1/test1.txt to s3://bucket-3/test1.txt
upload: dir_test1/test2.txt to s3://bucket-3/test2.txt
upload: dir_test1/test3.txt to s3://bucket-3/test3.txt

How to delete a file in an S3 bucket

To a delete a file we can use the rm argument and specify the path to the file we’d like to delete in S3.

$ aws s3 rm s3://bucket-3/test1.txt
delete: s3://bucket-3/test1.txt

How to delete an S3 Bucket

If you are absolutely sure you want to delete an S3 bucket and all objects within it you can run the following command below.

$ aws s3 rb s3://bucket-name --force  

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *