If you’re anything like me, you run S3 commands sparsely enough that you find yourself parsing through AWS’ documentation every couple months to figure out how to complete your task. This page should speed up that process by providing an easy to reference guide anytime you have something you need to do with S3.
Note: This guide does require you to have the AWS CLI installed and configured.
How to list AWS S3 buckets
To list buckets available in S3 run the follow command.
$ aws s3 ls
2020-07-05 15:36:03 bucket-1
2021-02-21 17:15:33 bucket-2
2021-07-15 10:27:00 bucket-3How to list objects in S3 buckets
To list the objects within a bucket we just need to add the --recursive flag and specify the bucket we’d like to see the objects for.
$ aws s3 ls --recursive bucket-2
2021-02-15 16:23:54 156 dir1/test1.txt
2021-02-15 14:11:34 156 dir1/test2.txt
2021-02-21 17:28:20 1390 dir2/test3.txt
2021-02-21 17:22:05 157 dir2/test4.txtHow to download a file from an S3 bucket
To download an S3 object we can use the S3 cp argument and first specify the path to the bucket and then the path where we’d like to download the file locally.
$ aws s3 cp s3://bucket-1/dir1/test1.txt /tmp/
download: s3://bucket-1/dir1/test1.txt to /tmp/test1.txtHow to download all files in an S3 bucket
If we want more than just one file we can add the --recursive flag to get everything in the bucket. As you run this command you will want to specify a new directory (i.e. /tmp/bucket-1) so that your files get dumped to the same top level directory.
$ aws s3 cp --recursive s3://bucket-1 /tmp/bucket-1
download: s3://bucket-1/dir1/test1.txt to /tmp/bucket-1/dir1/test1.txt
download: s3://bucket-1/dir1/test2.txt to /tmp/bucket-1/dir1/test2.txt
download: s3://bucket-1/dir2/test3.txt to /tmp/bucket-1/dir2/test3.txt
download: s3://bucket-1/dir2/test4.txt to /tmp/bucket-1/dir2/test4.txtHow to upload a file to an S3 bucket
To upload a single file to S3 we use the cp argument and specify first the local file we want to upload followed by the S3 bucket. If you want to upload your file to a specific directory in an S3 bucket simply add the path after the bucket’s name.
$ aws s3 cp test5.txt s3://bucket-2
upload: ./test5.txt to s3://bucket2/test5.txt
$ aws s3 cp test6.txt s3://bucket-2/dir1/
upload: ./test6.txt to s3://bucket2/dir1/test6.txtHow to upload a directory to an S3 bucket
To upload an entire local directory to an S3 bucket we need to add the --recursive flag and specify the path to the local directory we’d like to upload followed by the S3 bucket.
$ aws s3 cp --recursive /tmp/dir_test1 s3://bucket-3
upload: dir_test1/test1.txt to s3://bucket-3/test1.txt
upload: dir_test1/test2.txt to s3://bucket-3/test2.txt
upload: dir_test1/test3.txt to s3://bucket-3/test3.txtHow to delete a file in an S3 bucket
To a delete a file we can use the rm argument and specify the path to the file we’d like to delete in S3.
$ aws s3 rm s3://bucket-3/test1.txt
delete: s3://bucket-3/test1.txtHow to delete an S3 Bucket
If you are absolutely sure you want to delete an S3 bucket and all objects within it you can run the following command below.
$ aws s3 rb s3://bucket-name --force