Object Storage#
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9's) of durability, and stores data for millions of applications for companies all around the world. (https://aws.amazon.com/s3/)
Accessing Amazon S3#
Amazon S3 is an AWS native solution, which requires AWS credentials to access the S3 API. Amazon S3 is not a deployment in your pilot network that you can access.
After you have initialized your AWS credentials, you can start accessing Amazon S3.
Amazon S3 buckets#
Amazon S3 is an object storage that stores objects up to 5 TB in buckets. In your pilot you can upload data to your S3 bucket(s) and structure them accordingly in folders and subfolders. You can upload as many objects as you want in Amazon S3, the storage capacity will automatically scale. Depending on the permissions of your user group you can upload data to S3 and/or download data from S3.
AWS CLI#
You can use the AWS CLI to perform actions on Amazon S3. Use the AWS CLI if you want access S3 via the command line interface in a terminal.
Examples
Listing S3 files with AWS CLI#
$ aws s3 ls s3://bucket-name
Sync local files with Amazon S3#
$ aws s3 sync . s3://bucket-name/path
Sync files on Amazon S3 with local files#
$ aws s3 sync s3://bucket-name/path .
AWS SDK#
You can use the AWS SDK to perform actions on Amazon S3. Use the AWS SDK to access S3 by using any of the supported programming languages.
Downloading files with AWS SDK for Python:#
import boto3
s3 = boto3.client('s3')
s3.download_file('BUCKET_NAME', 'OBJECT_NAME', 'FILE_NAME')