Select Page

S3 Essentials

S3 Definition

“Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any user access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The services aim to maximize benefits of scale and to pass those benefits on to users.”

An online, bulk storage service that you can access from almost any device.

S3 Basics

  • S3 = Simple Storage Service
  • it is AWS’s primary storage service.
  • You can store any type of file in S3.

Components and Structure


  • Root level “Folders” you create in S3 are referred to as buckets.
  • Any “subfolder” you create in a bucket is referred to as a folder.


  • Files stored in a bucket are referred to as objects.


  • When you create a bucket, you must select a specific region for it to exist. This means that any data you upload to the S3 bucket will be physically located in a data center in that region.
  • Best practice is to select the region that is physically closest to you, to reduce transfer latency.
  • If you are serving files to a customer based in a certain area of the world, create the bucket in a region closest to your customers (to reduce latency for your customers).

Some AWS services only work with/communicate with each other if they are in the same AWS region.


Free Tier use is available for S3.

How are you charged for using S3?

  • Storage Cost:
    • Applies to data at rest in S3
    • Charged per GB used
    • Price per GB varies based on region and storage class
  • Request Pricing – moving data in/out of S3:
    • PUT
    • COPY
    • POST
    • LIST
    • GET
    • Lifecycle Transitions Request
    • Data Retrieval
    • Data Archive
    • Data Restore

Buckets & Objects

Creating an S3 Bucket:

Bucket Naming Rules

Uploading (Import) an Objects to a Bucket:

Creating a Folder in a Bucket:

Navigating S3 Properties:

Bucket Level Properties:

  • General Info
  • Permissions
  • Static Web Hosting
  • Logging
  • Events
  • Versioning
  • Lifecycle
  • Cross-Region Replication
  • Tags
  • Requester Pays
  • Transfer Acceleration

Folder Level Properties:

  • General Info
  • Details

Object Level Properties:

  • General Info
  • Details
  • Permissions
  • MetaData

Storage Classes

What is a Storage Class?

  1. A storage class represents the ” classification” assigned to each Object in S3.
    • Available storage classes include:
      • Standard
      • Reduced Redundancy Storage (RRS)
      • Infrequent Access (S3-IA)
      • Glacier
  2. Each storage class has varying attributes that dictate things like:
    • Storage cost
    • Object availability
    • Object durability
    • Frequent of access (to the object)
  3. Each Object must be assigned a storage class (“standard” is the default class)
  4. You can change the storage class of an object at any time (for the most part).

Description of Each Storage Class


  • Designed for general, all-purpose storage.
  • Is the default storage option
  • 99.999999999% object durability (“eleven nines”).
  • 99.99% object availability.
  • Is the most expensive storage class.

Reduced Redundancy Storage (RRS)

  • Designed for non-critical, reproducible objects.
  • 99.99% object durability.
  • 99.99% object availability.
  • Is less expensive than the Standard storage class.

Infrequent Access (S3-IA)

  • Designed for objects that you do not frequently, but must be immediately available when accessed.
  • 99.999999999% object durability.
  • 99.90% object availability.
  • Is less expensive than the Standard/RRS storage class.


  • Designed for long-term archival storage.
  • May take several hours for objects stored in Glacier to be retrieved.
  • 99.999999999% object durability.
  • Is the cheapest S3 storage class (very low cost)

Object Durability and Availability

Object Durability

Is the percent (%) over a one year time period that a file stored in S3 will NOT be lost.

For object durability of 99.999999999% (11 nines) that means is there is a 0.000000001% chance of a file in S3 being lost in a year.

If you have 10,000 files stored in S3 (@ 11 nines durability), then you can expect to lose one file 10 million years.

Object Availability

Is the percent (%) over a one year time period that a file stored in S3 WILL be assessable.

For object availability of 99.99% – that means is there is a 0.01% chance that you won’t be able to access a file stored in S3 in a year.

For every 10,000 hours, you can expect a total of one hour for which a file may not be available to access.

How to Change Between Storage Classes

    1. By default, all new objects uploaded to S3 are set to the Standard storage class
    2. If you want new objects to have a different storage class, then you need to set the proper settings prior to or during the upload process. You can do this by either:
      • selecting another storage class during the upload process (“set details”)
      • Using object Lifecycle Policies
    3. For the following storage classes:
        • Standard
        • Reduced Redundancy Storage (RRS)
        • Infrequent Access (S3-IA)

      You can manually switch the objects storage class amongst them (at any time) by changing the storage class in the objects “properties”.

    4.  To move an object to the Glacier storage class:
      • You need to use object Lifecycles.
      • The change to Glacier may take 1-2 days to take effect.

Object Lifecycles

What is an object lifecycle?

An object lifecycle is a set of rules that automate the migration of an object’s storage classes to a different storage class (or deletion), based on specified time intervals.

Using Object Lifecycles


  • I have a work file that I am going to access every day for the next 30 days.
  • After 30 days, I may only need to access that file once a week for the 60 next days.
  • After which (90 days total) I will probably never access the file again but want to keep it just in case.

By using a life cycle policy, I can automate the process of changing files storage clss to meet my usage needs AND keep my S3 storage cost as low a possible.

What is the best solution to meet usage needs and minimize storage cost?


Day 0-29 (30 days):

  1. Usage needs = Very frequent
  2. “Best fit” Storage class = Standard
  3. Cost = highest cost tier

Day 30-89 (60 days):

  1. Usage needs = Infrequently
  2. “Best fit” Storage class = Infrequent Access
  3. Cost = middle cost tier

Day 90+:

  1. Usage needs = Most likely never needed
  2. “Best fit” Storage class = Glacier
  3. Cost = lower cost tier

Lifecycle Management

  • Lifecycles functionality is located on the bucket level.
  • However, a lifecycle policy can be applied to:
    • The entire bucket (applied all the objects in the bucket)
    • One specific folder within a bucket (applied to all the objects in that folder).
    • One specific object within a bucket
  • You can always delete a lifecycle policy or manually change the storage class back to whatever you like.


What are S3 Permissions?

S3 permissions are what allow you to have granular control over who can view, access and use specific buckets and objects.

Settings and Using Permissions

  1. Permission functionality can be found on the bucket AND object level.
  2. On the bucket level you can control (for each bucket individually):
      • List: Who can see the bucket name.
      • Upload/Delete: Objects to (upload) or in the bucket (delete).
      • View Permissions
      • Edit Permissions: add/edit/delete permissions

    Bucket level permission are generally used for “internal” access control

  3. On the Object level, you can control: (for each object individually)
      • Open/Download
      • View Permissions
      • Edit Permissions

    You can share specific objects (via a link) with anyone in the world.

Making an Object Publicity Available

    1. On the object, create the following permission:
      • Grantee = Everyone
      • “Check” Open/Download
    2. Under ” Actions”, select “Make Public
    3. The “Link” under Properties is now live and anyone that has it can directly download the object.

To remove public access to the object, either delete the permission


What is S3 Versioning?

S3 versioning is a feature that keeps track of and stores all old/new versions of an object so that you can access and use an older version if you like.

Versioning “Rules”

  • Versioning is either ON or OFF.
  • Once it is turned ON, you can only “suspend” versioning“. it cannot be fully turned OFF.
  • Suspending versioning only prevents versioning going forward. All previous objects with versions will still maintain their older versions.
  • Versioning ca only be set on the bucket level applies to ALL objects in the bucket.


S3 Data Consistency

  • Buckets in all Regions have:
    • Read-after-write consistency for PUTS of new objects
    • Eventual consistency for overwrite PUTS and DELETES
  • Read-after-write consistency lets you retrieve objects immediately after creating them
  • Eventual consistency may return old data
Eventually Consistent ReadConsistent Read
Stale reads possibleNo stale reads
Lowest read latencyPotential higher read latency
Highest read throughputPotential lower read throughput

Mount AWS S3 Bucket on CentOS EC2 Instance


This article gives the steps to mount an Amazon Web Service Simple Storage Service on CentOS or RHEL 6 over a VPC Endpoint.

Create IAM User

Create an IAM user that will be used to access S3 from your EC2 instances.

  1. Login to AWS web console
  2. Open IAM | Users
  3. Select Create New User
  4. Enter a user name
    1. i.e. s3user
  5. Copy generated access keys
  6. Set a complicated/long password for the user

OR You can use an IAM Role then include the role in the s3fs mount command

iam_role (default is no role)

Create S3 Bucket

  1. Login to AWS web console
  2. Select Services | S3
  3. Create a S3 bucket
    1. mywebapp-uploads
  4. Add bucket policy to allow IAM User and VPC Endpoint access to all files inside the bucket.

Install Dependencies



Install Latest Fuse

s3fs requires fuse >= 2.8.4. Default yum repo on CentOS 6 at the time of writing this pulls down fuse-2.8.3-4. So you’ll want to uninstall the repo fuse version and install the latest fuse from source. Amazon Linux default is high enough version

  1. Uninstall fuse
  2. Download latest version
  3. Extract tar
  4. Change to the extracted directory
  5. Configure package
  6. Compile and Install package
  7. Symlink fusermount to /usr/bin (Optional)

Install s3fs (Github)

  1. Download s3fs
    1. git clone
  2. Change to the extracted directory
  3. Run Auto Generate Shell Script
  4. Configure package
  5. Compile and Install package
  6. Symlink s3fs to /usr/bin (Optional)

Install s3fs (Older Method)

  1. Download s3fs
  2. Extract tar
  3. Change to the extracted directory
  4. Configure package
  5. Compile and Install package
  6. Symlink s3fs to /usr/bin (Optional)

Install s3fs Libraries

  1. Edit /etc/
  2. Add this line to the top:
  3. Rerun ldconfig

Create Access Key File

Create a text file with the access keys.

Setup Auto Mount (fstab)

  1. Edit /etc/fstab
  2. Add the following
  3. Remount all in fstab