Mount AWS S3 Bucket on CentOS EC2 Instance

Purpose

This article gives the steps to mount an Amazon Web Service Simple Storage Service on CentOS or RHEL 6 over a VPC Endpoint.

Create IAM User

Create an IAM user that will be used to access S3 from your EC2 instances.

Login to AWS web console
Open IAM | Users
Select Create New User
Enter a user name
1. i.e. s3user
Copy generated access keys
Set a complicated/long password for the user

OR You can use an IAM Role then include the role in the s3fs mount command

iam_role (default is no role)

Create S3 Bucket

Login to AWS web console
Select Services | S3
Create a S3 bucket
1. mywebapp-uploads
Add bucket policy to allow IAM User and VPC Endpoint access to all files inside the bucket.

{
	"Version": "2012-10-17",
	"Id": "wiki-uploads",
	"Statement": [
		{
			"Sid": "allow-iam-user-readwrite",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::1234567890:user/s3user"
			},
			"Action": [
				"s3:PutObject",
				"s3:GetObject",
				"s3:DeleteObject",
				"s3:DeleteObjectVersion",
				"s3:GetObjectVersion"
			],
			"Resource": "arn:aws:s3:::wiki-uploads/*"
		},
		{
			"Sid": "allow-vpce-read-bucket",
			"Effect": "Allow",
			"Principal": "*",
			"Action": [
				"s3:ListBucket",
				"s3:ListBucketVersions"
			],
			"Resource": "arn:aws:s3:::wiki-uploads",
			"Condition": {
				"StringEquals": {
					"aws:sourceVpce": "vpce-12345678"
				}
			}
		},
		{
			"Sid": "allow-vpce-read-content",
			"Effect": "Allow",
			"Principal": "*",
			"Action": [
				"s3:GetObject",
				"s3:GetObjectVersion"
			],
			"Resource": "arn:aws:s3:::wiki-uploads/*",
			"Condition": {
				"StringEquals": {
					"aws:sourceVpce": "vpce-12345678"
				}
			}
		}
	]
}

{

"Version": "2012-10-17",

"Id": "wiki-uploads",

"Statement": [

{

"Sid": "allow-iam-user-readwrite",

"Effect": "Allow",

"Principal": {

"AWS": "arn:aws:iam::1234567890:user/s3user"

"Action": [

"s3:PutObject",

"s3:GetObject",

"s3:DeleteObject",

"s3:DeleteObjectVersion",

"s3:GetObjectVersion"

"Resource": "arn:aws:s3:::wiki-uploads/*"

{

"Sid": "allow-vpce-read-bucket",

"Effect": "Allow",

"Principal": "*",

"Action": [

"s3:ListBucket",

"s3:ListBucketVersions"

"Resource": "arn:aws:s3:::wiki-uploads",

"Condition": {

"StringEquals": {

"aws:sourceVpce": "vpce-12345678"

}

{

"Sid": "allow-vpce-read-content",

"Effect": "Allow",

"Principal": "*",

"Action": [

"s3:GetObject",

"s3:GetObjectVersion"

"Resource": "arn:aws:s3:::wiki-uploads/*",

"Condition": {

"StringEquals": {

"aws:sourceVpce": "vpce-12345678"

}

]

}

Install Dependencies

CentOS/RHEL/Amazon

yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap automake fuse-devel git libcurl-devel libxml2-devel make

1	yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap automake fuse-devel git libcurl-devel libxml2-devel make

Ubuntu

sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config

1	sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev libfuse-dev libssl-dev libxml2-dev make pkg-config

Install Latest Fuse

s3fs requires fuse >= 2.8.4. Default yum repo on CentOS 6 at the time of writing this pulls down fuse-2.8.3-4. So you’ll want to uninstall the repo fuse version and install the latest fuse from source. Amazon Linux default is high enough version

Uninstall fuse

yum erase fuse

1

yum erase fuse
Download latest version
1. http://sourceforge.net/projects/fuse/files/fuse-2.X/
cd /usr/src/ wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.4/fuse-2.9.4.tar.gz

1
2

cd /usr/src/
wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.4/fuse-2.9.4.tar.gz
Extract tar

tar xzf fuse-2.9.4.tar.gz

1

tar xzf fuse-2.9.4.tar.gz
Change to the extracted directory

cd fuse-2.9.4

1

cd fuse-2.9.4
Configure package

./configure --prefix=/usr/local

1

./configure --prefix=/usr/local
Compile and Install package

make && make install export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ldconfig modprobe fuse

1
2
3
4

make && make install
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
ldconfig
modprobe fuse
Symlink fusermount to /usr/bin (Optional)

ln -s /usr/local/bin/fusermount /usr/bin/fusermount

1

ln -s /usr/local/bin/fusermount /usr/bin/fusermount

Install s3fs (Github)

Download s3fs
1. git clone https://github.com/s3fs-fuse/s3fs-fuse.git
Change to the extracted directory

cd s3fs-fuse

1

cd s3fs-fuse
Run Auto Generate Shell Script

./autogen.sh

1

./autogen.sh
Configure package

./configure

1

./configure
Compile and Install package

make && make install

1

make && make install
Symlink s3fs to /usr/bin (Optional)

ln -s /usr/local/bin/s3fs /usr/bin/s3fs

1

ln -s /usr/local/bin/s3fs /usr/bin/s3fs

Install s3fs (Older Method)

Download s3fs
1. http://tecadmin.net/mount-s3-bucket-centosrhel-ubuntu-using-s3fs/#
Extract tar

tar -zxvf s3fs-1.74.tar.gz

1

tar -zxvf s3fs-1.74.tar.gz
Change to the extracted directory

cd s3fs-1.74

1

cd s3fs-1.74
Configure package

./configure --prefix=/usr/local

1

./configure --prefix=/usr/local
Compile and Install package

make && make install

1

make && make install
Symlink s3fs to /usr/bin (Optional)

ln -s /usr/local/bin/s3fs /usr/bin/s3fs

1

ln -s /usr/local/bin/s3fs /usr/bin/s3fs

Install s3fs Libraries

Edit /etc/ld.so.conf

vim /etc/ld.so.conf

1

vim /etc/ld.so.conf
Add this line to the top:

/usr/local/lib/

1

/usr/local/lib/
Rerun ldconfig

ldconfig

1

ldconfig

Create Access Key File

Create a text file with the access keys.

echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > /etc/passwd-s3fs chmod 600 /etc/passwd-s3fs

1
2

echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > /etc/passwd-s3fs
chmod 600 /etc/passwd-s3fs

Setup Auto Mount (fstab)

mkdir /tmp/cache mkdir /mnt/s3mnt chmod 777 /tmp/cache /mnt

1
2
3

mkdir /tmp/cache
mkdir /mnt/s3mnt
chmod 777 /tmp/cache /mnt
Edit /etc/fstab

vim /etc/fstab

1

vim /etc/fstab
Add the following

s3fs#<bucket> /mnt/s3mnt fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0

1

s3fs#<bucket> /mnt/s3mnt fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0
Remount all in fstab

mount -a

1

mount -a

Mount AWS S3 Bucket on CentOS EC2 Instance

Purpose

Create IAM User

Create S3 Bucket

Install Dependencies

Install Latest Fuse

Install s3fs (Github)

Install s3fs (Older Method)

Install s3fs Libraries

Create Access Key File

Setup Auto Mount (fstab)

Sources

Tags

Recent Posts

S3 Data Consistency

Finding an AMI ID Using AWS CLI in AWS Marketplace

Understanding Linux Permissions

Compute and Networking Services

EC2 Essentials