AWS: How to Mount S3 Bucket on EC2 Linux Instance Using IAM Role

AWS: How to Mount S3 Bucket on EC2 Linux Instance Using IAM Role (Complete Guide)

Mounting an Amazon S3 bucket directly onto an EC2 Linux instance transforms cloud storage into a local filesystem. This approach eliminates the need for manual S3 API calls or CLI tools like aws s3 cp, letting you use familiar Unix commands (cp, mv, ls) on what appears to be a regular directory. For DevOps teams, this means seamless integration of S3 into existing workflows—whether for log aggregation, media processing, or shared configuration storage—without the cost of EFS or the complexity of NFS.

In this tutorial, I’ll walk you through mounting an S3 bucket on an EC2 instance using s3fs-fuse and an IAM role (the most secure method). You’ll learn how to compile s3fs from source, configure IAM permissions precisely, and troubleshoot common issues like permission errors or mount failures. By the end, you’ll have a production-ready setup that scales with your infrastructure.

Prerequisites

• A running EC2 instance (Amazon Linux 2, CentOS 7/8, or RHEL 7/8) with root/sudo access.
• An existing S3 bucket (e.g., s3fs-demobucket).
• Basic familiarity with AWS IAM, EC2, and Linux command line.
• Outbound internet access on the EC2 instance (to install packages and clone Git repos).

Step 1: Update the System

Start by updating your instance’s package index and installed packages to avoid dependency conflicts:

sudo yum update -y

This ensures all security patches and package dependencies are current. Skip this step only if you’re working in a tightly controlled environment where updates are managed separately.

Step 2: Install Required Dependencies

s3fs-fuse relies on several libraries and tools. Install them with:

sudo yum install -y automake fuse fuse-devel gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

Key packages explained:

• fuse: The Filesystem in Userspace (FUSE) kernel module, which allows non-root users to create filesystems.
• fuse-devel: Header files for developing FUSE-based applications.
• gcc-c++: C++ compiler for building s3fs from source.
• openssl-devel: Required for HTTPS/TLS support when communicating with S3.

If you’re using Ubuntu/Debian, replace yum with apt and adjust package names (e.g., libfuse-dev instead of fuse-devel).

Step 3: Clone and Compile s3fs-fuse

Clone the s3fs-fuse repository and compile it from source:

git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure --prefix=/usr --with-openssl
make
sudo make install

What’s happening here:

• autogen.sh: Generates the build configuration scripts.
• configure: Sets up the build environment (e.g., enabling OpenSSL for encryption).
• make: Compiles the source code.
• make install: Installs the s3fs binary to /usr/bin.

Verify the installation by checking the binary’s location:

which s3fs

Expected output: /usr/bin/s3fs.

Step 4: Create an IAM Role for S3 Access

Using an IAM role is the most secure way to grant your EC2 instance access to S3. Unlike hardcoding AWS credentials, roles are temporary and automatically rotated by AWS.

4.1 Create the IAM Policy

Navigate to the IAM Console and create a new policy with the following JSON:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:ListAllMyBuckets"
      ],
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::s3fs-demobucket"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::s3fs-demobucket/*"]
    }
  ]
}

Policy breakdown:

• s3:GetBucketLocation: Required for s3fs to determine the bucket’s region.
• s3:ListBucket: Allows listing objects in the bucket (e.g., ls commands).
• s3:GetObject/PutObject/DeleteObject: Enables read/write/delete operations on objects.

Critical note: Replace s3fs-demobucket with your bucket name. The trailing /* in the third statement is essential—it grants access to objects inside the bucket, not the bucket itself.

4.2 Create the IAM Role

1. In the IAM Console, go to Roles > Create role.
2. Select

   AWS service

   > EC2 > Next: Permissions.
3. Attach the policy you created in Step 4.1.
4. Name the role (e.g., s3fsmountingrole) and create it.

4.3 Attach the Role to Your EC2 Instance

1. Go to the EC2 Console and select your instance.
2. Click Actions > Security > Modify IAM role.
3. Select the role (s3fsmountingrole) and save.

Pro tip: If you’re launching a new instance, attach the role during creation under Configure Instance Details > IAM role.

Step 5: Create a Mount Point

Choose or create a directory where the S3 bucket will be mounted. For example:

sudo mkdir -p /var/s3fs-demofs

Ensure the directory has the correct permissions. If non-root users need access, run:

sudo chown ec2-user:ec2-user /var/s3fs-demofs

(Replace ec2-user with your username, e.g., centos for CentOS.)

Step 6: Mount the S3 Bucket

Use the following command to mount the bucket:

s3fs s3fs-demobucket /var/s3fs-demofs \
  -o iam_role="s3fsmountingrole" \
  -o url="https://s3-eu-central-1.amazonaws.com" \
  -o endpoint=eu-central-1 \
  -o dbglevel=info \
  -o curldbg \
  -o allow_other \
  -o use_cache=/tmp

Flag explanations:

Flag	Purpose
-o iam_role	Uses the IAM role attached to the instance (instead of AWS credentials).
-o url	Specifies the S3 endpoint URL (replace eu-central-1 with your bucket’s region).
-o endpoint	Sets the AWS region (e.g., us-east-1, ap-southeast-2).
-o dbglevel=info	Enables verbose logging (useful for troubleshooting).
-o curldbg	Logs cURL debug output (for diagnosing connection issues).
-o allow_other	Allows non-root users to access the mounted filesystem.
-o use_cache=/tmp	Caches frequently accessed files locally to improve performance.

Important: If your bucket is in a different region, update the url and endpoint flags. For example, for us-east-1, use:

-o url="https://s3.us-east-1.amazonaws.com" -o endpoint=us-east-1

Step 7: Verify the Mount

Check if the bucket is mounted successfully:

df -h

Expected output (your bucket name and size will differ):

Filesystem      Size  Used Avail Use% Mounted on
s3fs            256T     0  256T   0% /var/s3fs-demofs

Test basic operations:

# Create a test file
echo "Hello, S3!" > /var/s3fs-demofs/test.txt

# List files
ls -l /var/s3fs-demofs/

# Verify the file exists in S3
aws s3 ls s3://s3fs-demobucket/

If the file appears in both the mount point and S3, the setup is working.

Common Pitfalls and Fixes

1. Permission Denied Errors

Symptom: ls: cannot open directory /var/s3fs-demofs: Permission denied.

Causes and fixes:

• IAM role not attached: Verify the role is attached to the instance in the EC2 Console.
• Incorrect policy: Ensure the IAM policy includes s3:ListBucket and s3:GetObject for the bucket.

• Mount point permissions:

  Run sudo chmod 755 /var/s3fs-demofs to allow non-root access.
• Missing allow_other flag: Add -o allow_other to the mount command.

2. Mount Fails Silently

Symptom: No error message, but df -h doesn’t show the mount.

Debugging steps:

• Check logs: journalctl -u s3fs -f (for systemd) or tail -f /var/log/messages.
• Enable debug mode: Add -o dbglevel=info -o curldbg to the mount command.
• Verify FUSE is loaded: lsmod | grep fuse. If missing, load it with sudo modprobe fuse.

3. Slow Performance

Symptom: High latency when accessing files.

Optimizations:

• Use -o use_cache=/tmp to cache frequently accessed files locally.
• Increase the cache size: -o use_cache=/tmp -o cache_size_mb=1024 (1GB cache).
• Disable SSL (not recommended for production): -o use_path_request_style -o url=http://s3.amazonaws.com.

4. Objects Not Appearing in S3

Symptom: Files created in the mount point don’t appear in the S3 bucket.

Causes:

• ACL permissions: In the S3 Console, go to your bucket > Permissions > Bucket Policy and ensure the IAM role has s3:PutObject permissions.
• Caching delay: s3fs caches writes. Force a sync with echo 1 | sudo tee /proc/sys/vm/drop_caches.

How to Automate the Mount on Reboot

To persist the mount across reboots, add an entry to /etc/fstab:

s3fs#s3fs-demobucket /var/s3fs-demofs fuse _netdev,iam_role=s3fsmountingrole,url=https://s3-eu-central-1.amazonaws.com,endpoint=eu-central-1,allow_other,use_cache=/tmp 0 0

Key options:

• _netdev: Delays mounting until the network is available.
• 0 0: Disables fsck checks (not applicable to s3fs).

Test the fstab entry with:

sudo mount -a

If no errors appear, the mount will persist after reboot.

Key Takeaways

• Security first: Always use IAM roles instead of hardcoding AWS credentials. Roles are temporary, automatically rotated, and scoped to specific resources.
• Precision in IAM policies: The policy must include s3:ListBucket for the bucket and s3:GetObject/PutObject for objects (/*).
• Debugging is your friend: Use -o dbglevel=info -o curldbg to diagnose mount failures. Logs are your best tool when things go wrong.
• Performance trade-offs: s3fs is not a replacement for EFS or EBS. It’s best for infrequently accessed files or write-once-read-many workloads. Use caching (use_cache) to mitigate latency.
• Automate persistence: Add the mount to /etc/fstab to ensure it survives reboots, but test with mount -a first.

FAQ

Can I mount multiple S3 buckets on the same instance?

Yes. Create separate mount points (e.g., /var/s3fs-bucket1, /var/s3fs-bucket2) and run the s3fs command for each bucket. Ensure the IAM role has permissions for all buckets.

How do I unmount an S3 bucket?

Use the standard umount command:

sudo umount /var/s3fs-demofs

If the filesystem is busy, force unmount with:

sudo umount -l /var/s3fs-demofs


Is s3fs suitable for high-throughput applications?
No. s3fs is built on FUSE and S3’s REST API, which introduces latency. For high-throughput workloads (e.g., databases, video processing), use EFS or EBS. s3fs is ideal for log storage, backups, or static assets.

How do I upgrade s3fs to the latest version?
Pull the latest changes from Git, recompile, and reinstall:
cd ~/s3fs-fuse
git pull
./autogen.sh
./configure --prefix=/usr --with-openssl
make
sudo make install
Unmount the bucket first, then remount it after upgrading.

Why does my mount disappear after a reboot?
If you didn’t add the mount to /etc/fstab, it won’t persist. Follow the automation steps above. Also, ensure the _netdev option is included to wait for network availability.
🛒 Recommended gear on Amazon
Disclosure: some links above are affiliate links — if you buy through them I may earn a small commission at no extra cost to you. Thanks for supporting the channel!