Beyond the block storage that powers servers, cloud providers offer powerful, standalone storage services. The most common and widely used storage service in AWS is Amazon S3 (Simple Storage Service). It is highly important for various profiles, including DevOps and web developers.
Amazon S3 is designed to store files, backups, media, and unstructured data in the form of objects. This article explores Amazon S3 and the file storage service, EFS.
Amazon S3 Architecture and Scale
Amazon S3 provides virtually infinite storage capacity. A single object (file) can be up to 5 TB in size.
- Buckets and Objects: Objects are stored in containers called buckets. The files stored inside are called objects. Bucket names must be globally unique, meaning no one else in the world can use the same name.
- Global Service, Regional Data: Amazon S3 is unique because it is a global service. It is not bounded to any specific region like EC2 or VPC. However, when you create a bucket, you must choose a region. This region determines the physical location where your data is stored.
- Durability: Amazon S3 is renowned for its durability, guaranteeing 99.99% (eleven 9s) of durability. This is achieved by automatically replicating data across multiple Availability Zones within the choosen region.
Amazon S3 Storage Classes and Cost Management
Amazon S3 allows you to optimize casts by matching your storage class to your data’s access frequency.
- S3 Standard: The default storage class, used for frequently accessed data. It provides low latency access.
- S3 Standard-Infrequent Access (Standard IA): For data accessed infrequently, but which still needs fast retrieval. It is suitable for a minimum storage duration of 30 days.
- S3 One Zone-Infrequent Access (One Zone IA): Data is stored in only one AZ. It is cheaper than Standard IA but less durable, as it is not replicated.
- S3 Glacier (Archival Classes): Used for long-term archival data. These offer the lowest storage costs, but retrieval takes longer.
- Glacier Instant Retrieval:Archives data while still allowing immediate access.
- Glacier Flexible Retrieval: Retrieval takes minutes to hours.
- Glacier Deep Archive: Offers the highest discount for data accessed less than once a year. Retrieval can take multiple hours.
Amazon S3 Lifecycle Rules
Lifecycle rules automate the movement of data between storage classes to save money. For example, a rule can transition objects from Standard to Standard-IA after 30 or 45 days. Rules can also be used for automation deletion (expiration) of objects.
Amazon S3 Features
- Security: Amazon s3 can be managed using Access Control Lists (ACLs) or Bucket Policies. ACLs are considered outdated. Bucket Policies, written in JSON code, are the recommended, modern approach. A single policy can define fine-grained access (Allow/Deny) for numerous objects and users.
- Versioning: It prevents accidental overwriting or deletion of data. When enabled, uploading a file with the same name create a new version. If you delete a file, S3 creates a “delete market” that hides the object, rather than permanently removing it.
- Replication: Cross-Region Replication (CRR) automatically copies data from a source bucket in one region to a target bucket in a different region. This is crucial for disaster recovery.
- Static Website Hosting: Amazon S3 can host static websites, which are website without a backend or database. This requires enabling public read access, often via a bucket policy.
Elastic File System (EFS): File Storage
Amazon EFS provides a managed, scalable, and elastic file storage solution.
- Shared Access: Unlike EBS, which connects to one instance at a time, EFS is a shared storage solution based on the Network File System (NFS) protocol. Multiple EC2 instances can access the same EFS data simultaneously.
- Elasticity: EFS is “elastic” because it eliminates the need to specify storage size in advance. You simply keep adding files, and the system scales automatically. You pay only for the data stored.
- Durability: EFS can be configured for regional availability, which provides high durability by storing data across multiple Availability Zones.
Conclusion: Unlocking Scalable Storage
You have now mastered the full spectrum of cloud storage. You know that Amazon S3 provides virtually unlimited, highly durable object storage, perfect for everything from backups to hosting static websites. Furthermore, you’ve learned how to save costs using Storage Classes and Lifecycle Rules, and how to protect data with Versioning. We also covered EFS as the ideal solution for shared file access across multiple servers.
Now that you have servers (EC2) and storage (EBS, S3), you need to connect them securely. In the next article, we will build our own secure, private network in the cloud using the AWS VPC (Virtual Private Cloud).
