AWS Cloud Storage

This post is continuation to my previous post about AWS introduction. I will try to explain each service that is described in the previous post in detail. In this post, I will discuss about storage cloud service of Amazon web services.

Amazon Simple Storage Service (S3)

A service that provides highly scalable and highly durable object storage in cloud. Here object storage means the architecture that manages data as objects, as opposed to other storage architectures like file systems and block storage. You can store and retrieve any amount of data, at any time from any where on the web through a simple web service interface.

S3 storage classes/tiers

• S3 - standard: For general purpose storage of frequently accessed data. Provides 99.99 availability, 11 9's durability designed to sustain the loss of 2 facilities concurrently.
• S3 - IA (Infrequent Access): For data that is accessed less frequently but requies rapid access when needed. Lower fee and retrieval fee - stored in multiple availability zones.
• S3 one zone IA : Lower cost than IA but stored in one AZ only.
• Glacier: For low cost archival data. Glacier provides three options for retrieval. Expedited ( retrieval in minutes), standard (retrieval in 3-5 hrs), bulk (retrieval in 5-12 hours).

S3 is for

• To store distributed files, static web content and media. S3 can server as an origin store for content delivery network (CDN), such as Amazon cloudfront. Eg: photo and video sharing sites.
• S3 can be used to host entire static websites.
• Backup and archival of data.

S3 is not for

• S3 is not meant for file system.
• Storing databases.
• Dynamic website hosting.

Pricing

You pay for the storage that you actually use. No minimum fee and no setup cost. S3 has three pricing components: storage (per GB per month), data transfer in or out (per GB per month) and requests (per thousand requests per month). For data transfer, there is transfer acceleration option which provides better performance than regular S3 transfer.

Cross Region Replication

Cross region replication allows objects to be copied across s3 buckets in different AWS regions asynchronously providing disaster recovery solutions for business continuity. 

Cross region replication needs versioning to be enabled on both buckets. You can change the class of the replication (S3 in source and S3-IA in destination). 

• Only new objects or the modified objects will be replicated (old ones will not be).
• Delete will replicate the delete marker.
• Deleting the delete marker will not be replicated.
• Deleting version of a file will not be replicated.
• Cannot replicate to multiple buckets.

Encryption
You can choose to encrypt the data using SSE-S3, SSE-C, SSE-KMS, or a client library. All four enable you to store sensitive data encrypted at rest in Amazon S3.
SSE-S3 provides a solution where amazon handles key management and key protection using multiple layers of security.
SSE-C enables you to leverage Amazon S3 to perform the encryption and decryption of your objects while retaining control of the keys used to encrypt the objects. 
SE-KMS enables you to use AWS Key Management Service to manage your encryption keys.
Client Library: Using an encryption client library, such as the amazon s3 encryption client, you retain control of the keys and complete the encryption and decryption of objects client-side using an encryption library of your choice.

Event Notifications
Amazon S3 event notifications can be sent in response to actions in amazon s3 like PUTs, POSTs, COPYs or DELETEs. Notificaiton messages can be sent through either Amazon SNS, Amason SQS or to AWS Lambda. Amazon S3 event notifications enable you to run workflows, send alerts, or perform other actions in response to changes in your objects stored in S3.

Points to remember

• S3 is object based.
• Simple key/value store (key - name of the file , value- data)
• Files can be from 0 bytes to 5Tb.
• Virtually unlimited storage.
• Files stored in buckets (like folder in cloud).
• Bucket name use universal namespace (must be unique globally).
• Read after write consistency for new objects.
• Eventual consistency for overwrite puts and deletes (some time to propagate and reflect).
• Large objects can be uploaded using multipart upload.
• Supports versioning. Versioning can only be enabled, it cannot be disabled (can be suspended).
• Normal delete will create the delete marker (which marks the object is deleted, the actual object will still be there).
• If you delete the delete marker then it will restore the object.
• Each object can be accessed through an unique http URL.
• You have to select a region while creating bucket.
• Transfer Acceleration utilises cloudfront to accelerate your uploads to S3. you upload to edge location and then transfer that file to S3. yo will get a distinct url to upload to.
• There are four mechanisms to control access to S3 resources: IAM (Identity and Access Management) policies, Access control lists, bucket policies and query string authentication.
• Amazon S3 Block Public Access is a new set of security controls that allows customers to enforce that S3 buckets and objects do not have public access. This setting overrides all other access level setting.
• To retrieve Amazon S3 data stored in Amazon Glacier, initiate a retrieval request using the Amazon S3 APIs or the Amazon S3 Management Console. The retrieval request creates a temporary copy of your data in the S3 RRS or S3 Standard-IA storage class while leaving the archived data intact in Amazon Glacier. You can specify the amount of time in days for which the temporary copy is stored in S3. You can then access your temporary copy from S3 through an Amazon S3 GET request on the archived object.
• Using IPv6 support for Amazon S3, applications can connect to Amazon S3 without the need for any IPv6 to IPv4 translation software or systems. You can get started by pointing your application to Amazon S3’s new “dual-stack” endpoint, which supports access over both IPv4 and IPv6.