Posts Tagged 'Amazon Storage Services'

Amazon S3 Storage: A Flexible Cloud Storage Solution

Amazon S3 storage is one of the AWS services that has been around the longest and is one of the most widely used. For those unfamiliar with S3, it enables individual data items, ranging in size from one byte to five terabytes to be stored in scalable storage. There is no limit to the number of items, referred to as objects, that can be stored in S3. Amazon have recently published the usage statistics for the end of Q1 2012 for this service. At this time there are 905 billion objects stored in S3 with Amazon routinely handling 650,000 requests per second for access to these objects. The number of objects is growing daily by more than one million. This is an incredible amount of data.

These results motivated me to write about the different ways in which users access S3 and the kind of ways they are using it. For those unfamiliar with S3, then storage buckets, which are URL addressable entities are created in S3 and the data objects placed in these. A simple usage scenario is storing Web application assets in S3. For example, a typical Web application has many images and handling the delivery of these images to remote clients, even when cached, places an unnecessary load on the applications servers. A cost-effective way to offload this work is to place the images on S3. Since these are URL addressable, the HTML pages can link to the images here. This means as pages are requested, the images are delivered from S3 storage, freeing the servers the Web application runs on from this workload.

Another way that S3 is being used is via the AWS Storage Gateway. This service, still in beta mode, enables a cost effective, secure, off-premise data backup and disaster recovery mechanism. On-premise storage appliances connect to the storage gateway, providing a secure communication channel to S3 storage. Data stored using the Storage Gateway is automatically encrypted. For disaster recovery, point-in-time snapshots can be taken of on-premise data and sent to S3 providing secure and immediate access to backups should the need arise.

What is interesting is the number of different ways Amazon AWS provides for accessing S3. The simplest is via the Web management console, but this is only really practical for small amounts of data. Other ways include a Web Services API, command line tools, AWS Import/Export developed for large data sets, the above mentioned storage gateway, direct connect pipes for performance as well as many third party backup services.

With a range of security settings on S3 buckets, controlling who has read/write/update access, S3 offers an incredibly safe and cost effective scalable storage solution. If you have a storage requirement that can be off-premise, needs to be low latency yet scale and be secure, I urge you to seriously consider Amazon S3. In my experience over the last two years its a fantastic service. The usage statistics are strong evidence that I am not the only one who thinks so.

For related course information check out Learning Tree’s course entitled Cloud Computing with Amazon Web Services™.

Chris Czarnecki

Deleting Attached Amazon EC2 EBS Volumes

When teaching Learning Tree’s Cloud Computing Course, I demonstrate various aspects of Amazons Infrastructure as a Service (IaaS). As part of this, not only do I provision various machine types but also associated Elastic Block Storage (EBS) devices and attach and detach these. Since I use a demonstration account for this, one task I undertake at the end of the course is to make sure that all resources are released/removed so that no unnecessary costs are incurred. For the demonstrations, I always use the Amazon Web browser administration interface.

On a recent teach, I tidied up the account – or more accurately thought I had. When the monthly bill arrived, charges were still being incurred, albeit minor. The charges were for an EBS volume, which I thought would be straightforward to delete. However, when I tried to delete this from the browser administration interface, I received an error saying that the volume was in use and could not be deleted and should be detached using the force flag. This is a feature not available from the browser interface. Equally there was no other resource running on my account that the volume could be attached too ! I was paying for a fault in the browser interface and how resources had not been cleaned up properly by this toolset.

So what was the solution ? To detach the volume I used the EC2 command line tools. If you are not sure of these Kevin Kell has written a post on how to install these. The command to detach the volume is then simply:

$ ec2-detach-volume volume-id -force

The force flag is important here. Once this has completed the volume could be deleted using the command:

ec2-delete-volume volume-id

Hopefully you will not encounter this scenario, but if you do, you now know the solution.


Cloud Computing Services and Alternatives

It’s common to categorize cloud computing as either Software as a Service (SaaS), Platform as a Service (PaaS) or Infrastructure as a Service (IaaS).  When we were moving to the cloud, it seemed more helpful to think in terms of the individual services and applications that we needed to provide our users.  Below is a list of the services we moved, or considered moving, to the cloud and the choices we evaluated.

Email Services
Managing an Exchange or other mail server is expensive, considering Google will do it for next to nothing. Google Apps for Business provides 50 mail accounts using your domain name for free; beyond 50 accounts, Google will charge $50 per user per year.  At a similar cost, Microsoft offers Exchange Online.

Document Sharing and Collaboration
SharePoint is a fantastic program and a huge success for Microsoft.  You might not have to install your own instance though.  Microsoft will host SharePoint for you for around $5 per user per month. Check out SharePoint Online to see how it works.  Google offers services similar to SharePoint using Google Calendars, Google Groups, and Google Sites.

Office Applications
If you choose Google for document sharing and collaboration, you might also like Google Docs for documents, spreadsheets, and presentations.  If you prefer Microsoft Office, check out Microsoft Office Live, which offers online storage and online versions of Word, Excel and PowerPoint similar to Google Docs.

Customer Relationship Management (CRM) began as CRM in the cloud and has expanded from there.  There are also hosted versions of Microsoft Dynamics CRM and SugarCRMIn the end, we chose to host SugarCRM Community Edition on our own virtual server using Amazon EC2.  See my prior post, Saving Money Using the Cloud and Open Source Software.

Database Services
If you need a relational database and like SQL Server, check out SQL Azure.  You can have a database up and running in seconds.  It will cost you about $10 per gigabyte per month.  Amazon Relational Database Service is an alternative based on MySQL, and costs as little as $500 per year.

If you need massive amounts of disk space, Microsoft, Google and Amazon all have cloud offerings.  Check out, Windows Azure Storage, Amazon Elastic Block Store, Amazon Simple Storage Service, and Google App Engine.

Application Hosting
If you’re a .NET, PHP or Java developer, Microsoft Windows Azure provides a complete infrastructure for deploying your applications.  Azure is a massively scalable, zero-administration platform based on Windows Server 2008.  It provides Web hosting and much more, including storage, load-balancing, elastic scalability, authentication services and integration with you local network.  Google App Engine offers services similar to Azure and includes APIs for Python and Java developers.  It’s even free up to a certain number of requests per month.

Sometimes it’s more cost-effective to deploy an application on your own server.  That doesn’t mean you have to buy a physical machine though.  Amazon EC2 allows you to spin up Linux or Windows virtual machines in minutes.  The cost of a Windows server is as little as $109 per year and you don’t need any hardware.

If you want to learn more about cloud computing, come to one of Learning Tree’s cloud-computing courses.


Amazon AWS New Storage Features

The rate at which Amazon has been adding to and improving its cloud computing services is impressive. If we just consider some of the announcements made to the storage services in June 2010. Firstly consider the Relational Database Service (RDS). This is now available in all AWS regions.The RDS can now be managed from the AWS management console, allowing MySQL instances to be launched, take real-time snapshots of instances and monitor key database statistics. A superb feature now available is the ability to create a MySQL instance that is synchronously replicated across availability zones to provide enhanced data protection and availability due to both planned and un-planned outages.

Another new feature is the Amazon S3 management console. This enables the management of S3 resources from a simple console with tasks such as bucket creation and data object upload available from the console. Offsite backup and disaster recovery planning have also been enhanced with the addition of the Amazon Import/Export feature. Data can easily be imported and exported from/to portable storage devices into S3 storage.

Another new storage related feature is the Reduced Redundancy Storage(RRS) facility just announced. RRS allows customers to store their non-critical data at lower levels of redundancy and therefore at a lower cost to S3 storage. Both S3 and RRS store data in multiple facilities and on multiple devices. S3 is designed to provide 99.999999999% durability. RRS is designed to provide 99.99% durability and both are backed by Amazon’s S3 service level agreement.

There have been other enhancements to services such as CloudFront and MapReduce too and all this month. It is this continuous rapid development and enhancement of the AWS services that is convincing more and more organisations to move to using EC2. The rapid growth in the AWS business has lead to recent predictions that within 5 years, the Amazon AWS business will be larger than its e-commerce business. Such a rapid growth provides concrete evidence that cloud computing is not just the current computing buzzword but is a serious technology being rapidly adopted by a large number of organisations.

If you are interested in how Amazon AWS can be leveraged for your projects in a risk minimal manner, why not take a look at the following overview, or for a more detailed insight why not take the Learning Tree class – it will really fast track you into exploiting the benefits of the cloud.


Learning Tree Logo

Cloud Computing Training

Learning Tree offers over 210 IT training and Management courses, including Cloud Computing training.

Enter your e-mail address to follow this blog and receive notifications of new posts by e-mail.

Join 53 other followers

Follow Learning Tree on Twitter


Do you need a customized Cloud training solution delivered at your facility?

Last year Learning Tree held nearly 2,500 on-site training events worldwide. To find out more about hosting one at your location, click here for a free consultation.
Live, online training
.NET Blog

%d bloggers like this: