Posts Tagged 'Azure'

Big Data on Azure – HDInsight

The HDInsight service on Azure has been in preview for some time. I have been anxious to start working with it as the idea of being able to leverage Hadoop using my favorite .NET programming language has a great appeal. Sadly I had never been able to successfully launch a cluster. Not, that is, until today. Perhaps I had not been patient enough in previous attempts, although on most tries I waited over an hour. Today, however, I was able to launch a cluster in the West US region that was up and running in about 15 minutes.

Once the cluster is running it can be managed through a web-based dashboard. It appears, however, that the dashboard will be eliminated in the future and that management will be done using PowerShell. I do hope that some kind of console interface remains but that may or may not be the case.

Figure 1. HDInsight Web-based dashboard

To make it easy to get started Microsoft provides some sample job flows. You can simply deploy any or all of these jobs to the provisioned cluster, execute the job and look at the output. All the necessary files to define the job flow and programming logic are supplied. These can also be downloaded and examined. I wanted to use a familiar language to write my mapper and reducer so I selected the C# sample. This is a simple word count job which is quite commonly used as an easily understood application of Map/Reduce. In this case the mapper and reducer are just simple C# console programs that read and write to stdin and stdout which are redirected to files or Azure Blob storage in the job flow.

Figure 2. Word count mapper and reducer C# code

One thing that is pretty cool about the Microsoft BI stack is that it is pretty straightforward to work with HDInsight output using the Microsoft BI Tools. For example the output from the job above can be consumed in Excel using the Power Query add-in.

Figure 3. Consuming HDInsight data in Excel using Power Query

That, however, is a discussion topic for another time!

If you are interested in learning more about Big Data, Cloud Computing or using Excel for Business Intelligence why not consider attending one of the new Learning Tree courses?

Kevin Kell

IaaS on Azure

Okay, this post may be a bit of a rant. Things should not be unnecessarily complicated … but they are.

I just wanted to launch a VM instance on Windows Azure Virtual Machines to take a look at the Visual Studio 2013 preview. Seems simple enough, right? Just log in to the Azure portal and spin up a new VM from the gallery from an image configured with VS 2013. Then, just step through the Wizard and connect with RDP.

Figure 1. Select VS 2013 Image

Give the machine a name, select instance size (small, in my case), and specify a username and password and we should be good to go.

Well, yes, and no.

Okay, the instance launches and appears to be running. As far as I know I am being charged for this resource now.

Figure 2. Instance is running

Cool. My instance seems to have a public IP address and I should be able to connect to it via RDP, login using the username and password I specified and party on. Just like Amazon EC2 and Rackspace.

But no! There is a problem.

Try and connect using RDP to actually use the instance and there is an error.

Figure 3. Denied!

So, what is going on here?

Right now I haven’t the slightest idea. Maybe it is something simple or obvious. Maybe they didn’t enable RDP on the image. Maybe I am doing something stupid.  Maybe I need to open a port or set a security group or something.  But … at this point I don’t care.  I don’t have the time or patience to troubleshoot it.


I’m going to spin up a Windows Sever 2012 instance on Amazon EC2, install the VS 2013 preview myself and take it from there. At least I know I will have no problem connecting to the instance.  Amazon IaaS is, imho, much more straightforward to use.

Kevin Kell

Windows Azure Marketplace DataMarket

As I prepare to teach Learning Tree’s Power Excel course in Rockville next week I have been taking a closer look at PowerPivot. Since the course now uses Excel 2013 we have expanded the coverage of PowerPivot which is now included with Excel. In 2010 it had been a separate add-in.

So what, you may ask, does that have to do with cloud computing? Well, as it turns out PowerPivot is really well suited to consume data that has been made available in the Windows Azure DataMarket.

The DataMarket is perhaps one of the less well known services offered as part of Windows Azure. In my opinion it has some growing to do before it reaches a critical mass. It has, however, made some impressive advancements since its inception in 2010. The DataMarket contains both free and paid-for data subscriptions that can be accessed using a variety of tools. Here I give a brief example of consuming a free subscription using PowerPivot.

The DataMarket does not appear anywhere on the Azure portal. To access it you need to create a separate account. You do that at . Once you have established an account you can subscribe to any of the various data that have been published. You can also subscribe and use the data from your browser but I found it very easy and intuitive to subscribe to the data right from within PowerPivot.

Figure 1. Consuming Azure DataMarket data using PowerPivot

I then chose to limit my selection to just the free subscriptions. In an actual application, of course, I would be able to search for data that was relevant to the analysis I was doing. For fun I decided to look at the USA 2011 Car crash data published by BigML. When I finish clicking through the wizard the data is imported into my PowerPivot Data Model and is available for my use. Here I can correlate it with other data I have to build up my analysis dataset.

Once the data is in PowerPivot I can quickly do analyses using familiar Excel tools. I can also use the reporting capabilities of Data View in Excel 2013 to create compelling presentations of the data.

Figure 2. Analysis of Car Crash data in Excel Power View

The easy integration between PowerPivot and the Azure DataMarket gives Excel users a powerful tool to augment their data analysis. In future posts I will explore some of the other services that Microsoft is offering through Azure to further enhance and simplify analysis of very large datasets.

Kevin Kell

Windows Azure Mobile Services

For some time now I have wanted to get started building mobile apps but I just haven’t been able to find the time. This week I decided to explore the Mobile Services in Windows Azure to see if that might offer a path of low resistance. I was pretty happy with what I found there.

Azure Mobile Services, which is currently in preview, is a collection of backend cloud services that support applications written for mobile devices. Included are services for storage/database, server side scripting, scheduled jobs, authentication, push notifications and others. Supported mobile platforms are Windows Phone/Windows Store, Android, iOS and HTML5.

Getting started with Mobile Services is very easy. From the Azure portal all you do is provision a new mobile service. You pick a name for the url endpoint, a geographic region to deploy into and specify to use an existing or a new SQL database for storage. Then, from the dashboard, you can either create a new mobile app or connect the service to an existing mobile app.

Figure 1. Creating a new Mobile App

Since I am creating a Windows Phone 8 app I needed to install the SDK. I did this right from within Visual Studio since I already had it installed. It was fairly painless with only a couple of warnings where I had to grant permissions, enable the Windows Hyper-V feature so I could run the emulator and turn on the Package Manager option in Visual Studio that allows NuGet to download missing packages during the build.

Create a Table provisions a SQL database that your mobile app can use for storage. Then, all you need to do is download and unzip a file which contains a Windows Phone 8 project which comes pre-configured to connect to the specific service you just created. Just open it in Visual Studio and it will run in the emulator.

Figure 2. Personalized Phone 8 project running in Emulator

This project is the starting point for the custom app you are creating. Back on the portal you will see that a database table has been automatically created dynamically from the code. Pretty cool.

Obviously there is still quite a bit of work ahead as you now build out your custom app. There are also other features and capabilities of the service and the API that need to be explored. Still, it is pretty easy to get started and get a working mobile app that uses the Azure Mobile Services as a back end put together with just a few mouse clicks.

Kevin Kell

Cloud Skills in Demand

According to a recent analysis by CyberCoders cloud computing talents will be the second most sought after IT skill in 2013. Interestingly the analysis specifically calls out AWS and Azure. That these two are included is not a surprise to those of us that have followed the evolution of cloud computing over the past few years. Conspicuously absent from the list, however, is Google but let’s not go there right now.

In the early days it seemed that Amazon and Microsoft took fundamentally different approaches to where they would operate in the cloud marketplace.  Amazon was primarily an IaaS play whereas Microsoft Azure was a PaaS. Today the distinction has blurred. Amazon has many offerings which are certainly more than just infrastructure and Microsoft now offers true virtual machines (including Linux!) as part of Azure. Obviously competitive pressures and market realities have caused this convergence.

It is my belief that Amazon is still the market leader in public cloud computing. Don’t count Microsoft out, though. Azure has many good technical merits and the new Web Sites service (which allows you to host up to 10 simple ASP.Net web applications for free) is totally awesome. I am also pretty happy with the SQL Reporting service as I previously have said.

When an IT skill is in demand there is a good possibility that someone will develop a certification program around it so that individuals claiming competence can be validated in some sense. This is now the case with Amazon AWS. Amazon has introduced a certification program for those who would like to demonstrate competence in AWS. The first certification they are offering is called “AWS Certified Solutions Architect – Associate Level“. Microsoft also seems to be adjusting their certification programs for the new reality of the cloud.  Even Learning Tree offers a Cloud Computing Certification Program!

Obtaining the Amazon AWS credential involves completing a comprehensive exam demonstrating in-depth knowledge of AWS services as well as general IT knowledge. Learning Tree course: Cloud Computing with Amazon Web Services is a good introduction to AWS and will definitely help you on your way towards success on the AWS exam. On the other hand Learning Tree course: Cloud Computing Technologies: A Comprehensive Hands-On Introduction will give you exposure to a broad spectrum of cloud offerings including Microsoft Azure. Both of these courses offer hands-on experience in what the data shows is one of the hottest technology areas for 2013.

Hope to see you soon at a Learning Tree Education Center or online via AnyWare!

Kevin Kell

Data Analytics in the Cloud – Microsoft Azure SQL Reporting

First, let me say I have been looking forward to using this service for some time. As a long time user of SQL Server Reporting Services (SSRS) I was eager to use this technology in some of my cloud computing projects. I was disappointed a few months ago, however, when I started down the path but then discovered that the minimum cost I could achieve would be in the order of around $600 per month. I could not in good conscience recommend that to my client. At the current cost of $0.16 per hour I think it is still a little expensive but may be preferable to building an on-premises solution. In this post I will take a quick look at using Azure SQL Reporting as a cloud-based business intelligence tool.

We can start from the Azure portal. Select the SQL REPORTING tab. A wizard walks you through the process of creating a Reporting Services service. Once the wizard completes you will have provisioned a Reporting Server on Azure. We can then go ahead and create reports and publish them to this server in exactly the same way we would publish to any SSRS server.

To create reports I usually use the Business Intelligence Template for Visual Studio 2012 (aka BIDS). I have also used Report Builder 3.0 but I find that I am more comfortable in Visual Studio and it offers more flexibility. For my report data source I created a database on Azure SQL DATABASE. In this way my whole solution is in the cloud. When you are finished creating the report to your specification all you then have to do is just deploy it to your Azure SQL Reporting service using the Web Service URL shown on the dashboard.

The only issue I found here is that when you create the data source in BIDS you need to specify that the connection type is Microsoft SQL Azure. If you don’t do this you will get an error when you deploy the report.

Figure 1. Data Source Properties

Once it is deployed you can view the report in a browser:

Figure 2. Report hosted on Azure SQL Reporting service

So, all in all, the process is pretty straightforward and painless. In this way you can easily deploy SSRS reports without the administrative requirement of having to install and configure a Report Server on-premises. I think that it is likely, however, that there may be some incompatibilities that I haven’t discovered yet. I will continue to look for these in the days to come.

Kevin Kell

Ch … Ch … Changes in the Cloud!


You take a few months break from teaching cloud computing and when you come back to it things are not the same. Most notably, there are several significant new features in Amazon Web Services and Microsoft Azure.

Let’s first take a look at Azure.

The Management Portal has completely changed. It now has a clean new look that gives clear access to all of the Azure services. Notice all the additional services that are now available beyond the basic cloud services, database, storage and service bus.

Figure 1. Azure Management Portal

It is nice to see that Microsoft have addressed the needs of people who want to just host a simple web site on Azure. This was not very cost effective using cloud services but now up to 10 web sites per region can be deployed for free using the Azure web sites service. There is also a smaller sized SQL database available for $5 per month which may be adequate for such applications.

For me the other exciting development is Business Analytics as a service. A key component of this is SQL Reporting. The SQL Reporting service looks to essentially be SQL Server Reporting Services (SSRS) in the cloud. At $0.16 per hour per deployed report server instance it is still a little expensive (imho) but is significantly reduced from the astronomical $.88 per hour it was previously. Business Analytics also includes an implementation of Hadoop which is going to very accessible and familiar for .NET developers.

The Amazon Web Services Management Console has also had a facelift.

Figure 2 AWS Management Console

Several services have been added including Redshift for data warehousing and OpsWorks for application management. Glacier, which is now several months old, offers a very cost effective solution for archiving. Just last week Amazon announced the ability to now copy Machine Images (AMI) from one region to another. This is a great enhancement and is something I have been wanting for a while. Certainly this opens up many options for high availability and disaster recovery solutions that span regions and not just availability zones.

Constant change is the nature of cloud computing. That definitely makes keeping up on the technologies a challenge. I look forward to getting back to teaching it now for a while. I am currently on the schedule to teach our Introduction to Cloud Computing course in April and June and our Amazon Web Services course in April and May. Why not consider joining us for one of those sessions?

Kevin Kell

Learning Tree Logo

Cloud Computing Training

Learning Tree offers over 210 IT training and Management courses, including Cloud Computing training.

Enter your e-mail address to follow this blog and receive notifications of new posts by e-mail.

Join 53 other followers

Follow Learning Tree on Twitter


Do you need a customized Cloud training solution delivered at your facility?

Last year Learning Tree held nearly 2,500 on-site training events worldwide. To find out more about hosting one at your location, click here for a free consultation.
Live, online training
.NET Blog

%d bloggers like this: