Azure Miniseries #4: Monitoring Applications

by Brian Hitney 26. February 2010 03:58

In this screencast, we'll take a look at monitoring Azure applications by capturing event logs and performance counters. We'll also look at using PowerShell to deploy and configure applications using the management API. Finally, we'll take a sneak peek at Azure Diagnostics Manager, a tool from Cerebrata that allows you explore event logs and look at performance counters visually.

Get Microsoft Silverlight

Here are some links from the screencast:

Finally, let’s get into some code snippets! Watch for wrap on the PowerShell lines, and note the single quote ` character as the line continuation:

Creating a self-signed certificate command:

makecert -r -pe -a sha1 -n "CN=Azure Service Test" -ss My -len 2048 -sp "Microsoft Enhanced RSA and AES Cryptographic Provider" -sy 24 AzureServiceTest.cer

Uploading a new deployment to the staging slot, and starting it (requires Azure CmdLets):

$cert = Get-Item cert:\CurrentUser\My\{thumbprint}
$sub = "{subscription GUID}"
$servicename = "{service name}"

$package = "CloudApp.cspkg"
$config = "ServiceConfiguration.cscfg"

[DateTime]$datelabel = Get-Date
$lbl = $datelabel.ToString("MM-dd-yyyy-HH:mm")

Write-Host "Label for deployment: " $lbl

Add-PSSnapin AzureManagementToolsSnapIn

Get-HostedService $servicename -Certificate $cert -SubscriptionId $sub |
New-Deployment -Slot Staging $package $config -Label $lbl |
Get-OperationStatus -WaitToComplete

Get-Deployment staging -serviceName $servicename -SubscriptionId $sub -Certificate $cert |
Set-DeploymentStatus running |
Get-OperationStatus -WaitToComplete

Increasing the number of instances:

Add-PSSnapin AzureManagementToolsSnapIn

$cert = Get-Item cert:\CurrentUser\My\{thumbprint}
$sub = "{subscription GUID}"
$servicename = "{service name}"
$storage = "{storage name}"

#get storage account name and key
$key = (Get-StorageKeys -ServiceName $storage -Certificate $cert -SubscriptionId $sub).Primary
$deployId = (Get-HostedService $servicename -SubscriptionId $sub -Certificate $cert | Get-Deployment Production).DeploymentId       

Get-HostedService $servicename -Certificate $cert -SubscriptionId $sub |
    Get-Deployment -Slot Staging |
    Set-DeploymentConfiguration {$_.RolesConfiguration["WebRole1"].InstanceCount += 1}

Updating the performance counters – specifically, adding total processor time, ASP.NET req/sec, and memory usage to be polled every 30 seconds, and uploaded every 1 minute:

Add-PSSnapin AzureManagementToolsSnapIn

$cert = Get-Item cert:\CurrentUser\My\{thumbprint}
$sub = "{subscription GUID}"
$servicename = "{service name}"
$storage = "{storage name}"

#get storage account name and key
$key = (Get-StorageKeys -ServiceName $storage -Certificate $cert -SubscriptionId $sub).Primary
$deployId = (Get-HostedService $servicename -SubscriptionId $sub -Certificate $cert | Get-Deployment Production).DeploymentId       

# rate at which counters are polled
$rate = [TimeSpan]::FromSeconds(30)

Get-DiagnosticAwareRoles -StorageAccountName $storage -StorageAccountKey $key -DeploymentId $deployId |
foreach {
    $role = $_
    write-host $role
    Get-DiagnosticAwareRoleInstances $role -DeploymentId $deployId `
        -StorageAccountName $storage -StorageAccountKey $key |

    foreach {
        $instance = $_

        $config = Get-DiagnosticConfiguration -RoleName $role -InstanceId $_ -StorageAccountName $storage `
             -StorageAccountKey $key -BufferName PerformanceCounters -DeploymentId $deployId            
        $processorCounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration `
                -Property @{CounterSpecifier='\Processor(_Total)\% Processor Time'; SampleRate=$rate }
        $memoryCounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration `
                -Property @{CounterSpecifier='\Memory\Available Mbytes'; SampleRate=$rate }
        $requestsCounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration `
                -Property @{CounterSpecifier='\ASP.NET Applications(__Total__)\Requests/Sec'; SampleRate=$rate }
        $config.DataSources.Clear()
        $config.DataSources.Add($processorCounter)
        $config.DataSources.Add($memoryCounter)
        $config.DataSources.Add($requestsCounter)
        Set-PerformanceCounter -PerformanceCounters $config.DataSources -RoleName $role `
             -InstanceId $instance -DeploymentId $deployId `
             -TransferPeriod 1 `
             -StorageAccountName $storage -StorageAccountKey $key                     
    }  
}

And finally, the webrole.cs class from the screencast:

public class WebRole : RoleEntryPoint
    {
        public override bool OnStart()
        {
            DiagnosticMonitorConfiguration diagConfig =
                DiagnosticMonitor.GetDefaultInitialConfiguration();

            diagConfig.PerformanceCounters.DataSources.Add(
                new PerformanceCounterConfiguration()
                {
                    CounterSpecifier = @"\Processor(_Total)\% Processor Time",
                    SampleRate = TimeSpan.FromSeconds(5)
                });

            diagConfig.PerformanceCounters.DataSources.Add(
                new PerformanceCounterConfiguration()
                {
                    CounterSpecifier = @"\Memory\Available Mbytes",
                    SampleRate = TimeSpan.FromSeconds(5)
                });

            diagConfig.PerformanceCounters.ScheduledTransferPeriod =
                TimeSpan.FromMinutes(1);

            diagConfig.Logs.ScheduledTransferLogLevelFilter = LogLevel.Information;
            diagConfig.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);

            DiagnosticMonitor.Start("DiagnosticsConnectionString", diagConfig);

            System.Diagnostics.Trace.TraceInformation("Done configuring diagnostics.");

            // For information on handling configuration changes
            // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
            RoleEnvironment.Changing += RoleEnvironmentChanging;

            return base.OnStart();
        }

        public override void OnStop()
        {
            System.Diagnostics.Trace.TraceWarning("Onstop called.");
            base.OnStop();
        }

        private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
        {
            // If a configuration setting is changing
            if (e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange))
            {
                // Set e.Cancel to true to restart this role instance
                e.Cancel = true;
            }
        }
    }

Tags: , , ,

Azure | Microsoft | Tech Tips

Azure Miniseries #3: ServiceConfig vs web.config

by Brian Hitney 22. February 2010 03:32

One of the challenges developers will face when developing Windows Azure web applications is: where do I put my settings?  In the ServiceConfiguration file or the web.config?

There isn’t one correct answer.  The challenge of keeping everything in the web.config is that it makes changes and deployment much more difficult.  Because the web.config is part of the deployment, any change to the file also requires a redeployment.  If you use a build system that targets your dev/stage/QA/prod environments automatically and can provide the correct settings in the web.config for you, this might mitigate the problem.

The answer then is to migrate these settings to the ServiceConfiguration file as it requires no changes to our deployment package.  In this screencast I’ll show you some strategies for doing that for components that are more difficult to migrate, like the SqlMembershipProvider…

Get Microsoft Silverlight

Link to original post and download links.

Tags: , ,

Azure | Microsoft | Tech Tips

Azure Miniseries #2: Deployment

by Brian Hitney 13. February 2010 13:39

In my first Azure Miniseries post, I showed setting up a new cloud service project and migrating an existing ASP.NET application into Azure.   Before I dive into other topics, I figured I’d jump to the end and discuss deployment – getting your Azure application into the cloud.  

Get Microsoft Silverlight

Link to original post with download links.

Tags: , ,

Azure | Microsoft | Technology

Azure Miniseries #1: Migration

by Brian Hitney 12. February 2010 03:54

I’m starting to put together some short form screencasts on Windows Azure related topics.  I’ll use my blog to dive into specifics or display code samples/downloads where appropriate – but first up is a quick look at getting a project setup and migrating existing applications into an Azure webrole.

Get Microsoft Silverlight

Tags: ,

Azure | Microsoft | Technology

MSDN Events and Roadshows Coming Soon…

by Brian Hitney 11. February 2010 13:38

Top line:  March 3rd, we’ll be in Raleigh, and March 5th, we’ll be in Charlotte for our next MSDN Event and Southern Fried Roadshow. 

This time it’s a full day of Azure – if you have an interest in cloud computing, be sure to come out!  See you then!

MSDN Events presents:  Take Your Applications Sky High with Cloud Computing and the Windows Azure Platform

Join your local MSDN Events team as we take a deep dive into cloud computing and the Windows Azure Platform. We’ll start with a developer-focused overview of this new platform and the cloud computing services that can be used either together or independently to build highly scalable applications. As the day unfolds, we’ll explore data storage, SQL Azure, and the basics of deployment with Windows Azure. Register today for these free, live sessions in your local area.

SESSION 1: Overview of Cloud Computing and Windows Azure

The Windows Azure platform is a set of high-performance cloud computing services that can be used together or independently and enable developers to leverage existing skills and familiar tools to develop cloud applications. In this session, we’ll provide a developer-focused overview of this new online service computing platform. We’ll explore the components, key features and real day-to-day benefits of Windows Azure.

SESSION 2: Survey of Windows Azure Platform Storage Options

Durable data storage is a key component of any cloud computing offering. The Windows Azure Platform offers many options, which can be used alone or in combination. Windows Azure itself offers ready-to-use and lightweight storage in the form of tables, blobs, and queues. Another choice for storage is SQL Azure, a true relational database in the cloud. In this session, we’ll explore the highlights of these implementations and how to both create and use storage in each form. We’ll give you guidance on choosing the right forms of storage for your application scenarios.

SESSION 3: Going Live with your Azure Solution

Windows Azure features a powerful, yet simple deployment model. By focusing on your application and abstracting away the infrastructure details, you can deploy almost any app with minimal fuss. In this session, we’ll walk you through the basics of Windows Azure deployment, including site monitoring, diagnostics and performance issues.

Tags: , ,

Events | Microsoft

Azure SLA Confusion

by Brian Hitney 10. February 2010 07:00

Azure SLA is something that gets discussed quite a bit but there’s something that I see causing a bit of confusion.  The SLA for Azure compute instances states:

For compute, we guarantee that when you deploy two or more role instances in different fault and upgrade domains, your internet facing roles will have external connectivity at least 99.95% of the time.

Some folks (for example, this post) incorrectly conclude that you need to deploy your solution across 2 or more datacenters to get this SLA.  Actually, that’s not true – you just need to make sure they are in different fault and upgrade domains.  This is something that is typically done by default.  You can think of a fault domain as a physical separation in a different rack, so if there’s a hardware failure on the server or switch, it only affects instances within the same fault domain.  Upgrade domains are logical groupings that control how deployments are upgraded.  For large deployments, you may have multiple upgrade domains so that all roles within an upgrade domain are upgraded as a group.

To illustrate this, I spun up 3 instances of Worldmaps running on my local Dev Fabric.  I have an admin tool in the site that shows all current instances, their role, and their domain affiliation:

image

The admin page uses the RoleEnvironment class to check status of the roles (more on this in another post), but also display their fault and upgrade domains.  (A value of “0f” is fault domain 0.  “0u” is upgrade domain 0, and so on).  So by default, my three instances are in separate fault and upgrade domains that correspond to their instance number.

All of these instances are in the same datacenter, and as I long as I have at least 2 instances and ensure they have different fault and upgrade domains (which is the default behavior), I’m covered by the SLA. 

The principal advantage of keeping everything within the same datacenter is cost savings between roles, storage, and SQL Azure.  Essentially, any bandwidth within the data center (for example, my webrole talking to SQL Azure or Azure Storage) incurs no bandwidth cost.  If I move one of my roles to another datacenter, traffic between datacenters is charged.  Note however there are still transaction costs for Azure Storage.

This last fact brings up an interesting and potentially beneficial side effect.  While I’m not trying to get into the scalability differences between Azure Table Storage and SQL Azure, from strictly a cost perspective, it could be infinitely more advantageous to go with SQL Azure in some instances.   As I mentioned in my last post, Azure Storage transaction costs might creep up and surprise you if you aren’t doing your math.  If you’re using Azure Table Storage for session and authentication information and have a medium volume site (say, less than 10 webroles but that’s just my off the cuff number – it really depends on what your applications are doing), SQL Azure represents a fixed cost whereas Table Storage will vary based on traffic to your site.

For example, a small SQL Azure instance at $9.99/month = $0.33/day.  Azure Table transactions are $0.01 per 10,000.   If each hit to your site made only 1 transaction to storage, that would mean you could have 330,000 hits per day to achieve the same cost.   Any more, and SQL Azure becomes more attractive, albeit with less scalability.   In many cases, it’s possible you wouldn’t need to go to table storage for every hit, but then again, you might make several transactions per hit, depending on what you’re doing.  This is why profiling your application is important.

More soon!

Tags: , ,

Tech Tips | Azure | Microsoft

Robocopy in Win7

by Brian Hitney 1. February 2010 13:17

I’m surprised I didn’t realize this earlier, but Win7 introduced multithreading support in Robocopy.  I’ve always used Robocopy for copying lots of files particularly over a network, and especially over a VPN.  I’ve also always specified the /E /Z switches to copy all folders, and support file resume. 

File resume is especially helpful over a VPN because the VPN connection may be lost for any number of reasons.  I’ve had situations where my access was off for hours, and it picks up right away on reconnect.   Now in Win7, you can specify a /MT[:n] switch that specifies the number of threads (default 8) to use.  

And if you prefer a GUI, RichCopy is a nice tool as well!

Tags: ,

Microsoft | Tech Tips

Pay Attention to your Azure Blobs

by Brian Hitney 13. December 2009 15:35

I have a fairly large Windows Azure migration I’m working on, and there are dozens of tips, recommendations, gotchas, etc. I’ve learned in the process.  This is one small item that cost me quite a bit of time, and it’s so simple I’m detailing it here because someone will run into this one.

First a bit of background: if you’re deploying a Windows Azure application, the package is uploaded and deployed as a whole.  If you have dynamic content or individual files that are subject to change, it’s a good idea to consider placing it in Azure Storage, otherwise you’ll have to redeploy your entire application to update one of these files.  In this case, I’d like to put a Silverlight XAP file in Azure storage, instead of the typical /ClientBin folder.

There are a number of references on the net for setting this up – searching for “XAP” and “Azure” returns a lot of good references including this one.  After checking my Silverlight project’s app manifest file, ensuring everything was correct, and uploading the XAP to my Azure storage account, my Silverlight app would refuse to run.  The page would load, but then … nothing.  I also checked Fiddler – the XAP file _was_ getting downloaded (and was downloading fine via a browser).  This is typically a tell-tale sign of a policy issue – yet I was certain everything was correct.  Here’s a screenshot (click for larger) of Fiddler downloading the file.  Can you spot the problem here?

image

The problem was indirectly with Cerebrata’s Cloud Storage Studio.   Cerebrata’s product is really nice and I enjoy working with it quite a bit to work with Azure storage (uploading/downloading files).  CloudBerry’s Explorer for Azure Blob Storage is another good one – I typically work with Cerebrata’s simply because it was easier to connect to development storage (not sure if this is possible in CloudBerry’s app).

Fast-forward and hour or two of frustration.  Staring at Cloud Storage Studio, I see:

image

Zip file as a content type?  This was in the Fiddler screenshot above, too.  I figured this was fine because after all, a XAP file is a zip file.  But as it turns out, this was the problem.   For kicks, I tried re-uploading the XAP file from CloudBerry to see how it handled the content type, and:

image 

Cloud Storage Studio does allow you to alter the content types, but truthfully I didn’t think this was all that important.  When I reloaded my application, though, the app loads and Fiddler comes to life as it should:

image

For kicks I changed the content type back to Zip, and the app would fail to load.  So, lesson learned!  Don’t forget about the content types.

Tags: , ,

Development | Microsoft | Tech Tips | Azure

MSDN Southern Fried Roadshow

by Brian Hitney 24. October 2009 08:54

We’re back!  Starting early November, we’ll start our fall MSDN Southern Fried Roadshow series.  Right now, we’re scheduled here:

11/4/2009 Greensboro http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032429239&Culture=en-US
11/5/2009 Raleigh http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032429321&Culture=en-US
11/6/2009 Columbia http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032429322&Culture=en-US
12/8/2009 Atlanta http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032429323&Culture=en-US
12/9/2009 Montgomery http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032429324&Culture=en-US

Check out our sessions below!

Introduction to ADO.NET Data Services

The new wave of Web applications are built on technologies such as AJAX and Microsoft Silverlight, which enable developers to build better, richer user experiences. These technologies bring a shift in how applications are organized, including a stronger separation of presentation from data. In this session, you’ll learn how ADO.NET Data Services enables applications to expose data as a REST-based data service that can be consumed by Web clients within a corporate network and across the Internet. We’ll look at how the data service is reachable over HTTP, and how URIs are used to identify the various pieces of information available through the service. You’ll get a solid understanding of data service concepts like HTTP verbs (GET, POST, PUT, and DELETE) and data formats (ATOM/APP and JavaScript Object Notation (JSON)).

Parallel Programming in .NET 4.0 - Much more than Threading!

Come learn how the next version of Visual Studio and the Microsoft .NET Framework can help you write better performing and more scalable applications. We take a tour of new .NET APIs, including the Task Parallel Library (TPL) and Parallel LINQ (PLINQ). We also introduce new features in the debugger that help you quickly identify concurrency issues and visualize the internal state of your application. You don’t need any experience with writing multi-threaded code to benefit from this session

Windows 7 and Windows Server – Exciting New OS Developer Features

Windows 7 and Windows Server 2008 R2 come packed with loads of new developer goodies.  Computers are starting to come with more and more hardware built-in – from Ambient Light Sensors to Accelerometers and GPS.  Come learn about how the Windows 7 Sensor API can integrate with your applications, and help them rock with these new hardware capabilities.  Not to be outdone, Windows Server R2 has seen some really incredible boosts to IIS with version 7.5 and the extensibility models.  Now you can manage your databases, delegate control to multiple IIS servers, and configure package deployment from all within one location!

Tags: , ,

Technology | Microsoft | Events

Visual Studio 2010 Beta 2 and Fx 4 Available

by Brian Hitney 19. October 2009 05:51

This is exciting!  Today, VS2010 Beta 2 and .NET Framework 4 is available for MSDN subscribers.  For more information on VS2010, visit the Visual Studio home page.   If you’re not an MSDN subscriber, general availability is October 21st.  Also, and more importantly, the official launch date:  March 22nd, 2010.   Mark your calendars!

Also, MSDN has received a bit of a makeover, so check it out while downloading.  You may have noticed a few SKU changes with VS/MSDN – I don’t have all the details yet, but it looks like Visual Studio with MSDN will have three main SKUs:

· Microsoft Visual Studio 2010 Ultimate with MSDN:

Microsoft Visual Studio 2010 Ultimate with MSDN is the comprehensive suite of application lifecycle management tools for teams to ensure quality results from design to deployment.

· Microsoft Visual Studio 2010 Premium with MSDN:

Microsoft Visual Studio 2010 Premium with MSDN is a complete toolset for developers to deliver scalable, high quality applications.

· Microsoft Visual Studio 2010 Professional with MSDN:

Microsoft Visual Studio 2010 Professional with MSDN is the essential tool for basic development tasks to allow developers to implement their ideas easily.

More to come on this soon.

Tags: , ,

Development | Microsoft

your host...

Brian Hitney
Developer Evangelist
Microsoft Corp.

About Me

My Worldmap