I love Windows Home Server.   I’ve been using Windows Home Server for years, and just purchased a Windows Home Server 2011(WHS) box from Newegg (great deal on a HP Proliant micro server). 

Many have asked me why I like WHS so much – it’s NAS, it’s a media server, it’s backup.   It’s a step up from a simple NAS device (although, admittedly, not as plug and play), offers more flexibility and is more cost effective than a Drobo.  A small backup agent can take snapshots of your PC, typically on a daily basis, so they can be restored to a given point in time.  I keep snapshots of my initial installation, for example.  Restoring to those backups is a simple process. 

I’m also a big fan of Sonos, a whole-home music solution that works amazingly well.  What Sonos has done exceedingly well is blend quality hardware, quality software, and reasonable (but not cheap) price points.  I have an extensive music collection, and I point Sonos to a share on my WHS box to index and stream music. 

However, I consider my laptop my “database of record” for my music.  It’s where I download stuff, and I take it with me because I’m often on the road.  The problem I run into is keeping my WHS library in sync with my laptop.   In my case, I want to mirror my library on the WHS exactly as it is on my local collection – and because I’m often reorganizing my collection, adding tags, etc., I need a simple way to do this. 

Enter Robocopy.   Robocopy (Robust file copy) is now built into Windows, and it’s a simple command line tool with a number of options to make this a snap.   For example, if I want to mirror a folder on my laptop with my WHS, this command will do it:

c:\>robocopy "D:\Music" "\\BEAST\Sonos\music" /mir /r:10 /MT:8

D:\Music is my local folder, my server is \\Beast.  The /mir command is for mirror – it’s the same as using /purge and /e:  /purge is to delete files at the target folder that no longer exist in the source, and /e is to copy all subdirectories, including empty ones.   The /r:10 will tell it to retry up to 10 times, in case of some network glitch, and the /MT:8 will have Robocopy use 8 threads to speed things along.   (If you’re familiar with Robocopy, I don’t recommend using /z (restartable) mode as it adds overhead, not needed given the size of files we’re dealing with.)

Now, what if you don’t keep all your music local, and just want to copy it over?   You don’t want to use /mir since it will remove files you otherwise want to keep!   The rest of the command will work fine, but if you move/rename files locally that were previously copied, you’ll have to remember to do that manually on the server.  Once Robocopy does its thing, you’ll get a nice summary:

            Total    Copied   Skipped  Mismatch    FAILED    Extras
Dirs :      1384        29      1355          0         0         0
Files :     15078       381     14697         0         0         0
Bytes : 117.188 g   3.212 g 113.975 g         0         0         0
Times :   0:16:40   0:02:52                       0:00:00   0:00:48

Here, it copied about 30 new folders.  It took about 16 minutes to run, but that’s largely due to new content, having copied some 3.2gb of new files.   Assuming minor changes only, the process typically runs in about 30 seconds.

If you want to get fancy, you could even have Robocopy monitor your folders for changes. 

The next challenge is to have Sonos update its music index once new files are copied over.   Sonos can update its index on a daily basis (or manually via the control software), but I want it done automatically after new files are copied over.  This one is a bit trickier, but thanks to some gurus in the Sonos forums, it’s not impossible.   I’m including the .exe file here for you to use.  Obviously, trusting an exe from someone on the web is not something I’d do, but it’s a .NET assembly which means you can use a tool like JustDecompile to crack it open and look at the source yourself.   Having said that, I’m not responsible if this code causes your computer to blow up, your music collection to vanish, or kills any puppies.

The source code looks like so, and it sends an SOAP packet to a specified Sonos unit to trigger an index rebuild:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Net;

namespace SonosIndexUpdater
{
    class Program
    {
       
        static void Main(string[] args)
        {
            string ip;

            if (args != null && args.Length > 0)
            {
                ip = args[0].Trim();
            }
            else
            {
                Console.WriteLine("Missing IP Address. Please add IP address for any Sonos unit.");
                return;
            }

            string header1 = @"SOAPACTION: ""urn:schemas-upnp-org:service
                :ContentDirectory:1#RefreshShareIndex""";
            string postData = @"<s:Envelope xmlns:s=""http://schemas.xmlsoap.org/soap/envelope/"" 
                s:encodingStyle=""http://schemas.xmlsoap.org/soap/encoding/"">
                <s:Body>
                <u:RefreshShareIndex xmlns:u=""urn:schemas-upnp-org:service:ContentDirectory:1"">
                <AlbumArtistDisplayOption></AlbumArtistDisplayOption></u:RefreshShareIndex>
                </s:Body>
                </s:Envelope>";
            string url = string.Format("http://{0}:1400/MediaServer/ContentDirectory/Control", ip);

            byte[] byteArray = Encoding.UTF8.GetBytes(postData);

            try
            {
                System.Net.WebRequest req = System.Net.WebRequest.Create(url);
                req.Headers.Add(header1);
                req.ContentType = "text/xml";
                req.Method = "POST";
                req.ContentType = "application/x-www-form-urlencoded";
                req.ContentLength = byteArray.Length;
                req.Timeout = 5000;

                Stream dataStream = req.GetRequestStream();

                dataStream.Write(byteArray, 0, byteArray.Length);
                dataStream.Close();

                using (WebResponse response = req.GetResponse())
                {
                    Console.WriteLine("Response from Sonos: {0}", 
                        ((HttpWebResponse)response).StatusDescription);

                    using (dataStream = response.GetResponseStream())
                    {
                        using (StreamReader reader = new StreamReader(dataStream))
                        {
                            string responseFromServer = reader.ReadToEnd();
                            Console.WriteLine("Data: {0}", responseFromServer);
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Exception occured: {0}", ex.Message);
            }

        }
    }
}

Jim, Peter, and I are gearing up for another road trip to spread the goodness that is Windows Azure! The Windows Azure DevCamp series launched recently with a two-day event in Silicon Valley, and we’re jumping on the bandwagon for the East Region.

We have five stops planned in December, and we’re doing things a bit differently this go-round. Most of the events will begin at 2 p.m. and end at 9 p.m. – with dinner in between of course. The first part will be a traditional presentation format and then we’re bringing back RockPaperAzure for some “hands-on” time during the second half of the event. We’re hoping you can join us the whole time, but if classes or your work schedule get in the way, definitely stop by for the evening hackathon (or vice versa). By the way it wouldn’t be RockPaperAzure without some loot to give away, so stay “Kinected” to our blogs for further details on what’s at stake!

Here’s the event schedule, be sure to register quickly as some venues are very constrained on space. You’ll want to have your very own account to participate, so no time like the present to sign up for the Trial Offer, which will give you plenty of FREE usage of Windows Azure services for the event as well as beyond.

Rock, Paper, Azure (RPA) is back!   For those of you who have played before, be sure to get back in the game!  If you haven’t heard of RPA, check out my past posts on the subject.   In short, RPA is a game that we built in Windows Azure.  You build a bot that plays a modified version of rock, paper, scissors on your behalf, and you try to outsmart the competition. 

Over the summer, we ran a Grand Tournament where the first place prize was $5,000!   This time, we’ve decided to change things a bit and do both a competition and a sweepstakes.   The game, of course, is a competition because you’re trying to win.  But we heard from many who didn’t want to get in the game because the competition was a bit fierce.  

Competition:  from Nov 25 through Dec 16, each Friday, we’ll give the top 5 bots a $50 Best Buy gift card.  If you’re the top bot each Friday, you’ll get a $50 gift card each Friday. 

Sweepstakes:  for all bots in the game on Dec 16th, we’ll run the final round and then select a winner at random to win a trip to Cancun.   We’re also giving away an Acer Aspire S3 laptop, a Windows Phone, and an Xbox Kinect bundle.  Perfect timing for the holidays!

Check it out at http://www.rockpaperazure.com!

One of the great new features of the Windows Azure platform is the Azure Traffic Manager, a geo load balancer and durability solution for your cloud solutions.  For any large website, managing traffic globally is critical to the architecture for both disaster recovery and load balancing.

When you deploy a typical web role in Azure, each instance is automatically load balanced at the datacenter level.   The Azure Fabric Controller manages upgrades and maintenance of those instances to ensure uptime.  But what about if you want to have a web solution closer to where your users are?  Or automatically direct traffic to a location in the event of an outage?   

This is where the Azure Traffic Manager comes in, and I have to say, it is so easy to set up – it boggles my mind that in today’s day and age, individuals can prop up large, redundant, durable, distributed applications in seconds that would rival the infrastructure of the largest websites. 

From within the Azure portal, the first step is to click the Virtual Network menu item.

On the Virtual Network page, we can set up a number of things, including the Traffic Manager.   Essentially the goal of the first step is to define what Azure deployments we’d like add to our policy, what type of load balancing we’ll use, and finally a DNS entry that we’ll use as a CNAME:

We can route traffic for performance (best response time based on where user is located), failover (traffic sent to primary and only to secondary/tertiary if primary is offline), and round robin (traffic is equally distributed).   In all cases, the traffic manager monitors endpoints and will not send traffic to endpoints that are offline.

I had someone ask me why you’d use round robin over routing based on performance – there’s one big case where that may be desirable:  if your users are very geography centric (or inclined to hit your site at a specific time) you’d likely see patterns here one deployment gets maxed out, while another does not.   To ease the traffic spikes to one deployment, round robin would be the way to go.  Of course, an even better solution is to combine traffic shaping based on performance with Azure scaling to meet demand.

In the above image, let’s say I want to create a failover for the Rock Paper Azure botlab (a fairly silly example, but it works).   I first added my main botlab (deployed to South Central) to the DNS names, and then added my instance deployed to North Central:

From the bottom of the larger image above, you can see I’m picking a DNS name of botlab.ctp.trafficmgr.com as the public URL.  What I’d typically do at this point is go in to my DNS records, and add a CNAME, such as “www.rockpaperazure.com” –> “rps.ctp.trafficmgr.com”.

In my case, I want this to be a failover policy, so users only get sent to my North Central datacenter in the event the south central instance is offline.  To simulate that, I took my south central instance offline, and from the Traffic Manager policy report, you’d see something like this:

To test, we’ll fetch the main page in IE:

… and we’re served from North Central.  Of course, the user doesn’t know (short of a traceroute) where they are going, and that’s the general idea.  There’s nothing stopping you from deploying completely different instances except of course for the potential end-user confusion!

But what about database synchronization?   That’s a topic for another post …

Chris Williams just let me know about an event happening in Atlanta this Saturday, Oct 8 … and it’s free!  Free is good.   Check it out here:

http://codemastery.eventbrite.com/

Sessions:

1. CSLA .NET intro - Rocky Lhotka

Topic will give attendees a high level overview of CSLA as an application framework. Key moving parts of CSLA will be covered, along with answering the most important question: Why use CSLA? Roles of business objects, data portal, rules, authentication and authorization will be covered in principal.

2. Business object design - Eric Blackwell

Session will concentrate of best practices for designing business objects. Single responsibility principal and maintainability will be covered in light of using CSLA. Key aspects of good CSLA business layer will be covered in detail, including properties, rules, data portal, data access, business method and validation. Particular attention will be paid to structuring classes and relationship between classes. Designing based on use cases will be an important aspect of the session.

3. Business, validation, and authorization rules - Tim Price-Williams

This session will be a deep dive into the world or rules. Topics such as validation rules, user authentication and authorization will be covered. Distinction between validation and business rules be drawn. Important key scenarios will be covered, such as synchronous and asynchronous rules, client / server rules, object creation and save scenario from rules perspective. Custom and built-in rules be covered in detail. A pattern for typical business rule/methods will be illuminated.

4. Data portal and n-tier architecture - Rocky Lhotka

This topic will cover in details all possibilities that CSLA provides when abstracting communication channels between client and server components. Difference between local and remote data portal will be discussed. Various configuration patterns will be highlighted along with usage scenarios for each one. Multi-tier deployment as it relates to data portals will be covered, as well as using external data sources instead of CSLA data portal in client only scenarios.

5. Data access - Travis Brown

This session is all about data access technologies and how they relate to CSLA data portal access. The topic will include patterns for abstracting data access for business objects to promote maintainability. Discussion of Microsoft technologies for data access will take place as well.

6. XAML and MVVM - Sergey Barskiy

This session will concentrate on using CSLA as business layer in XAML based user interfaces. Taking Silverlight as an example, session will highlight how CSLA base classes can be used to facilitate communication between UI and business objects. Adaptability of CSLA business layer to seamlessly alter user interface based on rules be will covered. Patterns for wiring business objects for Silverlight environment will be part of the discussion.

7. ASP.NET MVC - Mitch Gordon

This session will concentrate on using CSLA as business layer in ASP.NET MVC based user interfaces. The discussion will include CSLA provided base classes that will allow developers write less code. The session will illuminate patters for maintaining authentication and authorization rules between server calls. Patterns for adapting UI based on user rights will be discussed.

We’re back – this time with an International Grand Tournament in Rock, Paper, Azure.    So what’s new?

First, we heard many folks loud and clear that they weren’t happy it was U.S. residents only.   So, now’s your chance – we’ve opened up the tournament to Canada, the UK, Sweden, New Zealand, Germany, China, and of course the USA.   We’ve also included country flags in the leaderboard:

Next, we’ve changed some of the rules.  Specifically, players are now “blind” when they play in the GT.  What does that mean?   It means that your bot will not know the team name of the opponent.  While playing, the name of the opponent is a “?” and this is also reflected in the game history and log file:

Why this change?  Primarily, we felt it made the game a little more interesting as it focuses on algorithms as opposed to brute force.   We’ve created a GT Practice Round that is not blind, so if you wish, you can tinker in this round to get some exposure and fine tune your logic.  Of course, playing in the practice round is optional. 

Next, players will break down into heats during the GT.   After the round closes, we’ll segment players into a number of heats (as I write this, I can’t quite recall if we agreed on a random 25% in each heat, or 25 players per heat).  The idea is that this creates a ladder approach to get to the top and adds a bit of excitement to see how far up the ladder your bot can go.  It also scales nicer, since we’re assuming a higher involvement in the competition.

Finally, we decided to give away something a little sweeter than an Xbox.  This time, we’re got $5,000 riding on first place!  Additionally, what we’ve decided to do is spread out the winnings a bit more so second place receives $1,000, and the next ten players (3rd-12th place) all receive $250. 

So, why the prize structure?  Well, during an in-person event during our original 6 week competition, I heard someone remark that it would be too difficult to place in the top 3 to get a prize, much less win the Xbox.   I can understand that because, indeed, some of the bots we saw were really phenomenal.   What we wanted to do was make it so there were enough prizes to reward “pretty good play” for those (like myself) are interested in playing a little, but not spending a hundred hours coding a bot. 

With the new prize structure plus blind playing, it’s really anyone’s game with a little clever code.  We hope you think so, too… and have fun playing!  Questions or comments, feel free to ping us either here on my blog or through the www.rockpaperazure.com website.

By way of Glen Gordon:  Two great community guys – Chris Eargle and Chris Williams - are setting up community led Windows Phone Garage events next week in Columbia and Greenville.

Remember, at a Windows Phone Garage you get instruction on the basics of creating Windows Phone apps, but also you get a lot of time to build your app, with help from onsite experts.

In addition, one attendee who showcases his app (in development or complete) at the event will win a free Windows Phone device!

Register ASAP and make plans to attend this free event.

Columbia, SC – Thursday, June 16 at Midlands Tech

Greenville, SC – Friday, June 17 at Immedion

Unfortunately I won’t be there as we’ll be in Florida doing our Azure Tech Jam events, but the events are in great hands with Chris².

During a presentation the other day to the Charlotte ALT.NET group,  I made a joke that Rock, Paper, Azure is doing something completely ridiculous: we invite people to write code and we’ll run it arbitrarily.  (Well, not really arbitrarily, but it does present a unique security challenge.)

We’ve naturally had a few interesting submissions, so I’m posting some of them here for interest sake.

First up:

Thread.Sleep(...);

   1: if (dataset.Tables.Contains(opponent.TeamName))

   2: {

   3:      DataTable table = dataset.Tables[opponent.TeamName];

   4:      table.Rows.Clear();

   5:      foreach (Round round in rounds)

   6:      {

   7:     ...

… but in a short run game like this, I’d steer away from datasets.  (Plus, we have LINQ!)  This one gets caught in the filter not because it poses a specific threat, but we don’t allow anything from System.Data.

Third:

   1: try

   2: {

   3:       StreamWriter writer = new StreamWriter(@"C:\RPSLog.txt", append);

   4:       writer.Write(builder);

   5:       writer.Close();

   6: }

   7: catch (Exception)

   8: {

   9: }

WOW a lot of people are trying to write text files.  Not allowed.

And now for my all time favorite … an obvious hack attempt:

   1: StringBuilder builder = new StringBuilder();

   2: using (SqlConnection connection = new SqlConnection(ConfigurationManager.

   3:     ConnectionStrings[0].ConnectionString))

   4: {

   5:      using (SqlCommand command = new SqlCommand("SELECT * FROM sys.Tables", connection))

   6:      {

   7:          SqlDataReader reader = command.ExecuteReader();

   8:          while (reader.Read())

   9:          {

  10:              builder.Append(reader["[name]"]);

  11:              builder.Append(",");

  12:          }

  13:      }

  14: }

  15: you.Log.AppendLine("Here: " + builder);

This last one actually raises a legitimate issue and security threat – so much so that we can ban players (or worse) for this kind of thing.  Still, though, not much of a threat:  the code can’t get through, but even if it could, connection strings aren’t available to the app domain running the round, and even so, the engine only has execute permissions on the procedures necessary to insert the results.

I’m curious to see what else comes through!

In part 1, I detailed some of the specifics in getting the Rock, Paper, Azure (RPA) up and running in Windows Azure.   In this post, I’ll start detailing some of the other considerations in the project – in many ways, this was a very real migration scenario of a reasonably complex application. (This post doesn’t contain any helpful info in playing the game, but those interested in scalability or migration, read on!)

The first issue we had with the application was scalability.  Every time players are added to the game, the scalability requirements of course increases.  The original purpose of the engine wasn’t to be some big open-ended game played on the internet;  I imagine the idea was to host small (10 or less players).    While the game worked fine for < 10 players, we started to hit some brick walls as we climbed to 15, and then some dead ends around 20 or so. 

This is not a failing of the original app design because it was doing what it was intended to do.  In my past presentations on scalability and performance, the golden rule I always discuss is:  you have to be able to benchmark and measure your performance.  Whether it is 10 concurrent users or a million, there should always be some baseline metric for the application (requests/sec., load, etc.).   In this case, we wanted to be able to quickly run (within a few minutes) a 100 player round, with capacity to handle 500 players. 

The problem with reaching these numbers is that as the number of players goes up, the number of games played goes up drastically (N * N-1 / 2).   Even for just 50 players, the curve looks like this:

Now imagine 100 or 500 players!  The first step in increasing the scale was to pinpoint the two main problem areas we identified in the app.  The primary was the threading model around making a move.  In an even match against another player, roughly 2,000 games will be played.   The original code would spin up a thread for each _move_for each game in the match.   That means that for a single match, a total of 4,000 threads are created, and in a 100-player round, 4,950 matches = 19,800,000 threads!  For 500 players, that number swells to 499,000,000.

The advantage of the model, though, is that should a player go off into the weeds, the system can abort the thread and spin up a new thread in the next game.

What we decided to do is create a single thread per player (instead of a thread per move).  By implementing 2 wait handles in the class (specifically a ManualResetEvent and AutoResetEvent) we can accomplish the same thing as the previous method.  (You can see this implementation in the Player.cs file in the DecisionClock class.) 

The obvious advantage here is that we go from 20 million threads in a 100 player match to around 9,900 – still a lot, but significantly faster.   In the first tests, 5 to 10 player matches would take around 5+ minutes to complete.   Factored out (we didn’t want to wait) a 100 player match would take well over a day.   In this model, it’s significantly faster – a 100 player match is typically complete within a few minutes.

The next issue was multithreading the game thread itself.  In the original implementation, games would be played in a loop that would match all players against each other, blocking on each iteration.  Our first thought was to use Parallel Extensions (of PFx) libraries built into .NET 4, and kicking off each game as a Task.  This did indeed work, but the problem was that games are so CPU intensive, creating more than 1 thread per processor is a bad idea.  If the system decided to context switch when it was your move, it could create a problem with the timing and we had an issue with a few timeouts from time to time.   Since modifying the underlying thread pool thread count is generally a bad idea, we decided to implement a smart thread pool like the one here on The Code Project.   With this, we have the ability to auto scale the threads dynamically based on a number of conditions.

The final issue was memory management.  This was solved by design:  the issue was that original engine (and Bot Lab) don’t store any results until the round is over.  This means that all the log files really start to eat up RAM…again, not a problem for 10 or 20 players – we’re talking 100-200+ players and the RAM just bogs everything down.  The number of players in the Bot Lab is small enough where this wasn’t a concern, and the game server handles this by design by using SQL Azure, recording results as the games are played.

Next time in the deep dive series, we’ll look at a few other segments of the game.  Until next time!

You spoke, we listened.   Rock, Paper, Azure is going well, but one thing we’ve heard from many people is that they would like a history of their games.   The main reason for this is the “attack and retreat” approach  some people have taken – that is, they submit a bot with their main game playing skill, quickly observe/download the results, then resubmit a weak bot.   Because the MyBot page shows only the current results, odds are the other players won’t see the interim results.  

We originally thought the current results would be all people would want to see, as game history (aside from taking a lot of space) could potentially be a lot of data to go through.  But, we’ve been proven wrong, so we decided to keep the history for players to go through.  We wanted to incorporate this in a non-breaking way, so it’s optional data to look at.

On the MyBot page, you’ll still see the current games:

Under that, you’ll see game history that shows all interim games:

The filter allows you to select a single player to look at.  The number of games you see depends on the number of times your or your opponent’s bot was uploaded. 

We haven’t quite decided how long we’ll keep history, but at the very least, it’s there to use!  Thanks for the feedback!