Time for another series of posts!
This time, I'll talk about a few scalability options in ASP.NET applications. Getting higher performance and greater scalability isn't a one-time, one-shot thing -- rather, it's something that is measured and tweaked continually. While it's true that a good design up front is important, it's also true that you can't focus too much on performance tuning early in the game.
What alarmed me recently was taking a look at my performance stats
up until the last few days in January, 2008:
You can clearly see the worsening performance over time. Between March of '07 and Oct '07, my page gen time nearly doubled; between August of '07 and September of '07, my WorldMaps gen time _more_ than doubled (!), RSS and home page gen times were somewhat flat.
Because some of the numbers were fairly flat, I'm inclined to think that nothing profound happened on the server side (this can happen on shared hosting and is sometimes unavoidable). Rather, it's likely some "thermal layer" for traffic was crossed -- or perhaps more appropriately, we went off a scalability cliff somewhere. In the case of WorldMaps, the cliff analogy is certain more appropriate given the sudden change. This is why it's so important to have ways to measure performance. In fact, I'm not really interested in the specific numbers in as much as I'm interested in the trends.
I sat back over the weekend, dove into the logs a bit more in depth, and tweaked away. I deployed some changes in very late January, and looking at the February stats thus far, it seems to have made a dramatic increase in performance across the board: RSS went from about 45ms to around 5ms, home page from around 75ms to 45ms, average page and WorldMaps from around 225ms to roughly 50ms. Still too early to call it complete, but without a doubt, a step in the right direction. (Despite a small WorldMap bug a few folks alerted me to! Sorry 'bout that!)
The most obvious (and cost-effective) ways to increase performance would be to boost the CPU, RAM, or network capacity of the server. None of these were options for me.
The first thing I did is examine _where_ the application was running slow -- not just pages, but specific method calls. As Jeff Prosise points out in one of his MSDN articles
, once I/O requests start queueing, application performance tanks dramatically. This is what was happening to WorldMaps.
My data caching strategy was pretty good, but I needed to open things up with some asynchronous processing. I implemented this on a few of the pages, including WorldMaps, and I believe that explains the performance increase. Be sure to check out Jeff's article on implementing asynchronous pages in your application. Although I implemented my async tasks a bit differently this time around, I have used precisely the methods Jeff points out in the article in past projects -- the nice thing is, it's very easy to change the behavior for debugging purposes (you could even do this via precompiler directives).
Finally, especially with WorldMaps and to a lesser extent RSS, I needed some better client-side caching. The first series I'd like to do will focus on how to pull that off effectively. Specifically, I'll illustrate to how to do this within HTTP Handlers or within custom code. Stay tuned!