Like many Albertans, I have spent a significant amount of time over the last month paying attention to the election! Reading about the candidates, following all the drama, and spending lots of time with the #abvote hashtag on Twitter. As the candidates were making one final push over the weekend before the election, I decided to build a results dashboard. I like a good challenge and enjoyed building it, but it was especially rewarding to see that it proved to be quite popular too! In this post I’ll tell you a little about how and why I built the website, and what I learned from it.
If you haven’t checked out the dashboard, you can see it here. I’ve added a bunch of stuff since election night, which I’ll explain below.
By late Friday afternoon, my thoughts had drifted to election day itself. I started to think about how exciting it would be to see the results come in – I love election nights! I knew there would be television coverage and that the media would have some web coverage as well, but I also felt that I could build something unique and valuable. If only I had the data! So I looked around, and found the Elections Alberta results site. At that time, the results page was full of test data. I immediately saved a copy to my computer, and saved a few of the electoral division pages too. That proved to be a wise decision, because a few hours later the site went offline!
Before I took a crack at scraping the website, I wanted to know if there was a data feed of some kind available. I blindly emailed the general Elections Alberta address, and to my surprise, received a response shortly thereafter! Unfortunately there was no data feed available, so I set about writing a scraper. Within a couple of hours, I was correctly scraping the main results page as well as all of the electoral division pages. Now that I had the data, I felt pretty confident that I could build a dashboard over the weekend. I didn’t get back to the project until Sunday morning, so that meant I had to prioritize what I was going to build. It took about six hours, but my I finished my initial version late that evening.
This was not my first election results dashboard. If you’ve been reading my blog for a while, you’ll recall that I built a dashboard for the municipal election here in Edmonton back in 2010. I learned a lot from that experience, and I remember it being a lot more rushed and difficult than this dashboard! Among other lessons, it was clear that design and colors matter, and that mobile devices are important (even then lots of people were asking for mobile support). I also knew that forcing users to refresh the page is less than ideal – it’s not a very delightful experience, and it puts unnecessary strain on the server. I also disliked the limited real estate that I had to work with (the current ShareEdmonton page width is fixed…but I’m working on a new version that is fluid).
So, I wanted a mobile-friendly, fluid-width, Ajax-enabled, attractive looking design. I immediately decided to use Twitter Bootstrap. I have used it a few times now, and I absolutely love it. I can’t thank the folks at Twitter enough for making such an excellent framework available for free! It gave me everything I needed to get going from a UI perspective. In particular it features responsive design, which makes it possible for the pages to scale from the desktop down to mobile devices without much work. For the backend, I used ASP.NET MVC 3. I use it for everything, so I know it well.
For performance reasons, it definitely made sense to cache the data. I decided on a fairly straightforward approach: I’d scrape the data from Elections Alberta and would store it using Memcached for two minutes. That meant that every two minutes, a request would take slightly longer because it had to download the data again, but this seemed reasonable (and as it turned out, the Elections Alberta site was incredibly quick). I also designed the pages to poll for new data every 30 seconds, which prevented users from having to reload the page manually.
When I built the ShareEdmonton dashboard a couple years ago, it was hosted on one of my servers. That worked fine, but it did slow down under load and I didn’t have much ability to scale up or out without a lot of additional cost, time, and effort. I really wanted to avoid that situation this time, so I decided to host the dashboard using Windows Azure. I’m in the process of migrating ShareEdmonton to Azure, so I already had an account and was pretty familiar with how it worked. Deploying to Azure is so easy – I simply had to add a deployment project in Visual Studio, and then I could deploy new versions in just a couple of clicks.
Windows Azure supports a range of instance types – basically you get to choose how big and powerful you want your server to be. I started with “Extra Small”, the least powerful and therefore least expensive type. As the polls were about to close at 8pm, I scaled up to “Small”, which meant redeploying the app (which took about 8 minutes, but happened completely behind-the-scenes). About half an hour later, I had to add capacity because the site was starting to get quite sluggish. This time I scaled out, by adding a second instance. All I had to do was change a configuration setting in the Azure management console, and the service took care of everything. Within a few minutes, I had two load-balanced “Small” instances. The performance boost was immediately noticeable. About an hour later, I added a third instance, and kept the system running that way until about 1am. I scaled it back down in stages, and now have it running as a single “Extra Small” instance again.
Two Key Decisions
I think the two most important decisions I made were:
- Using Twitter Bootstrap
- Using Windows Azure
The decision first meant that the website looked good and worked across browsers, screen resolutions, and devices. I got all of that engineering effort and testing for free, which meant I could focus on building an election results dashboard rather than building a website. I didn’t have to figure out how to lay things out on the screen, or how to style tables. The second decision was perhaps even more important. By using Windows Azure, I could deploy new versions of the dashboard in minutes, plus I could scale up and out simply by changing a few settings. That meant I could quickly respond when the site came under load. The other big advantage of using Azure was the cost – running the site on election night cost me just $1.54. Incredible!
The dashboard served around 60,000 page views on election night alone, which is pretty good for a website launched just hours before the main event. Keep in mind that because the data on the site automatically updated, users didn’t have to refresh the page which kept that statistic lower than it would otherwise have been. The visit duration metric is another way to see that – 20% of all visitors spent at least 10 minutes on the site. I actually would have guessed a higher percentage than that, but perhaps the high mobile usage was the reason.
The top screen resolution for visitors was 320×480, not a desktop resolution! Roughly 36% of all visits that night were made on mobile devices (which includes tablets). The iPhone was the most popular device, followed by the iPad. Clearly using a framework like Twitter Bootstrap with responsive design was a good decision.
The other statistic worth sharing is that the vast majority of visitors (about 73%) found the site by way of social networks, and two in particular. Facebook accounted for 78% of all those visits, while Twitter accounted for 20%.
Since Monday I have made numerous improvements to the dashboard. Here’s a brief overview of the new features:
- All the data is now stored locally, which means I’m no longer reliant on Elections Alberta. They have made numerous updates over the last two days, and I have updated the site’s local data store accordingly.
- I updated the voter turnout chart and added regional voter turnout to the front page. I also added a table of the five closest races.
- District pages now show voter turnout and the list of polls is now sortable.
- There’s a new Districts Grid, which lets you see lots of information about all the districts in a single, sortable view. For example, you can quickly see which district had the best voter turnout, which were the closest races, and which had the most candidates.
- There’s also a Candidates page, which lets you see information about all of the candidates in a single, sortable view.
- Last night I also added a Maps page, which has interactive maps for the province, as well as zoomed-in maps for Calgary and Edmonton. Click on any region for details and a link to the district page.
I plan to keep the dashboard up as it is now, though at some point I’ll probably transition it from being a dynamic website to a static one (far cheaper to host over the long-run). If you have any suggestions on things to add or improve, let me know! I hope the site will serve as a valuable reference tool going forward.
Thanks for reading, and thanks to everyone who sent positive comments about the dashboard my way. It’s great to hear that so many people found it useful on election night!