Tag: open data

City Council data now available in Edmonton’s open data catalogue

Yesterday Edmonton became the first city in Canada to release “a fully robust set” of City Council datasets to its open data catalogue. A total of five datasets were released, including meeting details, agenda items, motions, attendance, and voting records. There are now more than 100 datasets available in the catalogue, with more on the way.

Here’s the video recording of the news conference:

The City also produced a video about the new datasets:

The Office of the City Clerk is responsible for managing Council & Committee meetings, boards, elections, and more. The release of this data (referred to as “Clerk’s data” by some City employees) is another example of the way that office has embraced technology over the years. Kudos to Alayne Sinclair and her team, as well as Chris Moore, Ashley Casovan, and the rest of the IT team for making this data available!

I’m really excited about the potential for this data. The information has long been available on the City’s website, it was just locked away in meeting minutes as “unstructured” data – possible for humans to read relatively easily, but not for software. Now that it is available as “structured” data in the open data catalogue, applications can be written that take advantage of the data. You can find the data under the City Administration tab of the catalogue. Unfortunately the datasets only go back to June 1, 2011 instead of the start of Council’s term in October 2010. Currently, the datasets are updated daily.

I’ve now had a chance to look through the data, and while it looks good, it is unfortunately incomplete at the moment. There’s quite a bit of data missing. I would love to do some statistical analysis on the data, but with so many missing records there’s a good chance that my conclusions would be incorrect. I have already summarized my findings and passed them along to the team, so hopefully they can resolve the issues quickly!

I have already added functionality to ShareEdmonton for this data, and as soon as the datasets are fixed, I’ll release it. I hate to say “stay tuned” but there’s not much choice right now!

Recap: CityCamp Edmonton

Canada’s first CityCamp was held here in Edmonton on Saturday at the Robbins Health Learning Centre downtown. Roughly 50 people attended the unconference focused on innovation for municipal governments and community organizations. We had a nice mix of municipal employees, developers, journalists, and citizens.

We started the day off with this timeline of Edmonton’s open data journey (click for a larger version):

A brief history of open data in Edmonton

It was a joint effort to create this. Jess put it together with input from myself and Ashley, and Raffaella created a Prezi for us. I was struck by two things when reflecting on the timeline:

  1. This journey really didn’t start that long ago. I guess two years in the technology world is a long time, but it’s pretty quick in the world of municipal governments.
  2. I was instantly reminded of London’s tube map when I looked at the timeline, which is fitting as so much of our open data journey in Edmonton has been based around transit. It was the GTFS feed that ETS made available to developers that really kicked things off back in 2009.

With that foundation in place, we invited everyone to pitch their topics for the day. There was a nice range of topics suggested, everything from “Modeling the Value of Open Gov” to “Increasing Awareness of Apps Among Edmontonians”.

CityCamp Edmonton

Raffaella and Ashley acted as our gridmasters, and arranged the topics into the day’s agenda. We broke into groups and the discussions began!

CityCamp Edmonton

The sessions I participated in looked at the media’s role in open government, the importance of archiving and digital preservation, and the benefits of open data in small municipalities. I really enjoyed the discussions and I think everyone got something out of them, even if it was just inspiration or motivation to go away and do something!

CityCamp EdmontonCityCamp Edmonton

In the session on small municipalities, I thought Devin made a really great point. He said that open data adoption today is kind of like website adoption was 15 years ago or so. At the time, having a website was a new concept, and municipalities approached it much like they are approaching open data now – with uncertainty, hesitation, and even resistance. Today every municipality has a website. How did we get over those initial roadblocks? Devin suggested that tooling has a lot to do with it, which echoed some of the discussion we had in the media session earlier in the day. Tools like FrontPage really opened up the floodgates and made it easy for people to create a website. We have since moved on to better things, of course, but it was an important enabler early on. Perhaps we need the open-data-equivalent of FrontPage for open data to really take off as well. In the media session we identified tools as one of the things holding up wider adoption of open data by journalists.

A session I didn’t participate in but which looked like a lot of fun was the modeling open government one. Jess was determined to have at least one session that wasn’t sitting around in a circle talking, and he succeeded!

CityCamp Edmonton

The space we were in was great, with lots of natural light (which I think is really important), though it was somewhat difficult to find in the morning. Everything worked out though!

CityCamp EdmontonCityCamp Edmonton

Though we did have a discussion about actions arising from the day’s sessions, I think drilling down on collective actions is always difficult at events like CityCamp. For me, the day was an opportunity to connect with other people thinking about the same issues and topics. That said, here are a few things you can do to take action right now:

  • Look at the open data catalogue, and start using the data. Share the things you’re doing with it!
  • As you come up with ideas that require data sets that aren’t currently available, add them to the UserVoice. I will take an action to clean that site up a bit!
  • Notes from the day will be going up on Civic Commons, so that’s a great place to capture and share your thoughts on open government in Edmonton.

Thanks to everyone who participated on Saturday! You can see my photos from the day here, and Jess’ photos here.

First look at Canada’s new Open Data portal: data.gc.ca

Yesterday the Government of Canada launched its open data portal at data.gc.ca. Open Data is one of three Open Government Initiatives, the other two being Open Information and Open Dialogue. Stockwell Day, President of the Treasury Board and Minister for the Asia-Pacific Gateway, issued a statement today on the launch:

“Today, I am pleased to announce the next step in our government’s commitment to enhancing transparency and accountability to Canadians. The expansion of open government will give Canadians the opportunity to access public information in more useful and readable formats, enable greater insight into the inner workings of the Government and empower citizens to participate more directly in the decision-making process.”

He goes on in the statement to say that Canada has historically led the way in providing information to citizens. Lately though, we’ve definitely fallen behind. I’m glad to see us moving forward once again. This development is no doubt the result of lots of work by many passionate Canadians, such as David Eaves. Here’s what he posted yesterday:

The launch of data.gc.ca is an important first step. It gives those of us interested in open data and open government a vehicle by which to get more data open and improve the accountability, transparency as well as business and social innovation.

David does a good job in that post of highlighting some of the issues the site currently faces, such as some problematic wording in the licensing, so I won’t repeat that here. Instead, I figured I’d do what I always do when I get new datasets to play with – make some charts!

The open data portal says there are 261,077 datasets currently available. Just 781 of those are “general” datasets, the rest are geospatial. That’s an impressive number of geospatial datasets, but they are somewhat less accessible (and perhaps less interesting) to the average Canadian than the general datasets. It looks like you need to be able to work with an ESRI Shape File to use most of them.

There are lots of general datasets you might find interesting, however. For example, here’s the Consumer Price Index by city:

Here’s another dataset I thought was interesting – the number of foreign workers that have entered Canada, by region:

Have you ever wondered how much of each type of milk Albertans consume? You can find that out:

There’s actually a fairly broad range of datasets available, such as weather, agriculture, economics, and much more. As David said, it’s a good first step.

I’m excited to see more ministries get involved, and I hope to see the number of datasets available increase over time. I’d also love to see the licensing change, perhaps by adopting the UK Open Government License as David suggested. Exciting times ahead!

Fun with Open Data and Excel: Edmonton’s Busiest Streets

Today the City of Edmonton added a new dataset to the open data catalogue – average daily street traffic volumes from 2004-2009. Here’s the description:

Include traffic counts obtained with automatic traffic recorders over the past six years. The volume shown is a total of all vehicles in both directions, over a 24 hour period, for a typical weekday in a year. Volumes shown as a dash "-" indicate no count was undertaken that year. Seasonal adjustments are made to reflect an average weekday during the year. An increase or decrease in volume does not necessarily indicate a trend for a given roadway. Road construction or the introduction of more advanced equipment in 2009 affect the traffic volume numbers.

I’d like to build something interesting with this data, perhaps as part of a future ShareEdmonton release. But to start, I decided to open the data up in Microsoft Excel, something I often do with new datasets. Here’s what I discovered.

First, a few summary points:

  • There are 1496 street locations in the dataset.
  • Of those, 823 were counted in 2009. The year with the most counted locations was 2006, at 1076. Just 32 locations were counted every year (2004-2009). There were 531 locations counted in three years or more.
  • One of the first things I noticed is that every count ends in zero. I know the description outlines that the data is an average, adjusted seasonally, and perhaps for other reasons, but I wonder how close to reality the numbers really are.

Looking at 2009 simply because it is the most recent, there’s a hint of a long tail for the counted locations:

Here are the 200 busiest locations in Edmonton according to the 2009 count:

To create that map, I used a free utility called Excel to KML. There are all kinds of useful free KML tools online!

Here are the top ten busiest locations in Edmonton, according to the 2009 count:

  1. Calgary Trail SW North of Gateway Park Road SW
  2. Capilano Bridge
  3. 178 Street South of 81 Avenue
  4. Highway 216 NE North of Sherwood Park Freeway NE
  5. Anthony Henday Drive North of  87 Avenue
  6. HIghway 216 NE North of Baseline Road NE
  7. Anthony Henday Drive West of Calgary Trail
  8. 170 Street North of  95 Avenue
  9. Groat Road North of Victoria Park Road
  10. Highway 216 NE North of Whitemud Drive NE

Some of those are expected, some perhaps not. I decided to look at locations that were counted in at least three different years. If you look at the average daily volumes for that subset, here are the top ten busiest locations in Edmonton:

  1. Quesnell Bridge
  2. Yellowhead Trail West of 231 Street
  3. Calgary Trail SW North of Gateway Park Road SW
  4. Capilano Bridge
  5. Whitemud Drive West of 149 Street
  6. Yellowhead Trail West of Fort Road
  7. Cloverbar Bridge
  8. Yellowhead Trail West of  97 Street
  9. Yellowhead Trail West of 107 Street
  10. Yellowhead Trail West of 127 Street

Plotting the top 25 on a map gives a better sense of the really busy roads – Yellowhead Trail and Whitemud Drive:

Interesting, isn’t it? Of course, it could get a lot more interesting with some mashups. I’d love to have collision data for the City of Edmonton, to see how closely the number of collisions is correlated with traffic volumes. That’s just one example.

Check out the new dataset in the open data catalogue. If you do something interesting with it, I’d love to hear about it!

Edmonton Transit (ETS) now offers schedule information via SMS text messaging

Today Edmonton Transit (ETS) officially launched its new text messaging service. You can now send an SMS text message to 31100 from virtually any cell phone to get bus schedule information for free (standard messaging rates may apply). The service is yet another example of the GTFS feed that ETS released in 2009 paying dividends.

The way it works is simple – text the bus stop number you’re interested in to 31100 and you’ll get a response like this (using 1859 as an example):

1859 (111) 01:26P 01:56P (112) 01:06P 01:36P (2) 01:08P 01:23P ETS – THE EVERY DAY WAY

That’s the stop number, followed by each route with two upcoming times for each. If you’re interested in just a particular route, say the 2, simply text the stop number followed by the route number, and you’ll get upcoming times for just that route at that stop:

1859 (2) 01:08P 01:23P 01:38P 02:08P ETS – THE EVERY DAY WAY

ETS has produced two videos that demonstrate how it works (they were filmed just before Christmas). Here’s part 1:

And here’s part 2:

Starting next week you’ll also be able to create a profile on the website to setup favorite routes and stops specific to your daily routine.

The ability to get route & stop schedules via text messaging might be new here in Edmonton, but it’s not a new concept elsewhere. Transit riders in Vancouver have had that functionality for years, first unofficially thanks to a couple of enterprising students, and later as an official service from TransLink. The story is somewhat similar here. Local developers Sean Healy and Joel Jackson hacked together a service called TXT.2.ETS back in March of last year, something they won a prize for in the Apps4Edmonton competition. They subsequently met with ETS representatives and talked about what an official service might look like.

I talked to Nathan Walters, Strategic Marketing Supervisor for ETS, about the new service yesterday. He told me the service is run by Vancouver-based Upside Wireless (presumably their Transit SMS product). Nathan confirmed that it uses the same GTFS data that anyone can download from the City of Edmonton’s open data catalogue. “The GTFS feed made things significantly easier, and brought the service to market a lot faster,” Nathan told me. He also said it “speaks to how much we stand behind that information.”

Though it is a pilot, don’t expect the service to shut down any time soon. Instead, ETS plans to monitor usage, gather feedback, and will be conducting market research later this year, and will re-evaluate and improve the service as necessary. In fact, if all goes well, the service will pay for itself and perhaps even drive an additional return to the City via the advertising service that will launch in the spring. The last 40 characters of the messages sent back to commuters will be allocated toward advertising (the “ETS – THE EVERY DAY WAY” part in the example above). I think that’s a smart move.

It turns out the service has actually been live for about a month, in testing by Upside and City of Edmonton employees. Apparently there has been quite a bit of buzz internally at the City about it. Today was not the planned launch date, but the accidental release of parts of the outdoor marketing campaign pushed things up. Nathan confirmed that we’ll see lots of advertising for the new service, including billboards, benches, and transit shelters. He sounded pleasantly surprised that they had been able to keep the service under wraps for so long (no doubt the holidays helped in that regard).

I had to ask Nathan the question he’s undoubtedly going to be asked over and over now that the text messaging service is live: scheduled information is great, but when are we going to get live information, perhaps via GPS? His answer: “The technologies that transit uses are always evolving, and the service will evolve as well.” In other words, nothing to announce, and don’t hold your breath.

I also asked about the shortcode, 31100. Nathan said they briefly considered a repeating number (such as TransLink’s 33333) or something like “txtets” but realized that using letters would be problematic considering most modern smartphones do not have traditional numeric keypads. They settled on 31100 because it offered great potential for the future. Just like 311, other departments at the City could make use of the shortcode for their services as well. Nathan confirmed that at least one business unit outside of transportation has already expressed interest in exploring that possibility.

This new service is exciting and will have a positive impact on the daily commute for thousands of Edmontonians. But it’s more than that. ETS made a decision to open up its data to anyone for free, and they continue to see a return on that very minimal investment. Releasing the GTFS data feed has resulted in Google Maps support, apps for the iPhone, and now text messaging. Importantly, all ETS had to do for all of this to happen was focus on its core competency – providing transit service. It’s a success story that other business units at the City of Edmonton should be very eager to replicate.

Having said that, I would have preferred to see a solution from a local company or even the adoption of Sean & Joel’s project, but I realize there are other considerations. As the Apps4Edmonton competition evolves, I hope we’ll see the City adopting more of the solutions that Edmonton’s talented entrepreneurs create.

The new text messaging service is a great way to start 2011. In fact, Nathan said it “sets the tone for the year” and indicated that ETS was going to be looking at additional opportunities to work with the community on other services as 2011 unfolds. I can’t wait!

You can learn more about the service here, and you can find additional Apps for ETS here. Check out the Open Data catalogue for ETS data and more, and follow the #yegtransit hashtag on Twitter for ETS-related updates.

UPDATE: Does the new text messaging service work for the LRT? The answer is yes! If you know the stop numbers, that is. The route for the LRT is always 201, but here are the stop numbers you need.

Station Name North Stop # South Stop #
Clareview 7797 7797
Belvedere 7692 7830
Coliseum 1889 1742
Stadium 1981 1723
Churchill 1691 1876
Central 1863 1935
Bay/Enterprise Square 1985 1774
Corona 1926 1891
Grandin/Government Centre 1754 1925
University 2969 2316
Health Sciences/Jubilee 2014 2019
McKernan/Belgravia 9982 9981
South Campus 2116 2115
Southgate 2114 2113
Century Park 4982 4982

 

Hopefully they can make it more user friendly in the future.

Edmonton Election 2010: Visualizing Results by Polling Station

Ever wonder where the candidates in last month’s municipal election received the most support? Which parts of the city supported which mayoral candidates? After seeing the maps that were created for Calgary’s top three mayoral candidates, I wondered about the same kind of thing here. Local software developer Josh Kjenner was also interested, and he has been busy visualizing the results by polling station ever since.

Josh wrote an application called Metroview for the City of Edmonton’s Apps4Edmonton competition, a project which he spent about 60 hours on. The tool is implemented in Processing, a programming language and environment that Josh called “a really really intense Java library.” He returned to the project after the City of Edmonton released the final election results by polling station, and spent another 20 hours or so improving it. Josh told me the biggest challenge he faced was conditioning the KML files from the open data catalogue (a common challenge that open data developers face…getting the data and the tools/technology working together).

The result is an interactive application that lets you visualize candidate support and other data on a map of Edmonton.

Here are a few of the data visualizations you can see in Josh’s metroview yegvote 2010 app (requires Java).

Eligible voters versus voter turnout:

Support for Stephen Mandel:

Support for David Dorward:

Support for Daryl Bonar:

You can see that Mandel received most of his support from the south part of the city, while Dorward received the most support from the north part. You can use the metroview tool to see similar results for every ward, public school ward, and catholic school ward too.

Thanks Josh for creating this tool! This is another great example of what can be created when the data is open and available.

UPDATE: It should be noted that you can’t really compare Mandel’s graph to Dorward’s. A dark area in Mandel’s is not equivalent to a dark area in Dorward’s, for example, because of the difference in the number of overall votes that each candidate received. The colors on each graph are in relation to the other areas on that graph for that candidate only. If you look at Josh’s app, you get the raw values as you hover over each area, and you can choose absolute instead of relative for the drawing mode.

Open Data and Municipal Election Results

As local readers will know, I created an election results dashboard for the municipal election that took place in Edmonton on October 18, made possible thanks to the City of Edmonton’s open data. I’d say it was very well-received! There was lots of positive feedback, and it resulted in the highest-trafficked day ever for ShareEdmonton. There were also hundreds of tweets about the dashboard, including some from people in other cities.

I was able to update the dashboard on-the-fly that night, fixing the graph colors and other issues as they were identified. Overall I was pretty happy with how it turned out, though I wish I had been able to add a few more features in time for the big event. Over the last couple days, I decided to extract the dashboard from ShareEdmonton and I have now added that new functionality. Specifically, the page updates automatically (no more refreshing) and there’s a mobile view as well, so it looks decent on your mobile device.

New Dashboards

Today is election day in Ontario, and thanks to open data from Toronto, Ottawa, and London, I was able to launch three new election results dashboards, all featuring the latest improvements:

Some new issues have appeared that I didn’t have to worry about in the Edmonton election (for example, there are 40 candidates for mayor in Toronto, so having them all appear on the graph is difficult at best) but the sites all seem to be running smoothly. I guess we’ll find out at 8 PM EST tonight when the results start to come in!

Open Data

All of these dashboards were made possible because the cities made election results data available, so kudos to each of them for doing so. The tricky thing for a developer like me is that all four cities (Edmonton, Toronto, Ottawa, and London) made the results available in a different format!

  • Edmonton – XML/JSON using OGDI
  • Toronto – plaintext, pipe-separated
  • Ottawa – plaintext, HTML scrape, kind of a pseudo-CSV
  • London – XML

Fortunately, when I had created the dashboard for Edmonton, I had built in a translation-layer. So instead of loading directly from the City’s catalogue into ShareEdmonton, I loaded it first into an internal format. So to get the dashboard working with the other cities, all I had to do was write that translation piece, from their catalogue to my internal format. As you can see, that strategy has worked fairly well (Ottawa was by far the most difficult, and did not provide me with enough information to calculate voter turnout, so that was removed from the ShareOttawa dashboard).

I’m not complaining about the need for this translation layer. I would much rather have a city make its data available than delay just to try to find a common format. But I do hope to see convergence over time. It would be great if these cities (and others that offer this data, like Vancouver) could agree on a common format. That would remove the extra work required to make an application reusable, and would hopefully result in even more applications that make use of the data.

And Beyond!

I’m pretty excited about the possibilities of open data, especially as it relates to politics and democracy. There are some really great apps being built, such as Vote.ca, and we’re really just scratching the surface. If we keep at it, I’m confident that open data will have a positive impact on the way we engage with politics.

Double props to the Edmonton Journal

As you probably know, I don’t shy away from criticizing the Edmonton Journal (or other local media). Though I don’t always succeed, I do try to be constructive, because I think there’s incredible opportunity facing The Journal. Two such opportunities: data-driven journalism, and real-time reporting.

Props to Brent Wittmeier & Lucas Timmons

Neither Brent nor Lucas have been with The Journal for very long, and maybe that’s why they were able to succeed with the unclaimed balances story. Brent was voluntold to write about the Bank of Canada’s unclaimed balances, which he did by teaming up with data journalist Lucas to create a searchable online database for Edmonton. They put together a three-part story, but it didn’t stop there:

We also got quite the response. Dozens of phone calls and emails poured in, and I began working on a follow-up story. And then two. And now, three. Some of these other stories are even better than the original… There should be an extensive piece either later this week or next weekend.

Start with some data, and more often than not a story will emerge. Brent noted: “In truth, they ended up being far more interesting stories than I thought.”

Props to Paula Simons & David Staples

I’m glad to see that Paula and David (with some help from other colleagues such as Todd Babiak) have started a new blog focused on local affairs, called The Edmonton Commons. They used it very effectively on nomination day to share stories about the candidates and the official start of the election. They’ve also got the #yegvote hashtag embedded on the page. Though they have cross-posted some columns, I’m hopeful that their use of the blog as way to forego the print deadline will expand.

Here is Paula’s first post, and here is David’s first post. I like what David had to say:

The sharpening of ideas, the accumulation of good information and the discarding of bad information is at the core of strong decision making. It’s what we hope to do here at this forum. In the past, there were more barriers in regards to entering into the great conversation of civil society, even for a newspaper writer…the conversation was largely one-way. The Internet gives us a new tool that enables that conversation to flourish.

Time will tell how successful the two are with the blog (will they still be writing as often in three months as they do now) but I think they’re off to a great start.

City of Edmonton launches Apps4Edmonton Competition

On Wednesday the City of Edmonton officially launched its Apps4Edmonton competition, which challenges residents to submit ideas for apps and developers to actually build them. There is more than $50,000 in prizes available, and everyone has a chance to win an Apple iPad. Here’s what the competition is all about:

Every Edmontonian can be an active participant in reshaping our city. Together with community partners the City of Edmonton is proud to launch Apps4Edmonton – a contest which encourages residents to develop applications that will benefit Edmontonians.

Using data sets from the Edmonton Open Data Catalogue, we’re challenging you to develop an app for either a smart phone, mobile device, or PC web browser. Mash up a map, create a visualization, or analyze data in a new way, the choice is yours.

To support the competition, the City has also released an update to the Data Catalogue. There are now 25 datasets available, including Ward boundaries and a list of City facilities. Even more datasets should be available soon, and if there’s something specific you’d like to have, request it from opendata@edmonton.ca.

If you have an idea for an app, submit it here. It can be anything, so be creative! Even if your idea isn’t possible or a developer doesn’t want to take it on, it might serve as inspiration for other apps, so don’t be shy. If you’re building or have already built an app, submit it here. You have until August 27 to do so. The FAQ has some good info, and be sure to familiarize yourself with the rules too.

Applications will be judged on four criteria: Accessibility, Usability, Creativity, and the number of community votes received (voting runs online from August 27 to September 10). In addition to three overall prizes (Gold, Silver, Bronze) there are six categories:

  • The Way We Live
  • The Way We Grow
  • The Way We Green
  • The Way We Move
  • The Way We Plan
  • The Way We Prosper

I’m very happy to see that IT has aligned itself with Transforming Edmonton for this competition! I am hopeful that Apps4Edmonton can help introduce more Edmontonians to those important documents that help guide us toward becoming a more innovative, inclusive, and creative city.

To learn more about Apps4Edmonton, visit the contest site. There should be lots of chatter about the competition at Startup Weekend too, and tonight at the Emerging Business Leaders’ Patio Party.

I can’t wait to see what everyone comes up with!

Talking Open at Technocon 2010

I’ve spent the last two days at Technocon 2010, a technology conference jointly hosted by the City of Edmonton and the University of Alberta for their respective IT employees (about 450 of which have attended). With six keynotes and more than fifty breakout sessions on a range of topics (everything from “Fulfillment and Freedom” to “BlackBerry Enterprise Server”) there has been a little something for everyone.

I’m closing the show this afternoon with a keynote on the topic of “open”, which is one of the three themes for the conference (the other two are “world class” and “transformation”). Keeping in mind that I stand between the audience and home-time, I’m going to keep things light and brief. Here are my slides:

The key thought I want to share is that open government is fundamentally about the relationship between government and citizens and less about technology. And related to that, we should avoid the temptation to jump on the open government bandwagon just because it’s the hot new thing, and focus instead on making sure we improve that relationship. Both sides have a role to play, and I hope to bring some of the citizen perspective to the audience. And of course, one of my key messages is that we can work together.

You can watch it live here. Thanks to Technocon for having me!