Tag Archives: yegdata

City Council data now available in Edmonton’s open data catalogue

By Mack D. Male ·

Yesterday Edmonton became the first city in Canada to release “a fully robust set” of City Council datasets to its open data catalogue. A total of five datasets were released, including meeting details, agenda items, motions, attendance, and voting records. There are now more than 100 datasets available in the catalogue, with more on the way.

Here’s the video recording of the news conference:

The City also produced a video about the new datasets:

The Office of the City Clerk is responsible for managing Council & Committee meetings, boards, elections, and more. The release of this data (referred to as “Clerk’s data” by some City employees) is another example of the way that office has embraced technology over the years. Kudos to Alayne Sinclair and her team, as well as Chris Moore, Ashley Casovan, and the rest of the IT team for making this data available!

I’m really excited about the potential for this data. The information has long been available on the City’s website, it was just locked away in meeting minutes as “unstructured” data – possible for humans to read relatively easily, but not for software. Now that it is available as “structured” data in the open data catalogue, applications can be written that take advantage of the data. You can find the data under the City Administration tab of the catalogue. Unfortunately the datasets only go back to June 1, 2011 instead of the start of Council’s term in October 2010. Currently, the datasets are updated daily.

I’ve now had a chance to look through the data, and while it looks good, it is unfortunately incomplete at the moment. There’s quite a bit of data missing. I would love to do some statistical analysis on the data, but with so many missing records there’s a good chance that my conclusions would be incorrect. I have already summarized my findings and passed them along to the team, so hopefully they can resolve the issues quickly!

I have already added functionality to ShareEdmonton for this data, and as soon as the datasets are fixed, I’ll release it. I hate to say “stay tuned” but there’s not much choice right now!

Recap: CityCamp Edmonton

By Mack D. Male ·

Canada’s first CityCamp was held here in Edmonton on Saturday at the Robbins Health Learning Centre downtown. Roughly 50 people attended the unconference focused on innovation for municipal governments and community organizations. We had a nice mix of municipal employees, developers, journalists, and citizens.

We started the day off with this timeline of Edmonton’s open data journey (click for a larger version):

A brief history of open data in Edmonton

It was a joint effort to create this. Jess put it together with input from myself and Ashley, and Raffaella created a Prezi for us. I was struck by two things when reflecting on the timeline:

  1. This journey really didn’t start that long ago. I guess two years in the technology world is a long time, but it’s pretty quick in the world of municipal governments.
  2. I was instantly reminded of London’s tube map when I looked at the timeline, which is fitting as so much of our open data journey in Edmonton has been based around transit. It was the GTFS feed that ETS made available to developers that really kicked things off back in 2009.

With that foundation in place, we invited everyone to pitch their topics for the day. There was a nice range of topics suggested, everything from “Modeling the Value of Open Gov” to “Increasing Awareness of Apps Among Edmontonians”.

CityCamp Edmonton

Raffaella and Ashley acted as our gridmasters, and arranged the topics into the day’s agenda. We broke into groups and the discussions began!

CityCamp Edmonton

The sessions I participated in looked at the media’s role in open government, the importance of archiving and digital preservation, and the benefits of open data in small municipalities. I really enjoyed the discussions and I think everyone got something out of them, even if it was just inspiration or motivation to go away and do something!

CityCamp EdmontonCityCamp Edmonton

In the session on small municipalities, I thought Devin made a really great point. He said that open data adoption today is kind of like website adoption was 15 years ago or so. At the time, having a website was a new concept, and municipalities approached it much like they are approaching open data now – with uncertainty, hesitation, and even resistance. Today every municipality has a website. How did we get over those initial roadblocks? Devin suggested that tooling has a lot to do with it, which echoed some of the discussion we had in the media session earlier in the day. Tools like FrontPage really opened up the floodgates and made it easy for people to create a website. We have since moved on to better things, of course, but it was an important enabler early on. Perhaps we need the open-data-equivalent of FrontPage for open data to really take off as well. In the media session we identified tools as one of the things holding up wider adoption of open data by journalists.

A session I didn’t participate in but which looked like a lot of fun was the modeling open government one. Jess was determined to have at least one session that wasn’t sitting around in a circle talking, and he succeeded!

CityCamp Edmonton

The space we were in was great, with lots of natural light (which I think is really important), though it was somewhat difficult to find in the morning. Everything worked out though!

CityCamp EdmontonCityCamp Edmonton

Though we did have a discussion about actions arising from the day’s sessions, I think drilling down on collective actions is always difficult at events like CityCamp. For me, the day was an opportunity to connect with other people thinking about the same issues and topics. That said, here are a few things you can do to take action right now:

  • Look at the open data catalogue, and start using the data. Share the things you’re doing with it!
  • As you come up with ideas that require data sets that aren’t currently available, add them to the UserVoice. I will take an action to clean that site up a bit!
  • Notes from the day will be going up on Civic Commons, so that’s a great place to capture and share your thoughts on open government in Edmonton.

Thanks to everyone who participated on Saturday! You can see my photos from the day here, and Jess’ photos here.

Fun with Open Data and Excel: Edmonton’s Busiest Streets

By Mack D. Male ·

Today the City of Edmonton added a new dataset to the open data catalogue – average daily street traffic volumes from 2004-2009. Here’s the description:

Include traffic counts obtained with automatic traffic recorders over the past six years. The volume shown is a total of all vehicles in both directions, over a 24 hour period, for a typical weekday in a year. Volumes shown as a dash "-" indicate no count was undertaken that year. Seasonal adjustments are made to reflect an average weekday during the year. An increase or decrease in volume does not necessarily indicate a trend for a given roadway. Road construction or the introduction of more advanced equipment in 2009 affect the traffic volume numbers.

I’d like to build something interesting with this data, perhaps as part of a future ShareEdmonton release. But to start, I decided to open the data up in Microsoft Excel, something I often do with new datasets. Here’s what I discovered.

First, a few summary points:

  • There are 1496 street locations in the dataset.
  • Of those, 823 were counted in 2009. The year with the most counted locations was 2006, at 1076. Just 32 locations were counted every year (2004-2009). There were 531 locations counted in three years or more.
  • One of the first things I noticed is that every count ends in zero. I know the description outlines that the data is an average, adjusted seasonally, and perhaps for other reasons, but I wonder how close to reality the numbers really are.

Looking at 2009 simply because it is the most recent, there’s a hint of a long tail for the counted locations:

Here are the 200 busiest locations in Edmonton according to the 2009 count:

To create that map, I used a free utility called Excel to KML. There are all kinds of useful free KML tools online!

Here are the top ten busiest locations in Edmonton, according to the 2009 count:

  1. Calgary Trail SW North of Gateway Park Road SW
  2. Capilano Bridge
  3. 178 Street South of 81 Avenue
  4. Highway 216 NE North of Sherwood Park Freeway NE
  5. Anthony Henday Drive North of  87 Avenue
  6. HIghway 216 NE North of Baseline Road NE
  7. Anthony Henday Drive West of Calgary Trail
  8. 170 Street North of  95 Avenue
  9. Groat Road North of Victoria Park Road
  10. Highway 216 NE North of Whitemud Drive NE

Some of those are expected, some perhaps not. I decided to look at locations that were counted in at least three different years. If you look at the average daily volumes for that subset, here are the top ten busiest locations in Edmonton:

  1. Quesnell Bridge
  2. Yellowhead Trail West of 231 Street
  3. Calgary Trail SW North of Gateway Park Road SW
  4. Capilano Bridge
  5. Whitemud Drive West of 149 Street
  6. Yellowhead Trail West of Fort Road
  7. Cloverbar Bridge
  8. Yellowhead Trail West of  97 Street
  9. Yellowhead Trail West of 107 Street
  10. Yellowhead Trail West of 127 Street

Plotting the top 25 on a map gives a better sense of the really busy roads – Yellowhead Trail and Whitemud Drive:

Interesting, isn’t it? Of course, it could get a lot more interesting with some mashups. I’d love to have collision data for the City of Edmonton, to see how closely the number of collisions is correlated with traffic volumes. That’s just one example.

Check out the new dataset in the open data catalogue. If you do something interesting with it, I’d love to hear about it!

Edmonton Election 2010: Visualizing Results by Polling Station

By Mack D. Male ·

Ever wonder where the candidates in last month’s municipal election received the most support? Which parts of the city supported which mayoral candidates? After seeing the maps that were created for Calgary’s top three mayoral candidates, I wondered about the same kind of thing here. Local software developer Josh Kjenner was also interested, and he has been busy visualizing the results by polling station ever since.

Josh wrote an application called Metroview for the City of Edmonton’s Apps4Edmonton competition, a project which he spent about 60 hours on. The tool is implemented in Processing, a programming language and environment that Josh called “a really really intense Java library.” He returned to the project after the City of Edmonton released the final election results by polling station, and spent another 20 hours or so improving it. Josh told me the biggest challenge he faced was conditioning the KML files from the open data catalogue (a common challenge that open data developers face…getting the data and the tools/technology working together).

The result is an interactive application that lets you visualize candidate support and other data on a map of Edmonton.

Here are a few of the data visualizations you can see in Josh’s metroview yegvote 2010 app (requires Java).

Eligible voters versus voter turnout:

Support for Stephen Mandel:

Support for David Dorward:

Support for Daryl Bonar:

You can see that Mandel received most of his support from the south part of the city, while Dorward received the most support from the north part. You can use the metroview tool to see similar results for every ward, public school ward, and catholic school ward too.

Thanks Josh for creating this tool! This is another great example of what can be created when the data is open and available.

UPDATE: It should be noted that you can’t really compare Mandel’s graph to Dorward’s. A dark area in Mandel’s is not equivalent to a dark area in Dorward’s, for example, because of the difference in the number of overall votes that each candidate received. The colors on each graph are in relation to the other areas on that graph for that candidate only. If you look at Josh’s app, you get the raw values as you hover over each area, and you can choose absolute instead of relative for the drawing mode.

Open Data and Municipal Election Results

By Mack D. Male ·

As local readers will know, I created an election results dashboard for the municipal election that took place in Edmonton on October 18, made possible thanks to the City of Edmonton’s open data. I’d say it was very well-received! There was lots of positive feedback, and it resulted in the highest-trafficked day ever for ShareEdmonton. There were also hundreds of tweets about the dashboard, including some from people in other cities.

I was able to update the dashboard on-the-fly that night, fixing the graph colors and other issues as they were identified. Overall I was pretty happy with how it turned out, though I wish I had been able to add a few more features in time for the big event. Over the last couple days, I decided to extract the dashboard from ShareEdmonton and I have now added that new functionality. Specifically, the page updates automatically (no more refreshing) and there’s a mobile view as well, so it looks decent on your mobile device.

New Dashboards

Today is election day in Ontario, and thanks to open data from Toronto, Ottawa, and London, I was able to launch three new election results dashboards, all featuring the latest improvements:

Some new issues have appeared that I didn’t have to worry about in the Edmonton election (for example, there are 40 candidates for mayor in Toronto, so having them all appear on the graph is difficult at best) but the sites all seem to be running smoothly. I guess we’ll find out at 8 PM EST tonight when the results start to come in!

Open Data

All of these dashboards were made possible because the cities made election results data available, so kudos to each of them for doing so. The tricky thing for a developer like me is that all four cities (Edmonton, Toronto, Ottawa, and London) made the results available in a different format!

  • Edmonton – XML/JSON using OGDI
  • Toronto – plaintext, pipe-separated
  • Ottawa – plaintext, HTML scrape, kind of a pseudo-CSV
  • London – XML

Fortunately, when I had created the dashboard for Edmonton, I had built in a translation-layer. So instead of loading directly from the City’s catalogue into ShareEdmonton, I loaded it first into an internal format. So to get the dashboard working with the other cities, all I had to do was write that translation piece, from their catalogue to my internal format. As you can see, that strategy has worked fairly well (Ottawa was by far the most difficult, and did not provide me with enough information to calculate voter turnout, so that was removed from the ShareOttawa dashboard).

I’m not complaining about the need for this translation layer. I would much rather have a city make its data available than delay just to try to find a common format. But I do hope to see convergence over time. It would be great if these cities (and others that offer this data, like Vancouver) could agree on a common format. That would remove the extra work required to make an application reusable, and would hopefully result in even more applications that make use of the data.

And Beyond!

I’m pretty excited about the possibilities of open data, especially as it relates to politics and democracy. There are some really great apps being built, such as Vote.ca, and we’re really just scratching the surface. If we keep at it, I’m confident that open data will have a positive impact on the way we engage with politics.

City of Edmonton launches Apps4Edmonton Competition

By Mack D. Male ·

On Wednesday the City of Edmonton officially launched its Apps4Edmonton competition, which challenges residents to submit ideas for apps and developers to actually build them. There is more than $50,000 in prizes available, and everyone has a chance to win an Apple iPad. Here’s what the competition is all about:

Every Edmontonian can be an active participant in reshaping our city. Together with community partners the City of Edmonton is proud to launch Apps4Edmonton – a contest which encourages residents to develop applications that will benefit Edmontonians.

Using data sets from the Edmonton Open Data Catalogue, we’re challenging you to develop an app for either a smart phone, mobile device, or PC web browser. Mash up a map, create a visualization, or analyze data in a new way, the choice is yours.

To support the competition, the City has also released an update to the Data Catalogue. There are now 25 datasets available, including Ward boundaries and a list of City facilities. Even more datasets should be available soon, and if there’s something specific you’d like to have, request it from opendata@edmonton.ca.

If you have an idea for an app, submit it here. It can be anything, so be creative! Even if your idea isn’t possible or a developer doesn’t want to take it on, it might serve as inspiration for other apps, so don’t be shy. If you’re building or have already built an app, submit it here. You have until August 27 to do so. The FAQ has some good info, and be sure to familiarize yourself with the rules too.

Applications will be judged on four criteria: Accessibility, Usability, Creativity, and the number of community votes received (voting runs online from August 27 to September 10). In addition to three overall prizes (Gold, Silver, Bronze) there are six categories:

  • The Way We Live
  • The Way We Grow
  • The Way We Green
  • The Way We Move
  • The Way We Plan
  • The Way We Prosper

I’m very happy to see that IT has aligned itself with Transforming Edmonton for this competition! I am hopeful that Apps4Edmonton can help introduce more Edmontonians to those important documents that help guide us toward becoming a more innovative, inclusive, and creative city.

To learn more about Apps4Edmonton, visit the contest site. There should be lots of chatter about the competition at Startup Weekend too, and tonight at the Emerging Business Leaders’ Patio Party.

I can’t wait to see what everyone comes up with!

Edmonton Public Schools & Open Data

By Mack D. Male ·

Today I’m very excited to announce that Edmonton Public Schools has taken a big step into the world of open data by releasing a data set containing information on all of their schools, including the six opening later this year. I understand the data will be made available in the City of Edmonton’s Open Data Catalogue early next week, but you can download the CSV file today if you like.

Download the Edmonton Public Schools Data in CSV

Back in early February I met Jeremy and Paul for lunch to chat about open data and the community here in Edmonton. One of the things we talked about was how Edmonton Public Schools could get involved. In addition to attending events like the Open City Workshop, Jeremy and his team also started working behind-the-scenes to pull together data that might be useful to share. I helped define the fields that should be included and did the geocoding work, but they did all the rest. The result is a great data set of public schools in Edmonton, containing the name, address, lat/long, grade levels, programming information, and contact details for each.

Edmonton Public Schools follows in the footsteps of the Edmonton Public Library in embracing open data. Both organizations should be applauded for being “early adopters” and for their enthusiastic participation in the open data movement here in Edmonton. They have set an example that others can follow. Specifically:

  • Start small! EPL released branch locations, EPSB released school locations.
  • Work with the community! In both cases, I was able to help with some of the work. There are many others in the community who are eager to help as well.
  • Engage the City! In addition to getting the data in the catalogue, which is really important to have a central repository, the City has also offered some suggestions for improvements.

Thank you to Jeremy and the rest of the team at Edmonton Public Schools for making this happen!

No post about a new data set would be complete without mentioning that the data is now available at ShareEdmonton! You can now see a list of all public schools and on the details page for each one, you can see the relevant school ward, grade level, and programming information on the right side (for example, McNally, the high school I attended). More improvements coming soon!

Edmonton Neighbourhood Census Data

By Mack D. Male ·

For a long time I’ve wanted to get the City of Edmonton’s neighbourhood census data in CSV format (or really any usable format other than PDF). Recently, with the help of Laura (and Sandra) at the City’s Election & Census Services department, who I met at the Open City Workshop, I finally got it. And now you can have it too!

Download the Edmonton Neighbourhood Census Data in CSV

I’ve also emailed this to the City’s open data team, so hopefully they can get it in the data catalogue soon.

Visualizing the Data

Why is having the census data in a format like CSV useful? Well for one thing, it enables creatives to do stuff with that data through code or other tools. For instance, I was able to generate a heat map for the City of Edmonton:

The darker sections are more heavily populated, the lighter yellow regions are less populated.

Not all neighbourhoods are reflected, as the City does not release details for neighbourhoods with a population between 1 and 49. Here are some other things we can learn from the data set:

  • Total population in the data set is 777,811, which means there are 4628 individuals unaccounted for (total for 2009 was 782,439).
  • The average neighbourhood population is 2424, or 3039 if you exclude neighbourhoods with a reported population of 0.
  • The median neighbourhood population is 2216.
  • Oliver and Downtown are the only two neighbourhoods with a population greater than 10,000.
  • More dwellings are owned (192,171) than rented (121,953).

ShareEdmonton

Another reason having this data in CSV is useful is because app developers can more easily integrate it into the things they are building. For example, all the census data is now available at ShareEdmonton! So when you view a neighbourhood, you’ll see the census data on the right side (see Alberta Avenue for example). You can also browse neighbourhoods by population. I’ve also fixed the neighbourhood search, so it works better now.

This is just the first of a few neighbourhood-related updates this month, so stay tuned for more!

Apps4Edmonton

Yesterday the City released more information on the Apps4Edmonton competition. The first phase, from now until May, is “accepting community ideas”. Basically they want you to tell them what data you want. Aside from the obvious “we don’t know what we don’t know” problem, I think the community has done a pretty good job of defining desired data sets already.

They City had a great start in January with the launch of the data catalogue, but we need more data. Especially data like the census data, which myself and many others have been asking for since the day the PDFs were released. There are clearly some internal issues that need to be worked out if I was able to acquire this before the open data team was. I hope they get everything resolved for the competition, because it’ll be a pretty boring one if we still only have twelve data sets (New York and other cities had dozens, maybe even hundreds, before their competitions).

That said, I know there are passionate, smart people working on it. Email opendata@edmonton.ca if you have data set requests or want to get involved in Apps4Edmonton.

Recap: Open City Workshop

By Mack D. Male ·

Saturday’s Open City Workshop at the Art Gallery of Alberta was a fantastic event! Excellent turnout, great discussions, and lots of enthusiasm for the open data/open government movement here in Edmonton. We started the day with some opening remarks from Councillor Don Iveson, CIO Chris Moore, and FusedLogic’s Walter Schwabe. I really liked Don’s key values: transparency, empowerment, and collaboration.

Open City WorkshopOpen City Workshop

Next up was the panel, featuring David Eaves, Nick Charney, Mark Kuznicki, and Alayne Sinclair. They talked about what open government meant to them, took an odd detour into voting, and took some questions from the audience as well. Here are a few of the key things that stood out for me:

  • The idea of government as a platform is not new!
  • It’s about empowering citizens to make the city their own, enabling them to go after their passions.
  • Engagement is often a sign that you need to do something differently, not necessarily that you’re doing something right.
  • Civil servants are citizens too!
  • Open government is part of a broader cultural shift, and citizens have a responsibility to become participants.

I really enjoyed the panel – it was the highlight of the day for me. It was great to finally meet David, Nick, and Mark in person too.

After some forced networking and a break for lunch, the unconference part of the day got underway. We broke into smaller groups to discuss things like the role of journalism and storytelling in open data, the digital divide, thinking beyond technology solutions, and timelines/deliverables.

Open City WorkshopOpen City Workshop

Open City WorkshopOpen City Workshop

I attended the role of journalism and storytelling first, and we seemed to reach the consensus that storytelling is vital for open data/open government to succeed. We need everyone to tell stories, so that we achieve a diverse range of views. We also felt that journalism has a role to play in bridging the gap that exists between those who “get it” and those who don’t. The second session I attended was on engaging residents not familiar with technology. We ended up talking a lot about deliberative dialogue, and I was left with a lot to think about – how will access to open data affect the way a citizen looks at the world? We’re naturally selective, does open data change anything in that regard? For some thoughts on the other two sessions, check out Stella’s recap.

Open City WorkshopOpen City Workshop

Another really great aspect of the day was that it was streamed live online by FusedLogic in both English and French. There were dozens of people participating virtually, from Edmonton and around the world! I understand that they even held their own breakout discussion in the afternoon! Kudos to the FusedLogic team for taking on such a big task, and making it work so well.

At the end of the day, Chris Moore took the stage once again to make a few announcements:

  • An RFP has gone out for the design and implementation of the next generation of productivity technologies at the City of Edmonton.
  • The City of Edmonton is partnering on open311.org, to bring open standards for 311.
  • On April 19, the City of Edmonton is planning to launch an iPhone app called CityWatch, developed by local company Touchmetric.
  • The City of Edmonton is calling for the creation of a “Code for Canada” organization, modeled after Code for America.
  • There will be an Apps4Edmonton app contest in the near future, with the prizes and winners to be showcased at GTEC2010 in October.

Exciting stuff! Congrats to Chris and his team for putting on a great event.

Stay tuned to the #openyeg and #yegdata hashtags on Twitter for updates, as well as the City of Edmonton’s open city page. You can see the rest of my photos here, and you can see the City’s photoset here. You can see a list of the attendees on Twitter here.

I’ll leave you with this TED video of Tim Berners-Lee talking about open data:

Bring on the data!

Open City Workshop & Participation Inequality

By Mack D. Male ·

The City of Edmonton is hosting a workshop tomorrow at the Art Gallery of Alberta called Building Community through Open Information (on ShareEdmonton). It’s a follow-up to the open data workshop that happened last year and the conversations that have taken place since, but is focused on connecting stakeholders, increasing a shared understanding of Government 2.0, and planning the way forward. Devin wrote some great thoughts on the workshop here.

I’m really looking forward to seeing a diverse local crowd, representing a variety of agencies and interests. I’m also looking forward to meeting special guests David Eaves, Mark Kuznicki, and Nicholas Charney in person. All three are key players in Canada’s open government space.

And while I’m hopeful that we’ll see some new data released tomorrow, I’m mindful that data is not the focus. The three objectives listed on the event details page are about establishing Edmonton as a leader, and then lots of talk. I think the way you establish yourself as a leader is by actually doing things, but I’ll keep an open mind tomorrow :)

One thing I do want to talk about at the workshop is the divide that a number of people have picked up on. It’s great that Edmonton and other places are making progress on opening up data, but how does that impact the average citizen? There’s definitely a perception that only techies understand and can use open data. I don’t think that gives the average citizen enough credit, but I’m willing to concede that open data is not as accessible today as it needs to be. There’s lots of room for improvement.

Having said that, I think it’s important to keep participation inequality in mind:

All large-scale, multi-user communities and online social networks that rely on users to contribute content or build services share one property: most users don’t participate very much. Often, they simply lurk in the background.

If we think of open data (and open government) as a large community, then we should absolutely expect that a small subset of that community will be responsible for most of the activity. In the context of the 90-9-1 rule, 90% of the community won’t participate (they just observe or read), 9% will occasionally participate, and 1% will participate a lot.

Here’s one way to visualize the open community:

I’m sure there are other ways to break it down, but this makes the most sense to me at the moment.

  1. Goverati: these are government employees, folks from related agencies, non-profits, etc.
  2. Creative Professionals: these are people such as myself, developers, designers, etc.
  3. Business: for-profit organizations.
  4. Citizens: ultimately, the beneficiaries of all this open government stuff!

Of course, this picture is somewhat misleading, because all goverati, creative professionals, and business folks are also citizens, but let’s set that aside for now. I think the goverati and some creative professionals fall into the 1% category, the rest of the creative professionals and business fall into the 9% category, and citizens account for the 90% category.

The 90-9-1 pattern can be seen in action all over the web, perhaps most notably on Wikipedia. A tiny percentage of Wikipedia’s user base is responsible for the vast majority of all content produced. I think we can do better with open data/open government, however. Through applications, interactive visualizations, and the other interesting things that the goverati, creative professionals, and businesses build, I think more and more citizens will move from the 90% category to the 9% category.

I guess the point I’m trying to make is that considering how the average citizen benefits from open data is important, but we shouldn’t let that hold us back from making progress at this point. We need to empower businesses and creative professionals to build things that the average citizen will ultimately benefit from. As we do that, there’s an opportunity to educate citizens if they want to be educated!

There’s only a few hours left to register for the event. Hope to see you there!