Tag: data

Edmonton Sun violates the EPS Crime Map Terms of Use

Back in July, the Edmonton Police Service launched its Neighbourhood Crime Mapping site. Like most people I was quite enthusiastic about the site, until I read the terms of use and realized how restrictive they were. Basically you can look at the numbers, but you can’t do anything with them (such as publish them on a blog). The Crime Mapping site is not open data. I emailed back and forth with the EPS, and was told that they wouldn’t be changing the terms of use. And, they haven’t.

That didn’t stop the Edmonton Sun, however. They apparently ignored the terms of use altogether, and published an article on December 20th summarizing a number of statistics from the website:

Some of Edmonton’s roughest neighbourhoods faced markedly fewer crimes in 2009, according to police statistics.

The statistics came through a new crime mapping system launched by Edmonton police last summer.

I had asked for permission to do something similar and was turned down. After reading the Sun article, I emailed the EPS to find out if the terms of use had been changed (despite the text on the website staying the same). Here’s what Acting S/Sgt. John Warden wrote back:

The Edmonton Sun did not have the EPS’ permission to use the information from the Crime Mapping website and the EPS is dealing directly with the Edmonton Sun in relation to this.

I emailed back a couple of follow-up questions, but have not yet received a response. The Edmonton Sun article is still active on the website, so I’m not exactly sure what “dealing directly with the Edmonton Sun” means.

I’m annoyed by this, obviously. Was it an honest mistake? Maybe. Is it a case of a large media organization getting off the hook? Maybe. Will it happen again? Probably. No one reads the fine print, we all know that.

I don’t think the current terms of use is appropriate, and I strongly urge the Edmonton Police Service to change it.

Canadian Finals Rodeo (CFR) Attendance Numbers

The 2009 Canadian Finals Rodeo wrapped up on Sunday, and although attendance was down from previous years, it was still pretty good. I’m always disappointed, however, when the press release or media article comes out and compares attendance only with the previous year, or sometimes with the record year. I’m often more interested in trends, and in comparing with other events. Slowly but surely, I’ll gather all of the data to make that easier! So far I have:

And now, I have some data for CFR. Here are the attendance numbers for this year compared with last year:

Day 4 is the Saturday, and is always higher because there are both matinee and evening events. Here are the attendance numbers from 2005 to 2009:

As you can see attendance peaked in 2006, the record year for CFR.

Download the 2008/2009 attendance data in CSV

Download the 2005-2009 attendance data in CSV

Open Data at ChangeCamp Edmonton

Tomorrow morning local politicians, bureaucrats, and ordinary citizens will gather at the University of Alberta for ChangeCamp Edmonton. I’m encouraged by the number of people that have registered, and by the conversations that have already started. That’s what tomorrow is all about: getting people together to discuss ideas and solutions.

I don’t know exactly which topics people will want to discuss tomorrow, but I know for sure that open data will be one of them. There’s significant momentum building for the concept, and we’re starting to see progress on making it happen throughout Canada (and elsewhere).

Open data here in Edmonton received a nice boost this week from Councillor Don Iveson when he submitted a formal inquiry to City administration:

In local, national and sub-national governments around the world there is a trend toward making up-to-date government information freely available on-line in generically accessible data formats as so-called ‘Open Data’.

  1. What level of awareness does the City Administration have regarding Open Data in municipal government?
  2. What current initiatives are underway within City Administration that might qualify under the spirit of Open Data?
  3. What further initiatives are under consideration within the city, and on what basis are they being evaluated?
  4. Is Administration monitoring any successes and or challenges with this trend in other jurisdictions, especially large Canadian cities, and if so what can be shared with Council?
  5. What would City Administration’s recommendation be on next steps regarding Open Data plans or strategies?

I know there was already some things going on behind the scenes at the City of Edmonton, but Don’s inquiry should expedite and give credibility to those things. This is an important step.

I’ve been pushing for open data in Edmonton for a while now, along with many others. I think ChangeCamp will be a great opportunity to further discuss the concept and next steps. I generally think about open data in the context of a municipality, but there’s room for discussion at the provincial and federal levels too. Here are some of the key things I think we can cover:

  • Let’s make sure everyone (citizens, politicians, City administration) is on the same page about what we mean by “open data”. This could be high level (what kinds of data are open) and low level (what formats are considered open).
  • What is the City working on? What are citizens working on? Let’s get a status report from both sides.
  • What kinds of data could be made open? Which data is most in demand by citizens? What data has been made available in other cities, such as Vancouver or Toronto?
  • Licensing is vital for open data to work. We need to ensure data is licensed as permissively as possible, otherwise we’re restricting its utility. Which licenses make sense? What have other municipalities used?
  • Often lost in the discussion about what data to make available is how to be notified of changes to that data. RSS feeds, email subscriptions – how should citizens be notified when data is updated or otherwise changed?
  • Another aspect that we need to consider: the creation of data. There is lots and lots of data that our governments can start making available in open formats, but there’s even more data created on a daily basis. What can we do to ensure that it is open data also? How about APIs or other mechanisms for citizens to provide input/data? Open 311 comes to mind.

Here are some links that might be useful tomorrow:

See you in the morning!

Open Data in Edmonton? Follow Vancouver’s lead

Last week Vancouver launched an open data portal, providing one-stop-shopping for open data provided by the city. David Eaves called the launch “a major milestone for Vancouver” and explained:

The Data Portal represents an opportunity for citizens, especially citizen coders, to help create a City that Thinks Like the Web: a city that enables citizens to create and access collective knowledge and information to create new services, suggest new ideas, and identify critical bugs in the infrastructure and services, among other a million other possibilities.

He was also quick to point out that getting access to the data is just the beginning. Citizens have to use it, or risk losing it. The next day he launched VanTrash, an application to make garbage collection sexier. Use it or lose it indeed!

I think it’s interesting that he started with garbage collection, because I too identified that as an area that could use some innovation. A couple months ago, I spent about an hour on the phone with a manger in the Waste Management department at the City of Edmonton, trying to get access to the data behind the garbage collection schedules. Currently you can enter your address here to download your collection schedule in PDF. But if you want to find the schedule for a different part of the city, you’re out of luck. And even if you manually tried enough addresses to find all the zones and collection schedules, they’d be in PDF, which means you can’t easily add them to a calendar.

By the end of the call, I think he finally understood what I was after, and he said he’d have to get back to me. He never did, unfortunately. I can only hope that my request had an impact and that it will eventually help to open the data floodgates in Edmonton.

Open Data doesn’t have to be difficult!

Take a look at the data available at Vancouver’s data portal. Most of the data there is simple and exists elsewhere, in a less “creative friendly” format. A good example is the list of libraries. You can download the data in CSV, XLS, or KML formats, but it really just comes from the Vancouver Public Library website. The CSV contains the library name, it’s latitude, longitude, and address. Simple stuff, but potentially really useful if combined with other data sets.

Here’s an example in Edmonton. Let’s say I want to know how the crime rate of neighbourhoods with libraries compares to those without. What data would I need for that?

  • A list of libraries, with their locations (see below)
  • A list of neighbourhoods, with their boundaries
  • Crime statistics by neighbourhood
  • Census data for neighbourhoods to find comparable ones without libraries

Could you find this today? Yes, but it’s definitely not easy! The EPL website lists the libraries with addresses, so you’d need to figure out the lat/long on your own. The City of Edmonton website lists the neighbourhoods, but you’d need to figure out the boundaries on your own. The EPS website provides reported crimes by neighbourhood. And finally, the City of Edmonton provides census data for neighbourhoods in PDF.

If I could get all the above data in CSV format, it would have taken a matter of minutes to find the answer (I should point out that not all of that data exists at Vancouver’s portal either). Instead, I had to do a lot more work. The very rough result (because I compared with a random sample of similarly populated neighbourhoods) is that neighbourhoods with libraries were 1.5 times more likely to have crime than neighbourhoods without libraries in 2008. Though if you don’t count Downtown, then the crime rate is about the same for neighbourhoods with libraries and those without.

Maybe you’re thinking “what a useless example” and that’s fine – it is one of just hundreds or thousands of possible uses for that data! Just imagine what would be created if software developers and other creatives in Edmonton had access to the data.

Libraries Data

All this talk of open data, why not give you some? I’ve created a CSV of the Edmonton Public Library locations in the exact same format as the Vancouver Public Library data (minus eplGO in the Cameron Library). Enjoy!

Download the Edmonton Public Library location data in CSV

Onward in Edmonton

I’ve heard rumblings that the City of Edmonton will be doing some stuff in the open data space in the next couple of months, but I’m not holding my breath. There haven’t been enough conversations taking place. I’m hopeful that the right people are envious of the progress that has been made in Vancouver, however. I sure am!

Attendance Numbers for Edmonton’s Capital EX

Edmonton’s Capital EX wrapped up yesterday. Sharon and I visited on Thursday evening and had a good time. Today Northlands released the attendance numbers, and though slightly lower than previous years, the ten-day festival still recorded an impressive 717,966 visits. I had been looking forward to the final numbers, so that I could compare it with previous years.

Here are the attendance numbers for the last ten years (you can download the raw data below):

Though much of the data is missing, I was able to track down some numbers going all the way back to 1879:

After getting this information, I decided to compare it to the population of Edmonton for the same time periods. Here is the comparison for the last ten years:

And the same comparison starting in 1879:

 

A couple things to note about the data in this post:

  • The event changed from Klondike Days (adopted in 1962) to Capital EX in 2006. This explains the large drop that year.
  • The event was a six-day fair from 1912 to 1967, and a ten-day fair thereafter (I think, certainly for the last 20 years or so it has been). I haven’t adjusted the figures for this.
  • The population data, which comes from the City of Edmonton, doesn’t account for surrounding communities.

Download the Capital EX Attendance & Edmonton Population data in CSV

Sources: iNews880, CBC, Edmonton Journal, Amusement Business (1, 2, 3, 4), City of Edmonton, Capital EX Fair History

EPS responds to my query about the Crime Mapping terms of use

Last week I posted about the new Crime Mapping site launched by the Edmonton Police Service (EPS). One of my criticisms of the site was the very restrictive terms of use or disclaimer that you must agree to before you can use the site. In particular:

While it is acceptable to pass the website link on to others in your community, you will not share the information found on the website with others other than with members of the Edmonton Police Service or other law enforcement agencies; and

You will only use this website and the information in it so you can inform yourself of, and participate in, this community policing initiative;

This is problematic because it effectively means that you can’t do anything with the data that EPS has now made available. You can look at it using their site, but you can’t then blog about that data, or add it to a PowerPoint presentation.

I emailed a request for clarification and received a response from Amit Sansanwal, Criminal Statistics Coordinator at EPS. I asked for and was granted permission (by their legal department) to publish his response:

The EPS views the Neighbourhood Crime Mapping website as a valuable addition to our community policing initiative.

The EPS, however, is of the view that this tool can only be effective and achieve its community policing objectives if people seeking the information visit the Neighbourhood Crime Mapping website directly themselves.

By visiting the website, hopeful participants in this EPS community policing initiative can learn about what kind of information is available to them (e.g. crime prevention and partnership programs) and how it fits within this program.

We appreciate your interest in this program and hope that you tell others about the existence of the Neighbourhood Crime Mapping website.

In a later email, Amit pointed out that the current preferred way to get EPS statistics is through Statistics Canada.

The crux of their position, if I understand it correctly, is that they don’t want people looking for crime statistics to come across an inaccurate or malicious source. That seems reasonable. The problem is that such a position assumes people are actively seeking the information. By opening up access to the data and allowing others to make use of it, they can potentially reach far more Edmontonians, not to mention the benefits that could come from mashups or other data visualizations. Furthermore, it seems as though they just want to force people to use the Crime Mapping site so that they can promote additional programs to users.

The Crime Mapping site is fun to look at, but I would argue its utility is restricted by the current terms of use. Unfortunately, it doesn’t look like that’ll be changing any time soon.

Data on Edmonton’s new 12-ward system

Last night City Council voted in favor of changing from the current 6-ward system to the more common 12-ward system used throughout North America. The change will take effect for next year’s municipal election. For more background, check out Dave’s post. You can also check out the City of Edmonton’s page for more information.

As an advocate of open data, I thought I’d share with you some data related to the new wards below. All of the data is available on the City website somewhere, but not in an easily consumable form. I’ve done the legwork to make it accessible.

Amendments made to the motion last night affected the wards a little:

  • Grovenor and McQueen neighbourhoods moved from Ward 1 to Ward 6.
  • CPR West moved to Ward 8 from Ward 10.
  • Calgary Trail North and Calgary Trail South moved from Ward 11 to Ward 10.
  • Some ravine boundaries were changed from “in-the-middle” to “top-of-bank”.

Here are the stats on the new wards:

In a table (download CSV file here):

Ward Population Electors
1 62,625 51,061
2 67,306 54,704
3 63,819 49,465
4 67,811 52,666
5 62,424 49,615
6 70,840 62,152
7 63,549 51,865
8 66,196 57,189
9 68,214 53,889
10 61,276 49,935
11 64,770 51,329
12 63,609 48,529

The average population of each ward is 65,203 and the average number of electors for each ward is 52,700. This data comes from the 2009 Municipal Census.

Here are the number of neighbourhoods in each ward:

I’ve also compiled a list of neighbourhoods in each ward which you can download in CSV here. Or if you’d rather just look, you can download the list in PDF here.

I’m trying to track down or create a good quality map of the 12 wards, but this’ll have to do for now. What I’d really love is lat/long coordinates for each ward. If you have something better than that graphic, let me know!

Go do something useful or interesting with this data, and then tell me about it. I’m looking to collect local examples to strengthen the case for open data at the City of Edmonton!

UPDATE: Here’s a better map in PDF format.

UPDATE2: Here’s an even better color map showing the wards and neighbourhoods in PDF format.

Calgary takes first steps toward becoming an Open City

A motion will go before Calgary’s City Council next week that outlines the first steps in the process of making Calgary an Open City. Calgary follows in the footsteps of Vancouver, which passed a similar motion back in May. DJ has all the details on the Calgary motion here. I think it’s pretty cool that the news is first announced on a blog!

Calgary’s motion will result in a report from City Administration to be presented to Council no later than December 2009, outlining the overall strategy for making Calgary an open city. In particular, the report will identify “opportunities to make more of The City’s data open and accessible while respecting privacy and security concerns , and ensuring that data is available through use of open standards, interfaces and formats.” Other aspects of the strategy will include increasing online citizen participation, procuring and supporting open source technologies, and increasing the number of City services available online.

This is exciting news for developers and other creative professionals in Calgary and elsewhere. I’ve been pushing for open data in Edmonton recently, and I really hope we’re not too far behind our southern neighbours on this issue. There are a number of advantages to making data available in open standards and formats:

  • Citizens can subscribe to data that is of interest to them
  • Data can be mashed together in new ways, revealing new information
  • Visualization of data can help citizens make better decisions
  • Citizens can work together to organize data
  • Government can learn more about its data from citizen contributions

Additionally, using well-understood, open formats such as XML or CSV helps to “future-proof” the data. You don’t need proprietary technology to read a CSV file – any programming language or software platform will work.

One issue that isn’t mentioned in Calgary’s motion but which is very important, is licensing. It’s important that when Calgary does make data available, that it does so in the least restrictive way possible. Either public domain, or creative commons, or something similar. It would be a shame if they made a ton of data available and then had ridiculous terms of use around it.

Open data is about empowering citizens to work with their governments. I’m encouraged by the recent interest among municipalities in Canada, and I hope the trend continues.

Edmonton Police Service (EPS) Crime Mapping tool now online

Back in June we learned that the Edmonton Police Service was planning to launch a new website that would enable citizens to find crime statistics for their neighbourhoods. This afternoon, the EPS Crime Mapping tool went online, and it does just that. You can search for stats on eight types of crimes in any neighbourhood across any time period since 2007. From the press release:

The new crime mapping tool will provide members and citizens with a better understanding of what is going on the neighbourhoods they work and live in.

I’ve been playing with the site today, and I like it. There are pros and cons, however.

How It Works

The first step is to agree to the disclaimer – more on that in a minute. Next, you pick the crimes you want statistics for. The eight types include assault, break and enter, homicide, robbery, sexual assaults, theft from vehicle, theft of vehicle, and theft over $5000. Third, you pick the neighbourhood – there are 357 listed in the system. Finally, you select the time period. There are some quick selections such as yesterday or the last 30 days, or you can enter any two dates. Click “Show Crimes” and your neighbourhood appears on the map, covered in colored dots to represent the reported crimes. Here’s what Oliver looks like for the last 30 days with all crime types selected:

There’s also a “View Statistics” tab above the map that will show you a table for the last three years broken down by month, with a graph below that.

The Good

There are some really good things about this site. First and foremost, the data is excellent. I’m glad that they included everything up-front, instead of doing a test release or something to start. Second, it’s built using Google Maps. This is a big win for EPS – it’s a stable technology that Google is continually making better, and I would guess that most Edmontonians are familiar with it. Third, it’s fast. Almost as soon as you click the button, your stats appear.

The Bad

There are two things about the site that I don’t like. First is the disclaimer – it’s too restrictive. These two points in particular are problematic:

While it is acceptable to pass the website link on to others in your community, you will not share the information found on the website with others other than with members of the Edmonton Police Service or other law enforcement agencies; and

You will only use this website and the information in it so you can inform yourself of, and participate in, this community policing initiative;

That effectively means you can’t do anything with the data. This is in direct contrast with what the press release would lead you to believe:

Providing our citizens with the real picture of neighbourhood crime is the first step in engaging them to do something about it. Members of the public will be better equipped with knowledge to work collaboratively with the EPS to reduce and prevent crime.

What’s the point of making the data available if you can’t do anything with it? Why can’t I blog about the crime stats in a particular neighbourhood? Or mash the crime stats up with some other data? I challenge the notion that simply being able to see the dots on a map equips me to do something about crime in my neighbourhood.

I’ve emailed the feedback address listed on the site asking about this, but I haven’t yet received a response.

The second bad thing about the site is that while it does make data available, it does so in an opaque and closed way. If Edmonton is going to become an open city (with respect to data), sites like the crime mapping tool need to provide information for multiple audiences. One is the average citizen who is happy to click around on the map. Another increasingly important audience is the creative professional who wants to do something with the data, and needs it in a machine-readable format such as a CSV or XML file.

The Undocumented API

The first thing I did after testing the site with my neighbourhood was poke around for clues about where the data comes from. It didn’t take long to realize that there’s a JSON web service behind the application. You can access it here. It’s probably not meant for public consumption, but it’s there and it works. I was able to throw some code together in about 30 minutes to get data out of the service. While it would still be good to have static data files available, the API largely negates the con I mentioned above. As it is unofficial however, who knows if it will remain active and working, so enjoy it while you can.

Final Thoughts

Overall I think the Crime Mapping tool is excellent. We need more applications and services like this, though with less restrictive terms/licensing and easier-to-access data. Kudos to EPS for building this, and let’s hope they improve it.

UPDATE: There are more details in this article. For instance, the tool apparently cost $20,000 to build, and is automatically updated each morning.

Foundations for an Open Edmonton

Today at BarCamp, I led a discussion around building an open Edmonton. Inspired by the great things happening in Vancouver, I wanted to stimulate the discussion here. I started with two fundamentals:

  1. The City of Edmonton must have the desire to be an open city.
  2. The primary audience is the Creative Class of Edmonton, the secondary audience is all citizens.

Next, I shared what I feel are the five basic foundations of an open city:

  1. Free – both financially and philosophically
  2. Permissive Licensing – things like Creative Commons, should be public domain
  3. Open Standards – formats that anyone can read and write
  4. Plentiful Data – make as much data available as possible
  5. Timely Access – eliminate delays and give everyone equal access

After my five slides (a photo for each of the above) we got into a great discussion about the idea. Here are some of the questions that came up:

  • Are citizens ready for so much data?
  • Why would City Council not want to be an open city?
  • What is the current state of progress on the idea in Edmonton?
  • How does privacy & security factor in?
  • What are some great examples of other cities doing this?

All things that we need to explore further. I’m not sure what the next step is, but eventually, I think it would be great to make a presentation on becoming an open city to Council.

In the meantime, Edmonton has already made some data available – a Google Transit data feed – and some other examples include London’s mySociety. Also, be sure to read Vancouver’s Open City Motion.