Mountains of data, right at your fingertips

Last week, two announcements caught my eye. The first was from Amazon.com, which announced that there is now more than 1 TB of public data available to developers through its Public Data Sets on AWS project. The second was from the New York Times, which announced its Newswire API, providing access all NYTimes articles as they are published.

This is a big deal. Never before has so much data been so readily available to anyone. The AWS data is particularly interesting. All of a sudden, any developer in the world has cost-effective access to all publicly available DNA sequences (including the entire Human Genome), an entire dump of Wikipedia, US Census data, and much more. Perhaps most importantly, the data is in machine-readable formats. It’s relatively easy for developers to tap into the data sources for cross-referencing, statistical analysis, and who knows what else.

The Newswire API is also really intriguing. It’s part of a growing set of APIs that the New York Times has made available. With the Newswire API, developers can get links and metadata for new articles the minute they are published. What will developers do with this data? Again, who knows. Imagination is the only limitation now that everyone can have immediate access.

Both of these projects remove barriers and will help foster invention, innovation, and discovery. I hope they are part of a larger trend, where simple access to data becomes the norm. Google’s mission might be to organize the world’s information and make it universally accessible and useful, but it’s projects like these that are making that vision a reality. I can’t wait to see what comes next!

Twitter and the future of journalism

twitter logo On Friday, David Schlesinger from Reuters posted an entry to his blog discussing Twitter and the future of journalism. David had been tweeting from the World Economic Forum, and found that his updates beat the Reuters newswire. No big surprise, but it prompted a bunch of questions:

Is it journalism?

Is it dangerous?

Is it embarrassing that my tweets even beat the Reuters newswire?

Am I destroying Reuters standards by encouraging tweeting or blogging?

David’s answers are: Yes, Potentially, No, and No. I love that someone in the “mainstream” media is able to answer these questions honestly and openly!

His entire post is filled with wonderful quotes, such as:

I have no idea what journalism will look like in five years except that it will be different than it is now. That’s a great thing, I believe.

Fantastic outlook on things, in my opinion. Twitter is changing journalism and news media for the better. Ignore it at your own peril. I can’t say it any better than David himself:

If I don’t beat the Reuters wire with a live tweet because I deliberately hold back, someone else will. If I don’t beat the Reuters wire because I’m slow or inattentive, someone else will.

There’s something to be said for holding back, providing context, and thoughtfully articulating a story or idea. But when you’re talking about breaking and spreading the news, speed is the name of the game, and nothing at the moment does it faster or more effectively than Twitter.

Another way to look at it is chunking. Give me bite-sized chunks as the news happens, and when I have time later, I’ll explore the topic in more detail. Twitter is perfect for chunking news.

Take a couple minutes and read the entire post. I’m looking forward to reading it again in a year – I suspect it won’t seem so unique and refreshing by then!