Mountains of data, right at your fingertips

Last week, two announcements caught my eye. The first was from, which announced that there is now more than 1 TB of public data available to developers through its Public Data Sets on AWS project. The second was from the New York Times, which announced its Newswire API, providing access all NYTimes articles as they are published.

This is a big deal. Never before has so much data been so readily available to anyone. The AWS data is particularly interesting. All of a sudden, any developer in the world has cost-effective access to all publicly available DNA sequences (including the entire Human Genome), an entire dump of Wikipedia, US Census data, and much more. Perhaps most importantly, the data is in machine-readable formats. It’s relatively easy for developers to tap into the data sources for cross-referencing, statistical analysis, and who knows what else.

The Newswire API is also really intriguing. It’s part of a growing set of APIs that the New York Times has made available. With the Newswire API, developers can get links and metadata for new articles the minute they are published. What will developers do with this data? Again, who knows. Imagination is the only limitation now that everyone can have immediate access.

Both of these projects remove barriers and will help foster invention, innovation, and discovery. I hope they are part of a larger trend, where simple access to data becomes the norm. Google’s mission might be to organize the world’s information and make it universally accessible and useful, but it’s projects like these that are making that vision a reality. I can’t wait to see what comes next!

xkcd in the New York Times

As a kid I loved Calvin & Hobbes, but that was about the only comic strip I ever read. I think I have all the books except maybe one or two. I still don’t read comic strips, but I do occasionally enjoy reading xkcd, a “webcomic of romance, sarcasm, math, and language.” It was profiled in the New York Times yesterday:

The site, which began publishing regularly in January 2006, has 500,000 unique visitors a day, he said, and 80 million page views a month. (Why “xkcd”? “It’s just a word with no phonetic pronunciation,” his Web site,, answers.)

Here is one of my favorites:

Facebook defines relationships.

A new comic is posted every Monday, Wednesday, and Friday.

The author, Randall Munroe, is only 23 years old! He’s one smart cookie. He also writes a fairly interesting blog. I think this entry from November 2007 is my favorite:

I was thinking of getting a couch or something for my room, somewhere for guests to lounge around.

Fuck new couches. I now have a ball pit in my room.

That’s right, he has a ball pit in his room (click through for photos). Freaking awesome! And in February, he quintupled the size of it.

My congratulations to Randall on the coverage! Definitely check out xkcd if you’re looking for something fun and interesting.

Fortune fires up the photocopier for PayPal story

Fortune published an article today about "the hyperintelligent, superconnected pack of serial entrepreneurs" who left PayPal for bigger and better things; a group of individuals they have dubbed "the PayPal mafia". Founded in 1998, PayPal itself is fairly interesting, but the people behind it are downright fascinating!

As I was reading the article, I had the strangest sense of déjà vu. It was like I had read the article already! A quick search revealed that I had, over a year ago, at the New York Times. I just gave it a quick re-read, and it’s really amazing how similar the Fortune article is to the one that appeared in the Times last October.

Here are a couple examples. From the New York Times article:

Since 2002, when dozens of employees left PayPal after it was bought by eBay for $1.5 billion, those workers have gone on to start or join a new generation of Internet companies and other ventures. They have remained a tight-knit group, attending each other’s parties, helping to shape each other’s business plans, backing each other’s companies and recruiting each other for new projects.

Silicon Valley was largely built by networks of people and companies whose interlocking relationships help to spawn new start-ups. But the PayPal alumni have been unusually prolific…

And from the Fortune article:

Most of PayPal’s key employees left eBay, but they stayed in touch. They even have a name for themselves: the PayPal mafia. And the mafiosi have been busy.

During the past five years they’ve been furiously building things – investment firms, philanthropies, solar-power companies, an electric-car maker, a firm that aims to colonize Mars, and of course a slew of Internet companies. It’s amazing how many hot web properties can trace their ancestries to PayPal.

Again, from the New York Times article:

The company was losing millions each month. It was besieged by hackers who used technological trickery to siphon off huge sums from the company’s coffers.

And the Fortune article:

Meanwhile, PayPal losses were multiplying. It battled Russian fraudsters who were filching millions by cribbing credit card numbers.

See what I mean? Both stories follow the exact same formula, and touch on the exact same points. Of course this happens all the time in the media, but over a year apart? Seems kind of strange to me. Granted, the Fortune article does go into a bit more detail, but still.

It’s an interesting story, even if it has been written twice now. I was going to pull out the list of companies that former PayPalers have been involved with, but it has already been done at Wikipedia, of course. Facebook and YouTube are the heavy-hitters.

Both articles do a good job of detailing the tightly-knit group of individuals behind PayPal and many other startups. The topic I wish they’d follow-up on? How to break into that group.

Read: Fortune

NY Times article on Pownce made me laugh

Post ImageAfter writing my review of Pownce a few weeks ago, I figured I’d never write about the site again. However, after reading an idiotic article published in the New York Times yesterday, I knew I’d have to. Author Jason Pontin had me shaking my head right from the opening paragraph:

JUST now, the hottest startup in Silicon Valley — minutely examined by bloggers, panted after by investors — is Pownce, but only a chosen few can try out its Web site.

Hottest startup in the valley? News to me. Maybe three or four weeks ago. Anyway, let’s continue.

Within days, invitations were selling on eBay for as much as $10. Mr. Rose has declined all requests to be interviewed about the service, including my own. But as a consolation, he sent me a coveted invitation. I enjoyed the rare thrill of cyberhipness — and got to experiment with the site.

Coveted? Are you kidding me? Pownce tells me I have nine invites to give out. I’ve had them for weeks. I am positive I’m not the only one. Sorry Jason, receiving an invite to Pownce is anything but a hip cyber experience.

After some general information and background on Kevin Rose, Jason concludes that media executives should keep an eye on Pownce:

What struck me most was the site’s potential to be powerfully disruptive. Most file-sharing occurs on public sites, which can be monitored by media companies; if the users violate copyrights, the sites or the users themselves can be threatened into compliance or litigated out of existence (as happened with the original Napster). File-sharing on Pownce would be difficult to police.

If I didn’t know any better I’d think Jason was trying to make a joke. Because I sure laughed.

The RIAA has sued children, senior citizens, and everyone in-between. They’ve shut down company after company, and they’ve successfully petitioned ISPs for records detailing the activities of their subscribers. Somehow I don’t think policing Pownce (a system which knows exactly who is sharing what with whom, btw) would be a problem. Evidently Jason hasn’t heard of BitTorrent, which actually does make it difficult to police file-sharing (especially with the recent work done on protocol encryption).

I really wish the NY Times would stop publishing useless fluff pieces like this one.

I should mention that my main criticism of Pownce is set to be remedied soon – they are starting an API. Should be available in September, though the undocumented API that their desktop app uses has already been, um, documented.

Read: NY Times