The Human Face of Big Data: an unlikely subject for a great book

Big data is, like many trendy IT buzzwords, an increasingly nebulous term. The Wikipedia definition, for example, is rather jargonistic and impenetrable. If you read big data conference information you'll typically see a lot of naked commercial stuff that might be terribly important to bigwigs but perhaps looks a little... dry... for the layman.

Indeed, an awful lot of the hype around big data is very commercial in focus. At its heart, big data is concerned with how modern technology allows us to generate, store and process information on a massive scale. For example, Eric Schmidt, executive chairman of Google, said in 2010 "there were five exabytes of information created between the dawn of civilization through 2003, but that much information is now created every two days, and the pace is increasing." (one exabyte is a staggering 1,073,741,824 gigabytes.) As is so often the case in human endeavour, a lot of this ends up being about selling people things: think of Google's ad sensing network or Amazon's "people who bought this also liked..." engine, for example.

You might be forgiven for thinking that's not the most logical subject for a high-production-value coffee table book, but that's exactly what creators Rick Smolan and Jennifer Erwitt have produced in The Human Face of Big Data; the book is also available as a US$2.99 iPad app, and all the profits from the tablet edition will be donated to charity.

Through their crowd-sourcing firm Against All Odds and a team of more than 200 researchers, photographers, writers and illustrators, this is a project that aims to illuminate and explain the parts of big data that matter to people who aren't the CTO of a Fortune 500 company.

So we get writeups of earthquake detection systems in Japan; of Shwetak Patel's sensor devices that can accurately calculate how much power each individual device in your house costs (and help inform you about which devices to replace with energy efficient ones); and of Nick Felton's obsessive gathering of personal data from how many miles he walks to how many hours he sleeps each year. We learn about Intel-GE Care Innovation's "Magic Carpet" prototype, which is a passive sensor net woven into the flooring of an elderly person's home that can learn the person's habits and routine and alert a relative or caregiver if it suddenly changes -- say, the person can no longer walk as fast, or starts spending long periods in bed. We hear from researchers John Guttag and Collin Stultz, who processed discarded EKG data of heart attack patients and identified subtle new early warning patterns to improve doctor's risk screening.

It's full of interesting things, then, and it makes a good case that big data could be the first step towards the Internet developing a "nervous system" of sorts; a detailed sensor network generating reams of data, plus the ability to meaningfully process and act on that data in real time. You may now jump to the comment box and make a Skynet joke.

It's worth pointing out that this is a most certainly a coffee table book, rather than an in-depth treatise, and as such it's more about the imagery than it is about the text. Most subjects get only a brief overview of a few hundred words, punctuated by some short essays of 1,500 words or so. This isn't the place to go for a lot of detail on each individual project, although of course most of them are covered in detail elsewhere on the web.

The book is going to be delivered free to 10,000 "key influencers" around the globe, as part of Smolan and Erwitt's mission to "start a global conversation about Big Data, and who owns the data all of us generate it." Indeed, one of their concerns is that most of the conversation around big data is being driven by commercial interests, but it inherently affects all of us -- it is, in a very real way, made of us -- and this book attempts to redress that. It's a noble goal, for sure.

The Human Face of Big Data is available in book form from Barnes & Noble internationally and from Amazon and IndieBound in the US. It costs around $35 and (in my opinion) would make a nifty gift for any CTOs you just happen to have in your social circle. The photography is attractive and enticing, the infographics are informative and in general it's the sort of book you flick through then end up reading half of as one thing after another catches your eye.

The iPad app should be available now for $2.99, with all profits going to charity: water. It has content rather like most iPad magazine apps -- swipe to page through the book, scroll up and down to read each article, tap on various zones in some pictures to drill down into the detail -- that sort of thing.

It's a nice app that uses the iPad Retina display to show off the great imagery from the print book, although inevitably some of the impact is lost in the transition to a much smaller canvas (the book measures 14 x 11 inches). Notably, the book also seems to have quite a bit more content -- partly, I think, because some of the more detailed illustrations like the stunning BibleViz (my personal favorite) won't scale down to the iPad's relatively small screen.