TMI – Information Overload

I’m a reader. I tear through a magazine in a couple of hours, reading, consuming, and drinking in information. I like to discover what others are thinking, what topics are hot, what’s happening in the greater world…outside of our little burg of Nashville. I surf industry related blogs. I subscribe to several business related magazines. I try to blaze through a good fiction novel every other week or so. I consume information. I’m a reader.

 

Boarding a plane for Jacksonville, FL in March, I picked up a copy of "The Economist," a magazine that appeals to the marketing nerd in me. The cover of the February/March, 2010 issue bore, in a typically inoffensive British font, the title, “The Data Deluge.” So I boarded my flight, eager to learn what this uber-smart Anglo rag had to say about Too Much Information.

The first example of exactly how much data we’re dealing with came in the first sentence of the first paragraph of this 14-page special report: “When the Sloan Digital Sky Survey started work in 2000, its telescope in New Mexico collected more data in its first few weeks than had been amassed in the entire history of astronomy.” A newer telescope in Chile, scheduled to come online in 2016, will gather, “the same quantity of data every five days.”

 

What to do with all this data? (Or is it “these” data?)

Bytes, megabytes, gigabytes, terabytes, we’re even up to petabytes (seriously??) at the commercial and government level, and world-wide there exist today about 1.2 zettabytes of raw data stored (1,000 times 10 billion copies of The Economist magazine). Yottabytes are just around the corner (no one has come up with a way to imagine this number yet.) The point is, there’s a lot of information out there and it’s growing…fast.

However, data is not knowledge. In its now rare singular form, the word “datum” means “a piece of information” – in other words, a single observation or occurrence. “Data,” as a plural noun or singular mass noun (the latter being more commonly used), refers to items of information or a body of information.

If you’ll bear with my brief epistemological rant, in the end, it’s really not data we’re after, but knowledge. Like the hyper-intelligent, pan-dimensional beings of Douglas Adams’s “The Hitchhiker’s Guide to the Galaxy” series, we often doggedly pursue the former, all the while hoping for the latter to magically spring forth, like Athena from Zeus’s head.

If you’re new to the “HHTTG” series, the pan-dimensional beings create a supercomputer called “Deep Thought” to calculate the “Answer to the Ultimate Question of Life, the Universe, and Everything.” After 7.5 million years, Deep Thought spits out the answer: which is, simply, 42. Deep Thought explains, in typically sci-fi earnest computer logic, that the question is really the crux of the issue, and that he is unable to compute it.

Knowledge requires a human element. The word's definition includes a nod to a familiarity with facts or data, but it suggests something greater than mere information storage. It suggests understanding, which can only be gained by experience.

Knowledge is the narrative we construct to tie together and explain the data we’ve collected. Which means…sometimes it’s wrong.

 

Lies, Damn Lies, and Statistics

The highlight for me of that issue of The Economist was the phrase, “Torture the data long enough and they will confess to anything.” I had a nice chuckle over that one, and then it set me to thinking about the potential pitfalls of collecting all this information. A computer may be able to store yottabytes of the stuff, but we humans possess a far inferior capacity to comb through it all, figure out what it means, and translate it into something coherent. To turn it into knowledge, in other words.

For a case study in too much information and a complete human failure to turn it into knowledge, let’s look to Northwest Flight 253, in which Umar Farouk Abdulmutallab attempted to detonate his crotch with a DIY thong bomb. In the post-operation analysis, it came to light that his parents had warned US officials that their odd son likely posed a threat to US national security. Those officials keyed his name into their huge database, containing over 550,000 people who “posed a threat.”

The database is rife with redundancy, typos, mis-filings, and it’s not at all unusual for a single name to get lost in the dross. No one connected the dots. No one really knew how. Robert Jervis, of the Boston Globe wrote in a January 2010 article, “The problems with our intelligence system aren’t primarily problems with information; they are problems with how we think.” They are problems with knowledge, and how we create it.

Jervis contends that we, as human beings, tend to “…excel at perceiving patterns and making up stories that bring the coherence that we need in order to act.” This seems like a good thing, a survival mechanism, an innate ability to quickly get our minds around a problem and do something. Problem is, once we see a pattern, we tend to get kinda stuck on that idea. We “see patterns quite quickly and then tend to ignore information that might disprove them.” This tendency to get stuck on an idea is known in psychological circles as “premature cognitive closure.” This is the exact type of intelligence process that lead the CIA into “knowing” that Saddam had weapons of mass destruction in Iraq, when, as we all know now, there were none.

One interesting observation that Jervis makes about the WMD snafu in Iraq is that the analysts ignored the fact that a substantial amount of information that should have been there, had Iraq been in possession of WMD’s, was missing. Think Sherlock Holmes making note of the dog “not barking” as the most important clue in a case.

The final point that Jervis makes is that the CIA’s conclusions, “often rest on assumptions that are not readily testable,” assumptions that may even be immune to dispute. If you assume that the lack of evidence pointing to WMD’s is a massive deception or cover-up on the part of Saddam’s regime, then the WMD program, like any negative, becomes virtually impossible to disprove.

Interestingly, people expect the government to have a perfect understanding of what the bad guys are doing, all of the bad guys at once, and yet people are, as Jervis notes, “often surprised to discover that their own spouses have been cheating on them.”

 

Data, Knowledge, and the Private Investigator

Fortunately, we private investigators usually don’t have to sift through yottabytes of data to identify suspected miscreants from a list of thousands. Our clients often supply us with that information up front, leaving us free to focus on a single target or person of interest. If we’re lucky, the client may be able to help paint a picture of the target’s activities and lifestyle, such a favorite bar or hangout or a subject’s typical daily schedule.

Our data gathering looks a little different (and more mundane) than the CIA’s does. We wade through records, pull trash, surf the usual social networking haunts, spend hours, days, or weeks quietly observing a subject’s behavior and activities, and we organize all of that intelligence for you.

For one case a few years back, we assembled two three-ring binders full of well-organized receipts, printed web pages, business cards, and personal correspondence, all pointing to the fact that the client’s spouse was having an extramarital relationship.  In addition to observing the subject (and paramour) and documenting their behavior with video and photographs, we were able to show our client (and subsequently the mediator) a certain amount of money that the cheating spouse had been spending on the paramour over the past several months. How much impact do you think that had on alimony?

Mostly, a private investigator’s job is just what I’ve described: to collect and organize data for his clients. But sometimes, a client needs something more: knowledge.

Often, our clients have hired us because they’ve found themselves in a bewildering and painful personal situation: a betrayal of some kind, by a spouse or business partner, say. These are situations in which the human factor, that magic x-factor that interprets data and sculpts it into a story that enlightens us, often leads us astray. When people are in the kind of emotional state that usually accompanies betrayal by a spouse, friend, or colleague, it’s particularly difficult to interpret facts objectively; we flawed humans tend to torture the information until it tells us what we want to hear, or what we have already convinced ourselves is true.

That’s why sometimes, a distraught client might need a little help on the knowledge front – an independent, experienced, objective third-party observer who can offer alternate interpretations of the facts at hand, uncolored by personal interest or emotion. A private investigator is your very own personal operative, the person you can rely on to gather the pertinent information and help you put it together in a meaningful way.

As PIs, we have to be very cautious about offering interpretations and opinions. However, it is certainly our job, and our responsibility, to test and question our clients’ assumptions, and to sometimes serve as our clients’ guide through the difficult and painful minefield of turning facts into knowledge. And it’s absolutely our job to be sure that we gather the facts necessary to support that knowledge beyond any reasonable doubt or refutation.