Big Data, Meet Long Data, Meet Blog Data
Big Data Meet Long Data by Jeff Bertolucci - @jbertolucci - column appears this week in InformationWeek to reminds us that "Long Data" or historical data is vital for analysis and comprehension of trends that span years.
Bertolucci's article links to a Wired article from January 2013 titled Stop Hyping Big Data and Start Paying Attention to ‘Long Data’ where the author, Samuel Arbesman - @arbesman - says "Big data puts slices of knowledge in context.
Bringing this home to a company's context, Bertolucci quotes Benjamin Bruce (Pitney Bowes - Marketing Director) as saying "Big data is more about taking a slice in time across many different channels" and that "long data involves looking at information on a much longer timescale.
"Over the last decade as I have worked on homeland security, we have had the chance to run some experiments that show that raw field intelligence has more utility over longer periods of time than intelligence reports written at a specific time and place.
In other experiments, we have demonstrated that narrative assessment of a battlefield picks up more weak signals (those things that after the event you wished you had paid attention to) than analytical structured thinking. "
He continues with explanation: "we live in a world subject to constant change, and it’s better to blend fragments at the time of need than attempt to anticipate all needs.
So when you leave data on the chopping block after completing an analysis, you are denying the next person an opportunity to go back to the raw data and run their own analysis, possibly for the same or different purposes.
I'll assert that what's needed is the thin slicing in big data concepts combined with the long data trends that allow for understanding change and gathering some picture of the future.
Bringing this back to "blog" data - thats where we can capture the vital narrative that Snowden says carries the weak signals.
Blog data in a TeamPage demo also offers a simple and easy example to explain the importance of thin slicing over long trends.
Now, when we use a date selector control, we can see it tells a very different story in 2012 vs.
|Date Selector: Year 2010||Date Selector: Year 2011|
In the example, you can see the company's attention has shifted from competitors like Nike and Vibram in 2010 to competitors like Merrell in 2011.
This sort of tag cloud view offers a pretty blunt view across a whole server or a particular space at a time.
In this search interface, Attivio also helps us extract and display tag cloud style views of any facet, including keywords, spaces, content authors, and more.
This brings us to Small Data.
A thick slice across all time isn't adequate to explain a course of events - a long view with thin slices and supporting narrative is vital.
Or, if you want to consider more thought provoking ideas about tags, other meta data and the role of time, please click over to my Tag Mush presentation linked at the bottom of Ontologies & Tagsonomies at Taxonomy Boot Camp to read more about the Information T.
This Information T model talks directly to the importance of Long Data, Big Data and context from Blog Data.