ImageNo one would argue that hidden in the piles of data resides information that can be used to transform the way we do business, how we reshape societies and to increase our awareness about the world we live in.  While it hasn’t always been called Big Data we have had it.  Whether we called it a data warehouse or simple let it go unlabeled as a mass of data that we acquired, utilized, manipulated and reported on it none the less is Big Data to us (in our own context of what constitutes as BIG).  Some may even shy away from the topic because it simply means more cost and at this time more spending is not a possibility.  But don’t overlook the systemic value that is produced with knowing something that could shape the way you do business, the opportunities that can be created, problems that can be avoided and the gain in market share that could be achieved.


We need to recognize that there is a difference between data and information.  There may even be cases where information then becomes data to feed more advanced processes, and analytics.  Data in this context can be in both raw native form or authenticated (edited) data that has gone through a filtering process to insure that only legitimate, valid and appropriate data is retained.

Another belief that needs to be understood that more does not necessarily mean better.  Even though that systems produce consistent output, there is a cost to process data.  There is a costs,

  • Storage,
  • Preprocessing,
  • Post Processing,
  • Analytic Engines,
  • Human Study,
  • Cost of General Processing (batch vs. real-time),
  • Authentication, and
  • Testing/Validation.

Although some of these costs are associated with the overhead of doing business making the leap to BIG Information for use in a knowledge based analytic process is not simply do a few reports and going through the motion of study.  Big Data is often considered large and volatile data sources that cannot be handled by conventional analytic tools.  While this is a definition I find it hard to think of many cases in which this doesn’t describe the realities of everyday data streams whether in the commerce or governmental sectors (and the potential interplay in between). 


Statistically speaking one can deduce as much from a sub-set of a larger data population that one can from using the entire population group.  There is no such thing as 0 (zero) error rate and 0 (zero) precision, and confidence levels can be as high as 99% but 100% illusive even when utilizing technology.  The human factor that would count a jar of beans is an dependable in achieving the right count as the human who creates a system to run Big Data from start to BIG Information.  For this reason those that are contemplating a Big Data/Information initiative must deal with the economics associated with it.  As was noted earlier in the small example list it is not something that is free.  We may not think of it in this way because its so easy an so transparent that we can turn on our computer, open a query engine, obtain big data results and further process it till we get information that can be examined and scrutinized   We had to buy a computer, acquire a service provider, take the time to do our mining, pay for the electric, replace failing components and may even have to do some manipulation using Excel which required us to obtain a license for.  In the context of commerce and public sector this example is several times more complex and will involve costs.

Costs Maze

Aside from doing a quick tabulation of costs we need to look at how these costs get accumulated, at what point in time, which are resting costs, and what is the costs associated with the speed of information delivery.  For these reasons more isn’t always better.  There is some data that simple is of little to no value except in support of some mainstream business function operationally but not necessarily valuable for BIG Information analytics.  It may be cheaper to harvest from other sources or its speed handling may be such that the results would already be too latent to be of strategic decision making value.  I discovered a very interesting diagram by Wipro (below) that addresses the topic and some of the early adopters.  Overlooking that its obviously to sell their analytic services it pretty much speaks for itself by illustrating the hidden potential that BIG Information offers.


While the early adopters are Western enterprises there is a solid justification for Eastern markets to utilize BIG Information for the purposes of market cohesion with the West.  To date this has gone untapped.  Even though that Westerns were early onset Big Data proponents most of the data/information is contained within their region and not globally.  So on one hand it’s good for them on domestic markets but horrible when it comes to understanding others.  Likewise knowing that this BIG Data source is about the West it offers insight into these potential target markets.   The question is whether BIG Information is public or private. 

The Fence Isn’t Just Around OUR Pen

There are bounds to which the BIG Information is accessible.  Private institutions may keep it safely under lock and key for it contains the magic answers that support their strategies.  But the public sector is totally different and thus its accessibility is generally more accessible.  We do have to consider the factors of timeliness, size, sterility, and coupling that would need to be performed.  But our domain of Big Data and Bid Information isn’t all that we should think about.  The phrase “no man is an island” is appropriate to the topic of BIG Data/BIG Information.  Let’s face it we live in a coupled world.  What we do has an effect elsewhere in society.  If weather systems report low than normal rainfall, with predictions that no change can expected in the near term, our crop based management BIG Data systems needs to consider this in order to regulate irrigation, harvesting and other preventive (and even investment) conditions.

BIG Data is NO Simple Topic

Its really good to see that the topic is being discussed and that some explanations are being delivered.  However this is no simple topic.  What might be a wonderful explanation is a mere starting point and the depth and extent of the discussions will go on for some time.  A few of the topics that need to be discussed and in depth include,

  • Cost/Value Management of the BIG Data/Information Initiative
  • Transitioning Conventional Data into BIG Data/Information Paradigms
  • Security and Protection
  • Culling, Thinning and Re-Organizing BIG Data
  • BIG Data/Information Analytics for Batch and Real-Time Attention
  • Utilization of Private and Public Clouds for BIG Data
  • Data/Information Authentication
  • Constructive Growth Management 
  • Effects of BIG Information on Business Transformation and Optimization

I hope that these few thoughts will help all readers to understand and appreciate value,Image purpose and level of commitment needed for getting the most from what you do and the world around you.