April 2013

ImageI once heard a person say “there is no such thing as a best practices”.   As I pondered this quote and reflected back on years of work experience I found the statement to be quite true indeed.  The “best” in anything is based upon the context (situation) to which the qualification is being given.   Any of you that has an addition to something, whether it be candy, coffee or some other vices, should realize that what is “best” isn’t always right when it comes to uncontrolled and wrongly inspired pursuits.

Some while ago I was dealing with a customer who was on a quest for excellence in software engineering.  They were throwing all of their efforts towards this pursuit.  Hiring some of the most talented people, utilizing state of the art technologies, and crafting some of the top methods in the industry.  One would think that they would be on a course for smooth sailing.  But to the contrary they were in utter chaos, motivation was at a low, and remedial solutions with feverishly being employed to counteract this downward spiral.  Off in the corner of this company a small group of engineers was merrily off crafting solutions, they weren’t perfect or entirely by the book but they were achieving the results that were asked for by the user (aka project sponsor).  At first glance one would say it was the people, others were quick to say that the burden of process was what was holding others back… and that the wayward direction of this team showed it wasn’t needed, while others were simply discounting luck as a means by which achievements were being made.  We were asked to try and sort this all out and to help them in achieving their pursuit.  What we discovered is not uncharacteristic of client/buyer companies and companies that deliver software solutions as a service… the goal doesn’t match the need.  What does this mean??  If we think of solutions, software or otherwise, we have certain expectations about its risk and durability.  If I buy a cheap watch I expect that it will last a certain amount of time, beyond that its a blessing to get more.  But if I buy an expensive timepiece I expect a whole lot more in terms of durability, longevity and event features.   I have to say that today we are overwhelmed with commodities that are embellished upon to draw market attention but because they are producing cheap watches the excess is not a virtue but is in fact a fatal inhibitor.  If that cheap watch comes with a temperature sensor and a flash light and both either fail to work or only for a limited time it then becomes a sore spot for criticism.  The long term effect is to push for a more simplistic solution and not with with excess perceived value (what some might refer to as Gold Plating).

In the software engineering context we have been a bit misguided by our pursuits.  Rather than to use these aspirations as a measure to advance, improve and afford economic value we have them as stick to drive behavior.  Unfortunately we have seen time and time again that it has driven us to exact polar opposite… its created chaos, confusion and a loss of direction.  In the case of our client company they had wonderful ambitions.  At the sametime these ambitions got in the way of addressing what level of excellence was prudent for the types of solutions being developed.  To user/sponsors who were looking for appropriate and reliable support, they were getting excesses that provided little value, higher maintenance and operational costs, and were fraught with issues.  What should have been an environment to achieve this had become in an of itself the culprit.  Our first question wasn’t about tools, techniques or people it was about the goal.   Was a top tier best practice and the formation of a center of excellence right for the type and complexity of the solutions being crafted?  In the context of a CMMi model, were you aspiring and working to deliver level 5 solutions in a level 3 context??

Today, 2013, we are inundated by companies, who profess their excellence in how they have set the stage for solution engineering.  But in setting that stage the cataclysmic result show a much different situation taking place.  No, these people are not hell bent on creating chaos, they are driven by excellence.  However this unrelenting pursuit is in juxtaposition to what is needed.

The correction can be painful.  It seldom requires a wholesale reversion to earlier practices, rather it requires a righting of the course.  It requires a modification of goals, of practices and even team compositions.  The utilization of software service engineering companies will require a reappraisal to insure that they are operating at the ‘right’ level, and not simply at a level where excellence cannot be achieved (despite all the pragmatism that may be touted).  Some would wish to believe that this will require an objective third party, which it may depending on internal politics, but it simple needs someone (usually a group) to undertake the work of reshaping, re-crafting and redirecting the initiative.  It isn’t always a lengthy effort, but one that requires meticulous attention to the fine details and the effect of change.  The time that can be substantial is in transitioning, redeployment, monitoring and advisory.  But this is a cost of learning, whether its by mistake or by redirection.

So at the end of engagement we were asked what the cost would be for putting things back in good order?  We had worked the numbers and arrived at a cost for the revamping project and put it up on the white board. The client said, “wow I thought it would have been more”, to which we responded it would have been if you stayed on the wrong path.  You would have put at risk everything, not just the software/IT operation but the business as well.  These are matters that cannot be implemented on the basis of popularity, common practice or casual analysis… they require heartfelt and earnest consideration that weights the benefits against reality and not based on wishful dreaming.   Make your best practices and your centers of excellence appropriate to your world.  Stretching a bit is okay as long as its within your attainable grasp.   These measures will contribute to your business, be responsive to needs, and serve as a foundation for the forward growth of the delivery enterprise.

ImageHaving been self-employed for a number of years I can attest to having those moments where promise, hope and opportunity are overcome with fits of frustration, anxiety and depression.  I thought it was simple me, and as I looked inside my personality and portfolio it raised many questions as to who I am and why am I subjecting myself to such abuse.  In pursuit of an answer I scoured the internet and bookshelves.  Sometimes delving into the bizarre and complex world of clinical psychology, and other times looking to ways people have vented themselves (possibly as a means of alleviating depression or whatever label we wish to assign).  I came across an interesting discussion that took place back in 2003 about this topic (http://www.feld.com/wp/archives/2013/01/are-entrepreneurs-more-prone-to-depression-and-divorce.html) by B.Feld (venture capitalist at the Foundary Group) and found that its more than just an occassional situation but one that seems to rustle around through the entrepreneural community.  Some confessed that they had been diagnosed while others gave way to the effects of depression.  I found no indication what so ever that it had effect on the ability to deliver service, but often it manifested itself in their personal lives.  Does this mean that our private lives are a second to business?   Does it mean that if we are going to sacrifice anything it will be those things that we emotion connected with?

Talking with some of these folks I found that people didn’t willingly discard their personal connections.  Rather the intense quest for answers to depression related to business consumed them to a point where there was no room for those personal things.  Those that lost business but kept their personal relationships were more apt to be geared toward having the ability to always start another venture.  But what did they do in these lulls between fits of depression and tranquility?  Was it productive and did it make a difference in abating the seemingly endless cycle of depression?

No, rest was not productive and in fact breaking the cycle of rest was a challenging as it was to break the cycle of depression.   While spits and spurts of innovation and creativity occurred allot failed to ground the concept with an outcome that would be judged as a successful.  How many people know that people like Abraham Lincoln, Johnny Depp, David Beckham, Kim Basinger, Princess Diana, and Billy Joel would suffer from such a gut wrenching affliction?  Outwardly supreme performers and accomplished individuals, yet deep inside they suffer on a daily basis.  What seems to be common with all forms of depression is the unbelievably intense desire to achieve and to fulfill.  As a trainer once told me, “education isn’t that hard to deliver but its the intense lond standing on stage that keeps the audience captivated that is so difficult to endure”.  Having stood in those shoes I could fully understand what he was saying.

As some of you sit there reading this piece I hope that you understand that you may not be alone.  Some may be vocal about it, while others keep it close for fear of being viewed as weird or unworthy (a mild case of paranoia).  I can assure you that most people have depression, anxiety and vacillating focus.  The difference is whether this is a brief, or a very occasional situation or whether its recurring.

We look for answers, some that have sever issues seek both counseling and pharmaceutical solutions.  But for the average entrepreneur we seek solutions in our own unique, sometimes quirky, and often experimental way.  Unfortunately we sometime exploit non-business solutions like drug and alcohol abuse, self-abuse through overwork (I’m sure you have heard those people who are self-anointed workaholics) and sometime other pleasures to which I will leave your imagination to run wild with.  Some of you may exploit outlets through over use of social outlets, endless and unconnected writing (I think I’m in that category) and even an endless parade of non-paid speaking engagements justified as ‘good marketing’.   None of these are bad, unless of course its a sin or illegal, but its truly a matter of moderation and not to excess.  I have found that many far Eastern souls love to use and explore quotes as a means to shadow their question to overcome depression, and some have even gone so far as to self-quote.  Again I say, whatever works.

Yes, I am envious of those that have a normal job where pay is a normality and work is something that one can dedicate themselves totally to.  As a serial entrepreneur we lack the luxury of a normal paycheck, we are driven by an intense desire to make an impact.  Whether this impact is on society or whether it is on one small aspect of a project for a customer.  Where I am still at odds is where, of if, entrepreneural depression is solvable or whether its a condition that cannot be overcome (only dealt with).   I hopeful that regardless of whether it is or not that we find peace, joy and a means where we can help ourselves and also help others.

I am no longer depressed… I am not guardedly optimistic.  My mind is free to think creatively and to control the desire to exercise a knee jerk reaction in favor of a much more controlled pragmatic though process.  

What does it mean to be successful.  Some measure it by acclaims, status, social recognition and well appointed through awards.  But is this really success or is it simply a response by our peers to a level of notoriety that only they can appreciate?

ImageAs a very young pre-school child I knew that I was gifted with a good mind.  While my grandmother and parents encourage me to be inquisitive and to dream, this was something that I had to develop and is not something that one can teach (or command) someone to do.  I remember thinking about how societies should stay within their borders and become self-sufficient and how every action resulted in a reaction (Newton’s 3rd law).  It was later in life when I was able to write, read and discern the differences in colors that I realized that my thinking was a bit different that all the rest of the children that I hung out with.  Later in life I realize that my thinking hasn’t changed all that much.  It is not the mainstream, it’s not uniform and consistent with social norms, and certainly it may be a bit confronting to others that have spent a career doing one thing whereas I have had multiple pursuits in life.  Each one of these pursuits were with a fervent intensity that even single disciplinarians are seldom found having.  These words are not a boastful claim but a matter of present reality as to my life.

I have found myself taking great interest in learning new things.  To be able to reach down deep into the depths of my knowledge to eek out similarities, inconsistencies and to bootstrap all of these things into an ideology.  I often find that these change as new facts are presented or as norms shift in the crazy world we live in.   To those cautious and often silent onlookers it appears as a every changing landscape, met with skepticism and often responded to in a defiant manner.  To accept and to embrace would mean to step outside the well worn path and to open up new possibilities.

The word of the last decade has been innovation but as we have learned innovation is not a science that can be taught.  It can be discussed and examined but you can’t encourage creative non-traditional thinking from the mainstream of society who prefer to norm rather than the exception.  Over this period of time what was born out of innovation transitions to a state of consistency.  Such everyday aspects (internet, email, social networks, cell phones) were at one time seeds of innovation.  These seeds have grown and become a piece of everyday life.  How many of us would feel comfortable not having connectivity using any one of these outlets?  Have we forgotten how we interacted and communicated prior to their advent?  The ability to revert is not easy because we have replaced and embraced, having ventured beyond include and explore.

Let us go back to my original point of this post and that is non-traditional behavior should not be met with opposition.  Even though one must be cautions of ‘wolves in sheep clothing’ there are non-traditional thinkers who should not be overlooked.  You are depriving yourself of the innovative behavior that you may not possess and it is highly possible that you are depriving yourself of a key to unleashing your own free thinking.

As people in society seek the magic answer behind those success stories we are overlooking those that offer something even more spectacular.  Maybe its because we can understand success and accomplishment, but can’t comprehend how someone who hasn’t risen to that level could ever be taken seriously.  I think back to a comparison between Steve Jobs and Dennis Ritchie (of Unix/C fame).  One man was highly successful financially and in business, while the other showed exceptional brilliance and contribution to computing.  Maybe the difference is aspiration.  Having known both I can tell you that while one wanted fame and the other excellence, they both desired to make a difference to society.  In their own and unique way they both achieved excellence.  What may be the untold story remains a question of legacy.  Were their acts such that they will endure a long and every lasting social impact?  

I really never wonder about whether I will be successful.  I feel that I have reached that point where success has been achieved in so many ways.  Business, technical and even personal my story is rich with color, humor, despair, frustration, confusion, bewilderment, and contentment.  But my last chapter isn’t written yet.  I sometimes lament and get a bit uneasy that to start another endeavor means that I may never get to see it reach its potential.  But then I realize that neither of these individuals (Jobs or Ritchie) cared…. they lived each day as though it was their last.  They took every opportunity they could and ran with it and it wasn’t the product that was their fame but the attitude by which they lived life.  Unfortunately this can not be duplicated or emulated, it has to be hidden inside of you.  Hidden in the same place as where my non-traditional thinking and drive exists.  Sitting there and waiting for it to be pulled out and put forth in a way that society can accept, embrace and make use of.

ImageNo one would argue that hidden in the piles of data resides information that can be used to transform the way we do business, how we reshape societies and to increase our awareness about the world we live in.  While it hasn’t always been called Big Data we have had it.  Whether we called it a data warehouse or simple let it go unlabeled as a mass of data that we acquired, utilized, manipulated and reported on it none the less is Big Data to us (in our own context of what constitutes as BIG).  Some may even shy away from the topic because it simply means more cost and at this time more spending is not a possibility.  But don’t overlook the systemic value that is produced with knowing something that could shape the way you do business, the opportunities that can be created, problems that can be avoided and the gain in market share that could be achieved.


We need to recognize that there is a difference between data and information.  There may even be cases where information then becomes data to feed more advanced processes, and analytics.  Data in this context can be in both raw native form or authenticated (edited) data that has gone through a filtering process to insure that only legitimate, valid and appropriate data is retained.

Another belief that needs to be understood that more does not necessarily mean better.  Even though that systems produce consistent output, there is a cost to process data.  There is a costs,

  • Storage,
  • Preprocessing,
  • Post Processing,
  • Analytic Engines,
  • Human Study,
  • Cost of General Processing (batch vs. real-time),
  • Authentication, and
  • Testing/Validation.

Although some of these costs are associated with the overhead of doing business making the leap to BIG Information for use in a knowledge based analytic process is not simply do a few reports and going through the motion of study.  Big Data is often considered large and volatile data sources that cannot be handled by conventional analytic tools.  While this is a definition I find it hard to think of many cases in which this doesn’t describe the realities of everyday data streams whether in the commerce or governmental sectors (and the potential interplay in between). 


Statistically speaking one can deduce as much from a sub-set of a larger data population that one can from using the entire population group.  There is no such thing as 0 (zero) error rate and 0 (zero) precision, and confidence levels can be as high as 99% but 100% illusive even when utilizing technology.  The human factor that would count a jar of beans is an dependable in achieving the right count as the human who creates a system to run Big Data from start to BIG Information.  For this reason those that are contemplating a Big Data/Information initiative must deal with the economics associated with it.  As was noted earlier in the small example list it is not something that is free.  We may not think of it in this way because its so easy an so transparent that we can turn on our computer, open a query engine, obtain big data results and further process it till we get information that can be examined and scrutinized   We had to buy a computer, acquire a service provider, take the time to do our mining, pay for the electric, replace failing components and may even have to do some manipulation using Excel which required us to obtain a license for.  In the context of commerce and public sector this example is several times more complex and will involve costs.

Costs Maze

Aside from doing a quick tabulation of costs we need to look at how these costs get accumulated, at what point in time, which are resting costs, and what is the costs associated with the speed of information delivery.  For these reasons more isn’t always better.  There is some data that simple is of little to no value except in support of some mainstream business function operationally but not necessarily valuable for BIG Information analytics.  It may be cheaper to harvest from other sources or its speed handling may be such that the results would already be too latent to be of strategic decision making value.  I discovered a very interesting diagram by Wipro (below) that addresses the topic and some of the early adopters.  Overlooking that its obviously to sell their analytic services it pretty much speaks for itself by illustrating the hidden potential that BIG Information offers.


While the early adopters are Western enterprises there is a solid justification for Eastern markets to utilize BIG Information for the purposes of market cohesion with the West.  To date this has gone untapped.  Even though that Westerns were early onset Big Data proponents most of the data/information is contained within their region and not globally.  So on one hand it’s good for them on domestic markets but horrible when it comes to understanding others.  Likewise knowing that this BIG Data source is about the West it offers insight into these potential target markets.   The question is whether BIG Information is public or private. 

The Fence Isn’t Just Around OUR Pen

There are bounds to which the BIG Information is accessible.  Private institutions may keep it safely under lock and key for it contains the magic answers that support their strategies.  But the public sector is totally different and thus its accessibility is generally more accessible.  We do have to consider the factors of timeliness, size, sterility, and coupling that would need to be performed.  But our domain of Big Data and Bid Information isn’t all that we should think about.  The phrase “no man is an island” is appropriate to the topic of BIG Data/BIG Information.  Let’s face it we live in a coupled world.  What we do has an effect elsewhere in society.  If weather systems report low than normal rainfall, with predictions that no change can expected in the near term, our crop based management BIG Data systems needs to consider this in order to regulate irrigation, harvesting and other preventive (and even investment) conditions.

BIG Data is NO Simple Topic

Its really good to see that the topic is being discussed and that some explanations are being delivered.  However this is no simple topic.  What might be a wonderful explanation is a mere starting point and the depth and extent of the discussions will go on for some time.  A few of the topics that need to be discussed and in depth include,

  • Cost/Value Management of the BIG Data/Information Initiative
  • Transitioning Conventional Data into BIG Data/Information Paradigms
  • Security and Protection
  • Culling, Thinning and Re-Organizing BIG Data
  • BIG Data/Information Analytics for Batch and Real-Time Attention
  • Utilization of Private and Public Clouds for BIG Data
  • Data/Information Authentication
  • Constructive Growth Management 
  • Effects of BIG Information on Business Transformation and Optimization

I hope that these few thoughts will help all readers to understand and appreciate value,Image purpose and level of commitment needed for getting the most from what you do and the world around you.