Digital transformation & the enterprise – Big Data and the failure problem
Size matters in business—the way a smaller organization operates and behaves is radically different from a large enterprise. As we enter the digitally-powered business era, these differences in behavior will have a profound impact on the overall success of that organization.
The most profound difference revolves around the “fail fast” mentality that has arisen from Silicon Valley and spread across the startup world. Larger enterprises have a tendency to “stay the course” on a project and plough through difficulties. Their smaller, more agile rivals on the other hand, adopting many of the principles espoused by “Lean”, may choose to abandon the project when presented with the same challenge, only to move rapidly onto something that will work much quicker.
In previous eras this “stay the course” mentality gave the larger enterprise and edge, and resulted in competitive advantages that were hard to replicate by their smaller rivals. In today’s era of big data however, the large enterprise may find itself with projects becoming far off course due to this cultural difference. This is because while the sheer number of possibilities that big data brings to the enterprise is unprecedented, the opportunities for flawed analysis and project failure are equally legion. Of course, so too are the burdens Big Data puts upon the IT infrastructure of any organization.
When we talk about big data, we often discuss large, unstructured datasets. These take a lot of processing power from which to discern trends—making any attempt to glean insights from the dataset, more often than not, a fairly costly experience. The cloud computing revolution has offset this somewhat, but the costs are still there. There are also very unclear advantages to such datasets—will the insights gleaned be truly valuable, or actionable? A larger enterprise could end up with a poor overall investment, where a smaller one would choose not to pursue the project.
But there’s also the challenge of how to integrate data-driven decision making into the whole business process—making understandable, actionable data available for all who might need it. In a large organization, that’s a lot of decision makers!
Even if a big data project has an initially very limited scope, interest is bound to creep in from other areas of the business once good results are shown.
There’s a lot of pitfalls here for larger enterprises, but what this means for us as software testers is an equally unprecedented challenge. Systems that were never designed to interact may now, to a limited degree, need to become interactive. Data driven decision making may lead to much more rigorous uses of applications that may simply be not designed for the new level of use.
Big data has the potential to expose all the weak points in a network, simply due to the way it will alter how that network is used on a day-to-day basis. If systems start to fail, larger enterprises may get punished for their ambition and drive, giving their smaller rivals an edge for completing much simpler, smaller projects whilst their rivals struggle.
The promise of Big Data driven business however is too big to ignore— so how do we ensure big data doesn’t break the enterprise?
The solution is to change the way we test, moving from large individual projects to smaller, more focused agile testing cycles. This will mean testing smarter and more accurately by identifying the critical areas within a network and understanding how the business works holistically. With increased stress on the network coming from the business, it will become critical to be able to ensure quality at every stage of the project cycle. To do this, you need to be confident your suppliers are fully committed to deliver what they promised, or have someone on your side you can trust to make sure they do!
These trends are what will set testers like Experior apart from the pack. Large enterprises do not want to fail – fast or otherwise. They therefore need a partner they can trust to work intelligently with them to make sure a project succeeds and stays on course. We’re based in the UK, so if any of the above feels familiar, please do give us a call.
Originally posted October 23, 2015, by Martin Mackay, Experior Group