Ask any insight professional what the driver of business decisions is and you’ll likely hear one answer. Data. Whether it’s driving marketing investments, financial planning, customer service structure, programs of personalisation or even real-time product interactions – data is everywhere. In fact, data is so ubiquitous it’s becoming increasingly hard to define exactly what a business has access to.
It’s important to note too that data, as many have pointed out, is not knowledge. And it’s not insight. A singular data point tells little story. A user interaction, a purchase value, a written response to a feedback survey. In isolation, none of these mean very much at all.
In order to understand what’s happening throughout our businesses, therefore, data must be collected on a massive scale. It must be stored, analysed, interpreted and shared. But managing that complex process brings a set of unique set of challenges. For a start, data is distributed across departments, functions, structures and uses. Some may be related, while others are not. And some may be tied together in surprising ways.
Consider the example of a manufacturer that sells consumer electronics to retailers. The production teams need to monitor data from the factory machinery closely. The marketing teams will require customer feedback, market data and competitive intelligence. Procurement, finance and retail teams rely on knowledge of costs, delivery timescales and inventory levels.
But let’s take what seems, at first glance, a simple commercial factor. Production rate. There are two potential questions we could ask:
The first question is easy to answer. It simply requires measuring the output of our factory. That can be done by one team with one data point. The second question, however, is both more commercially relevant and exponentially more complicated. It’s also the one that provides the greatest competitive advantage if you’re able to answer it. So, what would you need to know? Well, to start with:
Why so much information? Because these all have a bearing on the ultimate cost of production that output rates can impact. While it may seem logical to work at the fastest rate, if consumer demand is down or a retailer puts orders on hold due to financial difficulty – holding onto that excess stock might be more expensive than simply producing new stock at a lower rate.
Clearly, such critical points are important for business leaders to address. But to do so requires multiple teams collecting relevant data points, feeding them into a centralised management system and others – who may not be responsible for analysing that data – to make decisions based on what it’s telling them. To an insight professional, that can be a pretty scary thought.
No longer is customer insight the driving force that it has historically been. Today, it’s just one of many data sources that feed the decision-making machines of commerce. A machine that isn’t operated or owned by one department, but set by the culture and competencies of the business as a whole. There are, however, a number of ways that researchers can positively influence the way in which data is managed, even in a modern, agile setting. Let’s take a look at a few of them.
This approach to data management involves creating a master list of business use cases for analytics and data. This list can also include ways that data can help shape better products, services or internal processes. Next, take an inventory of the different types of data associated with each item and identify the most important customer characteristics.
From this analysis, teams should be able to order the identified opportunities by importance, discussing the governance, architecture and data requirements for each. Ultimately, this activity helps to highlight which of the most valuable business priorities existing data can be applied to and which forms of data are missing. These gaps should be addressed as a joint exercise between insight, IT and relevant stakeholder teams to build the relevant qualitative or quantitative data capture, analysis and distribution functions required.
This is a collaborative, technology driven solution to the data management challenge. It involves breaking down the barriers that exist between teams – created through requests and single sources of ownership. Rather than passing data requests back and forth, the joint ownership approach borrows from agile and scrum philosophies.
In short, it involves bringing representatives from teams responsible for data collection together into a single room to take shared responsibility for piping data to relevant locations. And, as with any agile methodology – the process works best in short sprints, where stakeholders have the opportunity to input into regular reviews and requests are prioritised within a long-term backlog of work.
A microservice, in software engineering terms is a type of architecture that enables distributed applications to be deployed using multiple containers. Designed to be scalable, microservices arrange applications (that can be drawn from a single data lake, or multiple sources) into loosely related services. The purpose is this is to provide large teams a way to bring multiple services to life at once, independent of each other.
| Tweet This | |
| Microservices aren't just a novel approach to systems engineering. They can provide a blueprint to how the containerisation of insight leads to better results with less risk. |
For researchers, insight professionals and data engineers – this is an important concept, and one that can provide a blueprint to handling multiple, complex data needs at scale. Instead of thinking in terms of projects, inputs and outputs, the containerisation model encourages thinking at the service level. What collections of data can be grouped to serve to business units and functions, and for what purposes? Combined with an effective knowledge management system, this can be a powerful way to deliver better, more relevant data whilst lowering the risk of engaging in an increasing number of business decisions.
But none of these solutions alone will be enough. Because of the core of agile data management is a complex set of skills and competencies. A set of skills that blend the need for human-centric communication and empathetic project management with knowledge of systems engineering, decision making, technology and research. The brands that succeed in the world’s most competitive marketplaces are those that know the value of their data, and invest in the right people to manage it – supported by cutting edge technology and proven frameworks.