Predictive analytics – art or science?

analytics word cloud on digital tablet

Predictive analytics uses statistical techniques and mathematical algorithms to analyse existing data and extrapolate it forward to predict what’s likely to happen in the future. Which customers are most likely to cancel their contracts? Which ad campaign will people most likely respond to? Which insurance claims are most likely to be fraudulent? On which assets should maintenance resources be most urgently focused?

Most organisations understand how to use their existing data to understand what happened in the past – which campaigns were most successful, which customers left the organisation and so on – but predictive analytics takes this to the next level and projects the analysis forwards, telling you not just what has already happened but predicting what’s going to happen.

Predicting the future is an art as well as a science, in the same way that any business decision is based on a combination of hard facts, data and inferences as well as more intuitive factors like experience and situational understanding. Predictive analytics enables you to use information you already hold to compensate for information not yet held, such as the outcome of a future event.  The ability to do this well depends as much on intuition and imagination as on algorithms. Bringing these two elements together successfully is what sets the predictive analytics of today aside from more simplistic business intelligence analysis or from the rigour of hypothesis-driven statistical science.

In the past the processes and tools required to extract and organise data were so complex that analysts had to work with a limited range of rigidly structured material because there was no time for anything else.  The result was no space for creative exploration, innovative ideas, and intuitive leaps. However today all that has changed. Fast and flexible tools like IBM SPSS Modeler, coupled with the accessibility of all sorts of data from new sources in new formats have freed analysts to spend their time on the “art” of discovery.

A business cannot know for certain exactly which customers will buy product A over the next 12 months—but they will have a lot of data that could be used to understand and improve the targeting of customers most likely to buy product A, enabling marketing to get the right message to the right customers at the right time through the right channel.

Somewhere in the company, there is data about customers who have bought product A in the past.  There is information about other products they have bought and the patterns of activity that drove them to purchase.  There is descriptive information about who these customers are, how old they are, where they live, how many children they have.  There is transactional information about when they last bought, how much they spent, how often they spend. There will be browsing behaviour, basket analysis of what products they buy in what combinations. There is information on what kind of offers they respond to.  There is data on how they like to pay, how quickly they pay and if payment type alters their spending pattern. You get the picture.

Some basic analysis can be done using this data by means of canned queries, pivot tables, summarized reports, and so on— but that’s basic ‘how much, how many, how often’ analysis – it’s not predictive analytics.  The ultimate value of all that data will only become evident if the people who know how to think about the data can access it easily, explore different views, test various hypotheses, and share their findings effectively to discover why customers buy in certain ways.  Understanding why customers behave the way they do and making predictions about what will happen next is where the real value is.

If you give your analyst the right tools they will be able to follow up multiple avenues of investigation and create a range of projections for review, all in nearly real time, with no need for IT intervention. With fast, user-friendly tools, analysis can be done with input from the people who understand the business context and the data rather than the people who only understand the database technology. This makes a huge difference to the results because in-depth business knowledge is a major key to success in predictive analytics.