Thinking of using spreadsheets for advanced analytics? Think again.

When we’re talking to potential clients about advanced analytics we often ask them what tools they’re currently using. More often than not they say they’re using spreadsheets. Spreadsheets are one of the most widely used tools for statistical analysis and of course, most businesses couldn’t run without them. However, when it comes to advanced analytics spreadsheets have some very significant limitations. Use them beyond their capabilities and the potential cost can be significant. As with anything, it’s important to use the right tool for the job. So, what are the things you need to consider when it comes to using […]

The benefits of survey research and how online platforms can make the research process more efficient and effective

Surveys remain one of the most popular forms of market research, and consequently we regularly work with customers on large survey-based market research projects. Most commonly this is customer satisfaction surveys, and these can be one of the most valuable forms of data that a company has. We also have clients who conduct broader market research, going beyond their own customer base in order to learn more about a particular market or segment in which they are interested. Of course surveys are not the only way you can find out what your customers think. Sometimes they’ll tell you what they […]

How to ensure advanced analytics gives you a concrete competitive advantage

Most companies these days are aware of the potential value that advanced analytics could offer them. Lots of people are talking the talk of advanced analytics, hiring data scientists and investing in sophisticated analytics technologies. But these things on their own aren’t going to give you the competitive advantage that you’re hoping for. It’s not possible to bolt advanced analytics as an adjunct to your marketing or strategy function and hope that the magic will happen just like that. We regularly talk to people who’ve invested a lot of money in advanced analytics software and are wondering why they’re not […]

How will the GDPR affect big data analytics?

With less than a year to go until it comes into effect, organisations are really starting to get to grips with what GDPR will mean in practice. We’ve talked to lots of customers who are concerned about the implications that GDPR might have for the way in which they collect and analyse customer data. Much of what constitutes ‘big data’ is personal data and the use of this kind of data definitely does have implications for data protection, privacy and individuals’ associated rights. And these rights are going to be strengthened by GDPR. So does this spell trouble for big […]

Which data science tools should you learn?

I’ve blogged several times now about different aspects of data science. A conversation I’ve been having more and more frequently now is about what tools people should learn if they’re hoping to develop a career in data science. Obviously there are many different factors to be taken into account here. You’ll want to think about whether there’s a tool that’s the standard in your particular industry. You’ll also want to consider whether you want to specialize in a particular area of data science and build a reputation as an expert in a range of related tools, or whether you’d prefer […]

Six questions to ask before you opt for open source software

It’s not uncommon for people to say to us that they don’t understand why they should pay for industry standard analytics products like SPSS or SAS when there are strong open source alternatives freely available such as R. Indeed the development of R has really transformed the analytics marketplace in many ways. It’s tempting to make a comparison between R and commercial alternatives such as SPSS and SAS on price grounds alone. When you look at it that way it might seem as though there’s no contest. SPSS and SAS can both involve a significant investment whereas R is free. […]

What’s the difference between business intelligence and predictive analytics?

It’s not uncommon to talk to potential clients who consider themselves to already be very much data-driven in the way that they operate. However it’s very rare to find a potential client that truly is exploiting the full potential of the data that they hold. That’s because companies often confuse business intelligence with predictive analytics, or think that once they’re using their data for business intelligence that they’re doing all they can to get value from it. Neither of these things is true. Predictive analytics is not the same thing as business intelligence, and if you’re just using your data […]

Data science is everywhere, so why no data scientists to be seen?

Data science is everywhere at the moment. Nearly as everywhere as big data, but not quite. Books out there are making the concepts behind statistics and predictive analytics more and more accessible not only to those in business making decisions everyday but also to the average man or woman on the street.  Try Super Crunchers by Ian Ayres, Moneyball (the book  or the film which has the advantage of featuring Brad Pitt and therefore making the business of statistics much sexier than it has been),Freakonomics or the newer Superfreakonomics or pretty much anything by Malcolm Gladwell. All of these books have […]

Data science projects – what skills do you need and where can you get them from?

Data science is on the rise. A couple of years back Harvard Business Review suggested that ‘data scientist’ is the sexiest job title of the twenty first century and the hype around data science shows no sign of abating. The term ‘data scientist’ itself was only coined in 2008 but since then the number of data science roles in organisations has grown exponentially, as the volume of data available for analysis also grows. But this presents a challenge for organisations – in such a new and fast-changing field how can they identify the skills they need, find appropriate people who have those skills, […]

Nine tips for effective data mining

In my career I’ve seen many examples of successful and unsuccessful data mining projects. I’m often asked how clients can maximise the chances of their project being successful and, based on the many projects I’ve been involved with over the years, I think there are 9 things that really help. When these factors are in place it always suggests to me that the project has a much higher likelihood of success. Think carefully about which projects you take on. To maximise your chances of success try and focus on those projects which are most clearly aligned with important business issues […]

Predictive analytics – five ways your business can benefit

When talking to companies who have yet to invest in predictive analytics I am often asked if I am sure that they will benefit. Will it be worth the time and effort involved in setting up a predictive analytics project? What kinds of benefits can they realistically expect? Are they large enough to reap the benefits of predictive analytics? This last question is the easiest to answer. If you’re selling direct to consumers and you have a sufficient customers that you cannot realistically have a genuine personal relationship with each one of them then the chances are that predictive analytics […]

Using predictive analytics and the tax inspector’s nose to spot fraud

Organisations hold information, lots of it. Often it’s all over the place and sometimes its not acknowledged as being useful but there is always lots of data from customer interaction and transaction information to production line or stocking information. The trick is learning how to use past data to do better in the future. Organisations are a lot bigger than they once were with more and more employees who have a shorter and shorter tenure. These changes have significant implications for how analytics is done. Where data volumes are small and employees are knowledgeable it’s often possible to get a […]

Do you have to be a data scientist to do predictive analytics?

What is a data scientist? The predictive analytics field seems to love nothing more than giving a new name to an established concept. In my last post I argued that the concept of ‘big data’ itself is nothing new. Over the last few years I’ve seen more and more job ads recruiting ‘data scientists’  and I find myself rather unjustifiably irritated by the rise of this new job title. I can’t quite put my finger on what it is that irritates me about it but I think it’s tied up with the creation of a new job title for something which isn’t really all that new, and a […]

Five myths about big data

Big data is everywhere at the moment. There’s a lot of talk about it, much of which presents big data as a problem to be solved rather than as an opportunity to be seized. There are many organisations that could be using big data for powerful predictive analytics but aren’t, because they’ve bought into one of the many myths about it that are in common circulation. Here are five of the big data myths that I come across most frequently in my work 1. Big data is new The terminology is new (and time will tell if it’s a term […]

Assessing the value of your customers

Moving beyond RFM analysis to create truly data-driven customer segments RFM stands for recency, frequency, monetary value and is a method for calculating or assessing overall customer value.  Its most common use is in direct marketing or database marketing and it is particularly heavily used by retailers and charities. RFM analysis is predicated on the idea that your most valuable customers are those who spent most recently, who spend most often (frequency) and who spend the most (monetary value). Most organisations hold this kind of data about their customers which makes RFM analysis relatively accessible and easy to do. Typically […]

Do you really need special software for predictive analytics?

When I’m talking to prospective clients, one of the questions I’m regularly asked is why they should invest in expensive predictive analytics software when they already have Excel. Excel has some useful features and for many organisations it can be a useful toe in the water of analytics but it’s really not built for the rigors of true predictive analytics. Organisations that stick to using Excel will never be able to truly reap the potential rewards that predictive analytics can offer. I’ve recently been working with a large B2B online publishing organisation and my experiences there illustrate this point perfectly. […]

Predictive analytics – art or science?

Predictive analytics uses statistical techniques and mathematical algorithms to analyse existing data and extrapolate it forward to predict what’s likely to happen in the future. Which customers are most likely to cancel their contracts? Which ad campaign will people most likely respond to? Which insurance claims are most likely to be fraudulent? On which assets should maintenance resources be most urgently focused? Most organisations understand how to use their existing data to understand what happened in the past – which campaigns were most successful, which customers left the organisation and so on – but predictive analytics takes this to the next […]

IBM SPSS Modeler – 8 reasons why it is still brilliant after all these years

IBM SPSS Modeler has been through quite the name changes since it first came onto the market as Clementine in the 1990s. In 1998 it was acquired by SPSS. Controversially, in my mind, SPSS then changed its name to SPSS Modeler (spelled the American way which causes no end of confusion in spell checks or when writing documents).  Even after nearly 5 years many of its long term fans and user base still refer to it as Clementine and indeed there are still pockets of people out there who don’t know that Clementine and Modeler are the same (rebranded) product.  I mention […]

How charities can make better use of predictive analytics

People often think of predictive analytics as something that’s primarily useful for commercial organisations or is too expensive for charities to make use of, but this really isn’t the case. At Smart Vision we work with a wide range of charitable organisations, many of whom are using SPSS Statistics or SPSS Modeler software for data mining and analytics. The vast majority of these organisations use statistical analysis for programme delivery, outcomes measurement and policy work. It’s no secret that many need to do this in order to report back to grant making bodies. This is a fairly conventional use of […]

Are the results of predictive analytics really that surprising?

It’s common to hear claims about how businesses will be transformed through the use of predictive analytics techniques with surprising and shock results that they never would have imagined to be the case. It’s particularly common for product or service vendors to make these kinds of claims when in pursuit of a possible sale. After all, who could resist the prospect of a magic analytics bullet that promises to completely transform their organization? I completely agree that the business benefits of analytics can be significant, but I think it’s a myth that predictive analytics will always produce surprising results that […]

Top ten predictive analytics questions

During a recent trawl of my archives I happened across an article from DM Review, now Information Management, first published back in 2000. The topic: Top Ten Data Mining Business Questions. This was back in the days before the term predictive analytics was common parlance but I read through the list and wondered – have we really changed that much?  I'm paraphrasing a bit, but the list was this. 1.     What are the business benefits? Have you figured out what you can do with this? Can you quantify and measure the benefits?  Have you really worked out what the actual […]

The first step in predictive analytics – understanding your data

I speak to a lot of people in organisations just starting out on their analytics journey, organisations that have started to recognise that they could make better decisions if they could find the hidden patterns and nuggets of information in their data. Data talks and you can tell very quickly if it has something interesting to say. With all the current hype around big data the irony is that, in my experience, the most common worry in the early stages of investigation is that the organisation doesn't have anything to analyse. They are waiting for a new CRM system or […]