Blog
Adding New Modules To Jython Scripting In IBM SPSS Modeler
IBM SPSS Modeler supports Python scripting using Jython, a Java[tm] implementation of the Python language. Modeler versions 16 and 17 use Jython 2.5.1 which includes a number of useful and popular modules. However, many other modules are available and customers often want to use their own so a frequent question is how to include them. There are two approaches: Copy the module to the “site-packages” folder under modeler-installation/lib/jython/Lib. This has the advantage of making modules available to anybody who uses that Modeler installation but usually requires someone with a level of administrative privileges to update the Modeler installation. Define a JYTHONPATH environment variable and […]
Using the find all() Function to search for nodes
Most SPSS Modeler scripts include code that locates an existing node e.g.: stream = modeler.script.stream() typenode = stream.findByType(“type”, None) However, some scripts need to search for all nodes – maybe by node type but also matching some other criteria. The Modeler scripting API documentation (PDF) mentions a findAll() function: d.findAll(filter, recursive): Collection filter (NodeFilter) : the node filter recursive (boolean) : if True then composite nodes should be recursively searched Returns a collection of all nodes accepted by the specified filter. If the recursive flag is True then any SuperNodes within this diagram are also searched. Unfortunately, the NodeFilter definition is not specified. NodeFilter is a base class […]
An overview of the four main approaches to predictive analytics
This infographic provides an overview of the four main families of approaches to predictive analytics. Prediction encompasses applications that aim to estimate or predict the values of a key target field. Segmentation refers to techniques such as cluster analysis which attempt to find the most ‘naturally occurring” groups within a dataset. Association modelling discovers groups of categories that are likely to co-occur such as items in a shopping basket. Forecasting refers to methods that extrapolate time trends such as sales of products so businesses can anticipate future demand. Each family is further detailed with colour coded examples of the popular […]
Using SPSS Modeler’s cache_compression setting to speed up your modelling
There are a number of configuration settings associated with IBM SPSS Modeler Server that control its behaviour. The default settings aim to ensure that stream execution will complete successfully even if the host machine is being used by a number of other applications i.e. Modeler Server is trying to be a “good citizen”. However, if Modeler Server is the primary application on the host, then tweaking these settings can reduce the execution times of some streams significantly. One of these settings is cache_compression. The cache_compression setting is used to control whether data that gets spilled to disk is compressed before being written to […]
Expert insight: Gertrude Sai, Quantitative Manager, Girl Effect
Gertrude Sai is Quantitative Manager, Evidence and Insights, at Girl Effect. Lorna: Can you start off by telling me a bit about your background and how you came to be where you are now? Gertrude: Yes, sure. My background is quite varied but mainly analytical. My degree in Psychology was definitely centered around statistics and research. My third-year dissertation project, which was fairly quantitative, was published and that really cemented my love of research and data analysis. Related to that, I’ve always had an inherent interest of working with vulnerable groups and young people, so once I left university I […]
Expert insight: Hanna Adeyinka, Customer Researcher, Hyde Housing
Lorna: Can you start by telling me something about how you came to be working with statistics and data? Hanna: I’ve always been interested in the psychology of why people do things and that led me to studying psychology at A-level, for my undergraduate degree and for my masters as well. Whilst doing my undergraduate degree, I also did a placement working for Goldsmiths University doing work as a research assistant. I really enjoyed that so when I finished my masters I looked for jobs in psychology, market research and similar areas and pretty soon I was offered a job […]
Expert insight: Inna Yordanova, Senior Researcher at IPSE
Inna Yordanova is Senior Researcher at IPSE – the Association of Independent Professionals and the Self-Employed. IPSE is the largest organisation representing freelancers and self-employed people. Its 78,000 members are freelancers, contractors, consultants and other self-employed people from all sectors of the economy. Lorna: Can you start off then just by telling me a little bit about your own background and how you’ve come to be working with data? Inna: Yes, sure. I did my undergraduate in politics and international relations and we had a couple of modules using statistics and SPSS. From there I started developing a passion for […]
What ever happened to Data Collection Reports for Surveys?
The good news is, it is a product that is still very much alive and in development, albeit it has had a long history of ownership and name changes! Back in the mists of time Data Collection Reports for Surveys was a staple component of the SPSS Market Research suite of tools. When the company SPSS and all its analytical software tools were acquired by IBM back in 2009 the full suite of tools were rebranded and Data Collection Reports for Surveys became IBM SPSS Survey Reporter. Arguably an improvement on its previous moniker which was a bit of a […]
Expert insight – Adam O’Shaughnessy, SBD Automotive
Through independent research, evaluation and strategic consulting support, SBD Automotive helps vehicle manufacturers and their partners create autonomous, more secure and better connected cars. Adam O’Shaughnessy is a Senior Specialist – Connected Car. Lorna: Can you tell me a little bit about your background and how you’ve come to be working with data and statistics and so on? Adam: My degree was in mechanical engineering. There’s obviously a fair amount of data analysis there. After I left university, I joined SBD Automotive – a market research and technical consulting firm that operates in the automotive sector. The head office is […]
Expert insight: Rob Woods, Analytics Solution Architect/Data Scientist, Watson FSS
Rob Woods has worked in the analytics industry for over 20 years. He’s currently an Analytics Solutions Architect working on IBM’s Watson FSS suite of products. All views here are his own and not IBM’s. Describe your own background and how you came to be working with statistics / data / analytics I started as a university lecturer and research assistant in the early 1990s working with econometrics and GIS software. When I look back on it now, we were focussing on areas which are now widely accepted as important work but at the time were seen as pure research […]
Choosing a predictive analytics project
At Smart Vision we’re in a pretty strong position to talk authoritatively about the reality of predictive analytics. That’s because we’re comprised of a team of veteran practitioners with decades of experience where we’ve all witnessed plenty of success stories but also one or two ‘data science’ train wrecks. Moreover, like anyone else, we’re exposed to the seemingly constant torrent of stories about the latest developments in machine learning, data science or AI. But we’re often struck by the fact that there seems to be such a focus on emphasising the power of analytics or on explaining how machine learning […]
Statistics in court: the story of a dataset
Like a lot of consultants working in the analytics industry, I’ve built up an extensive portfolio of materials to illustrate different kinds of applications and approaches. Some of these consist of files and slide decks used to explain quite esoteric procedures such as TURF analysis or Partial Least Squares. However, there are certain materials that can be used to demonstrate such a wide number of statistical and predictive analytics techniques, that I’ve found myself immediately reaching for them again and again over the years. One of these is the SPSS Statistics sample dataset ‘Employee data.sav’. Most statistical software programs come […]
What’s new in IBM SPSS Statistics v26?
In April of this year, IBM released the latest version of SPSS Statistics. Version 26 introduces a number of additional analysis procedures as well as new command enhancements. If you’re an existing SPSS user and you’d like to upgrade to v26 there’s more information about how to do that here. If you’re interested in trying SPSS Statistics for the first time then do please get in touch – we’ll be happy to help. New analytical procedures Quantile Regression In standard ‘least squares’ regression the model predictions are based on a single regression line. This line can be used to estimate the […]
Expert insight: Clifford Budge, Head of Ecommerce Data Science, major toy manufacturer
Lorna: Please could you start just by talking a little bit about your own background and how you’ve come to be working in data? Cliff: I went to university to do a mapping science degree, during which time I had a year out in industry. I worked as a surveyor for a company that was laying the cables for the first optical fibre telecommunications network in the UK. Within about six weeks of starting the job, I got access to the marketing data and thought I could use my mapping skills to create a visual representation of the sales and […]
I work for an analytics company, but I don’t want to talk about analytics
Here is an interesting confession. I work for an analytics company but I would really rather not think about or talk about analytics. Let me explain. I once worked with an operations director. He had a great team of analysts. He valued their work and was interested in the information they gave him but the mention of algorithms or spreadsheets made his eyes glaze over. When I met him for the first time he happened to be in the room where we were meeting his team. The technically-minded contingent quickly got caught up in fast-paced conversation. There was a feeling […]
Regular Expressions for IBM SPSS Modeler: performance comparison
The Regular Expressions for IBM SPSS Modeler node pack provides 4 nodes that integrate the power and flexibility of regular expression pattern matching into SPSS Modeler. However, some of these capabilities can be supported using the extension nodes built into SPSS Modeler and that begs the question – why buy the Regular Expression nodes? One obvious answer is ease of use. The extension nodes built into SPSS Modeler require expertise in either R or Python programming languages since they are general “code” nodes. Although many data scientists may already have that expertise, most people use SPSS Modeler because of its […]
Don’t overlook the value of quick wins in your analytics project
I was talking to a potential client recently who was frustrated by the lack of ‘real analysis’ happening in his organisation. He had a team of analysts working for him, most of whom were using Excel when he really wanted them to be using Python. Although we don’t generally recommend Excel for advanced analytics – see my earlier blog on this topic – it’s still a useful tool that can generate real insight and is heavily in use in many organisations. Quick wins are important This idea that ‘real analytics’ only happens if you’re using what are, for many people, […]
A first look at SPSS Modeler v18.2
In this video Jarlath Quinn takes a first look at SPSS Modeler v18.2 and demonstrates some of the new functionality that’s included within this release. IBM® SPSS® Modeler adds the following features in this release. New look and feel. A new modern interface theme is available via Tools > User Options > Display. For instructions on switching to the new theme. New data views. You can now right-click a data node and select View Data to examine and refine your data in new ways with advanced data visualizations. IBM Data Warehouse. Database modeling with IBM Netezza Analytics now supports IBM Data Warehouse. Gaussian Mixture node. A new Gaussian Mixture node is available on […]
Expert insights: Major Lester, founder of SPSS UK talks about fifty years of SPSS
This year SPSS is 50 years old. Development started in 1965 by a team of political scientists, frustrated at how much time they had to spent on manual data cleaning before they could start their analysis. The first release was in 1968 and by the end of the 1960s SPSS was in use in over 60 universities. SPSS expanded into the UK in 1985 and Major Lester was its first employee. We spoke to him about the history of SPSS, why it’s been so successful and how he sees its future. Lorna: Could you start off just by talking to […]
Three questions to ask when reading articles about artificial intelligence
You may have noticed by now that there seem to be a couple of recurring themes in the plethora of articles and news programmes about artificial intelligence (AI). These themes can be summed up as a) “The dangers of AI” and b) “The limitations of AI”. Articles addressing the dangers of AI tend to focus on issues such as the threat of widespread job losses to AI, the possibility of inherent bias (such as racism and sexism), the lack of transparency in decisions made by AI systems and, as a result, the inability to plead your case with AI (“Computer […]
Thinking of using spreadsheets for advanced analytics? Think again.
When we’re talking to potential clients about advanced analytics we often ask them what tools they’re currently using. More often than not they say they’re using spreadsheets. Spreadsheets are one of the most widely used tools for statistical analysis and of course, most businesses couldn’t run without them. However, when it comes to advanced analytics spreadsheets have some very significant limitations. Use them beyond their capabilities and the potential cost can be significant. As with anything, it’s important to use the right tool for the job. So, what are the things you need to consider when it comes to using […]
Expert insight – Paul Jackson, Head of Advanced Analytics, Bonamy Finch
Can you tell us a bit about yourself, your background and how you came to be where you are career-wise? I started my career in 2001 at Research International’s Marketing Science Centre, following a degree in Sociology and Social Policy. Back then I was working mainly on analysis of market research surveys. Over time I built up my expertise in branding and segmentation and then joined Bonamy Finch, providers of advanced analytics services to global clients around the world, in 2007. Back then we were mainly serving clients and agencies with statistical analysis and support on surveys, mostly Segmentation, but […]
Expert insight – John Gill, Head of Insight and Analytics, Betfred
Can you start off by telling us a bit about your background, and how you came to be working in the analytics field? I came to analytics via a somewhat circuitous route. My first degree was in psychology, then I did a postgraduate qualification in newspaper journalism. Obviously psychology is all about the science of behaviour which is very relevant to the field I’m now in – analytics and gaming. Newspaper journalism has also been very useful as it’s given me an ability to communicate facts in a way that’s compelling, and that people can understand. Professionally, I started out […]
Expert insight – Nick Di Paulo, Lead Customer Researcher, Hyde Housing Group
Can you start off just by talking a little bit about your background and how you came to be working in analytics in the first place? Yes, I did research methods at university and was experienced at using spreadsheets and things like that, and that’s where I picked up SPSS, and then I got into survey design and have been working with it for seven or eight years now. In terms of job roles, I’ve moved around quite a bit but in similar kinds of areas with a focus on not for profit, and I’m now in a Customer Insight […]
5 key trends affecting technical training provision in predictive analytics
I have been in the business of delivering software applications and solutions that have advanced and predictive analytics at their core, in one form or another, for last 25 years. In a recent conversation with colleagues we were discussing how the wider market has changed and, more specifically, how this has affected the way that people who analyse data, create predictive models and help their organisations use them go about developing their skills and keeping them up to date. It feels to us that the dramatic developments in the availability and use of technology have had an equivalent impact on […]
Expert insight – Karsten Shaw, Director of Analytics, Populus
Could you start off just by telling us a bit about your background and how you came to be working in the area that you’re working in with analytics? I studied statistics with economics undergraduate level, then I went on to do a Masters in statistics at the LSE part-time whilst I was working at GFK. I worked on a joint project between the LSE and GFK looking at using statistical models to forecast demand for consumer goods. I’ve stayed in the industry since then. I’ve worked across a number of large organisations, and I’m now at Populus. What’s your […]
Expert insight – Matt Harrop, Head of Yield Management and Analytics, Ladbrokes Coral Ltd
What’s your background? How did you come to be working in analytics? I came from an accounting background and then moved into retail where I worked across a variety of teams in finance, marketing analytics and strategy for various different retailers. I then left that to join a consultancy working with loyalty card data. I really got into analytics in that role, and from there I moved across into the gambling industry which of course is very rich in terms of customer data and analytics. I’m now Head of Yield Management and Analytics, leading a team of eight people doing […]
Expert insight – Emma Brooker, Customer Insight Manager, L&Q
What’s your background? How did you come to be working in analytics? I suppose accidentally really. I studied politics at university, then I happened upon a short-term contract for a market research company doing data entry and progressed from there. I found the work interesting and enjoyed the projects I worked on. I moved to L&Q six years ago and have progressed up to where I am now, running the customer insight team. Did you study statistics at university? I had done bits and pieces of analysis as part of my degree but it’s mostly been a case of learning […]
7 things you need to know about key driver analysis (KDA)
In most businesses it’s not enough to simply be measuring outcomes like customer satisfaction, sales, customer churn rates, subscription renewals, customer loyalty, cancellation rates and so on. To gain competitive advantage you also need to know what’s driving those outcomes. Which aspects of the service you provide most influence how likely someone is to renew their subscription at the end of the year? Which factors most drive recommendations? Key driver analysis (KDA) can help you to answer these kinds of questions. KDA enables you to look for relationships between aspects of customers’ attitudes, needs and behaviours that you’re interested in, […]
The benefits of survey research and how online platforms can make the research process more efficient and effective
Surveys remain one of the most popular forms of market research, and consequently we regularly work with customers on large survey-based market research projects. Most commonly this is customer satisfaction surveys, and these can be one of the most valuable forms of data that a company has. We also have clients who conduct broader market research, going beyond their own customer base in order to learn more about a particular market or segment in which they are interested. Of course surveys are not the only way you can find out what your customers think. Sometimes they’ll tell you what they […]
