While I was preparing my thoughts as a panelist on the upcoming Digital Analytics Association Symposium in Seattle, I was asked the question “what is the impact of big data on analytics?” and how has data analytics changed due to big data.
The answer, in my opinion, is “only in terms of processing and quality, but not in terms of analytics.” Data changed in volume, variety of sources/formats and speed by which they update (See HBR October article “Big Data: The Management Revolution). However, the fundamental data analytics techniques we apply today are the same as what we used 15-20 years ago.
Let’s use predictive modeling as an example. Before the internet, predictive analytics were mostly performed in database marketing (think credit card issuers use modeling to select credit-worthy customers to send applications to). Then came the internet in the 90’s and with it came the web site data. Today the data sources extend across multiple devices including tablets and mobile. This vastly increased the amount of “signals” analysts can use in their predictive models, making predictive analytics more interesting and challenging at the same time.
Meanwhile, the types of “outcome” data also became increasingly digital. One example is phone calls. 20 years ago, the traditional direct marketers used analogue advertising channels (magazines, postcards, flyers…) to drive analogue responses – phone calls and cash register rings. Today the digital marketers use digital channels (web, mobile, etc.) to drive the same analogue outcomes and new digital responses – mobile calls, online purchases.
However, we still apply the same predictive modeling techniques – logistic regression, linear regression, classification and regression trees, etc. Marketers are still interested in the behavior of the same customers, only that we have more (digital) touch-points to reach them and we are capturing more of their behavior from multiple devices.
Volume or Variety?
As an analyst, the biggest opportunity here is to take advantage of the “variety” of the data, not the “volume.”
For a statistician, “big” is not necessary for data. A good random sample can replace the “big data” and provide the same analytical results. Moreover, big data also means “big noise.” I see big data as more of a challenge to finding the right information from data.
The variety of data, on the other hand, puts analysts in a much better position to deliver deep insights. For example, if an insurance company invested in media to drive calls to their call center, we now have the technology to not only attribute calls to the media sources, but also classify calls based on the outcome. Since calls are strong indicators of purchase (20-30% average conversion rate from calls to purchases), they are better metrics to gauge marketing success and is a better outcome variable to build predictive models against. 62% of consumers call a business from a local search (source: Google Mobile data), the most over other outcomes. Marketers should take advantage of call data. The chart below (Source: BIA/Kelsey) shows how fast call volume (and therefore call data) has increased (and will increase) over time driven by mobile.
Another change in variety is that data mining is not just about numbers any more – it’s increasingly about text and context. Digital phone calls can be transcribed just like customer reviews can be captured in text. Mining text information provides marketers with deeper understanding of customers’ true intent and needs, new product opportunities and competitive information. For example, text mining results represented by word cloud, as illustrated below, can help an advertiser in education identify what are the most sought-after degrees and professional training.
In summary, the digitalization of information changed data analytics in terms of the volume and variety of data. However, we still need the rigor of traditional statistical and algorithmic methodologies to solve very similar problems as we faced 15 years ago. Analysts need to be creative in casting new problems in a simple ways and most importantly take advantage of the myriad of digital “signals” to generate new insights.
Principal Marketing Analyst