The Elusive Holy Grail of Consumer Data Science and Intelligence

Understanding consumer data science is very important in the current context. In today’s marketplace, brands, agencies and organizations must have fast, streamlined access to complex consumer-based data in order to stay competitive. Meanwhile, sensitivity around data collection, responsible use, and legislation has changed the climate in which we operate. External pressures from the global pandemic, widespread natural disasters and societal upheaval are also causing the need for consumer insights to speed up, as people change their sentiment and behaviors quickly.

Consumer Data Science in Market Research

In market research, the holy grail of consumer data has long been behavioral data – or also referred to as observational data. This may include purchase data or media consumption. Due to challenges in collecting this data, many have resorted to survey, or claimed, data as the basis of our insights. To obtain information, we ask consumers to recall things like: How much did you spend at Amazon? What did you watch on TV last night, at what time and on what channel?” We understand all too well the limitations around claimed data.

Because we have more control over discovering information with a survey, and the promise of behavioral data hasn’t come to fruition, we’ve come to rely on it as a vehicle to obtain data – any data – about audiences. But this approach only gives us a small part of the picture. In fact, in order to really start to understand our customers, we must marry both types of data – survey and behavioral.

This has proved to be an elusive goal at best for those in the market research industry. But why?

Poor and clumsy user experiences have kept consumers from adopting any methods that allow the collection of behavioral data. Low levels of trust in companies and the data collection process, in general, is further hampering progress.

The industry has not been able to or not been willing to compensate users effectively to make it worth their while. We found in a recent study that 78% of participants expressed interest to connect to passive data collection for appropriate rewards.

Technology platforms such as mobile app stores are placing restrictions on what applications can do to collect this data, partly due to pervasive concerns about trust and a perception that collection is not in the best interest of the user. In addition, cookies, which have been a critical element in data collection, are being phased out.

Legislation is creating real consequences for data collection and data has become a liability for some organizations, and consumers are growing weary of data collection practices.

The need for consumer-based data will not go away. Demand will continue to go up, as both complexity and data sensitivity rise. In order for brands to achieve the insights they need for decision making, consumer needs must be put at the forefront. We must begin to communicate the value of their data, pay them accordingly, and provide them with experiences and business practices, not just technology, that builds trust with them. This takes time, but it is essential to the future of market research and consumer insights.

The next era of data collection, behavioral and beyond, will be when these consumer-centric elements are placed into play as one. As a society in general, we all benefit from sharing data, but only when it’s done responsibly, and in the best interest of individuals – not the organizations. Maybe this approach will facilitate the marriage of data, behavioral and claimed, and we can behold the holy grail of consumer insights.