Artificial Intelligence (AI) adoption in marketing so far has been mainly driven by predictive analytics based on behavioral and historical data, content personalization of web pages and conversational AI. But there is another domain that remains largely untapped: the analysis of unstructured text data to generate marketing insights and increase revenues. Unstructured data like emails, social media posts, requests for proposals or market reports traditionally have been a tough nut to crack for insight engines because of some intrinsic characteristics of natural language.

The richness of our vocabulary and the astonishing capacity humans have in expressing the same idea with different words are a major hurdle for automation. Text analytics software often relies on a savant construction of rules. For example, if the email header contains “proposal” or “request” or “inquiry” or “price”, then forward this email to the sales department. Despite an exhaustive list of conditions, they will never match humans’ creativity in articulating the same concepts in various ways. Consequently, an important share of incoming messages cannot be correctly forwarded. Precious time is lost before the message reaches the appropriate person. From a marketing perspective, low responsiveness has a negative impact on the probability of closing a deal and boosting revenues.

Marketing Technology News: New Conviva Data Signals Global Streaming Slowdown and Challenging Times Ahead for Connected TV Devices

In the past years, new technology advancements in the field of Natural Language Processing (NLP), especially the emergence of transformers models based on deep learning, have brought major improvements in analyzing natural language. However, they come with a caveat. They use terms as input features for their models, which means that they only understand terms they have seen during training. Consequently, extensive training data sets are required to reach a good level of precision – with 20,000 training documents, they might reach a precision level of 70%. The major problem is that most customer data sets contain less than 2,000 documents. In other words, term-based models fail at delivering acceptable results in most real-world use cases.

To accurately process the 5,000 to 10,000 of emails a company receives daily, NLP models need to be able to manage terms they previously have not seen. A new approach to text processing, called Semantic Folding, is addressing this challenge.

Semantic Folding augments machine learning models with outside knowledge, allowing learning algorithms to perform better with less data. Semantic Folding, in combination with regression algorithms for example, enables emails and social media posts to be classified with human-like accuracy. Our Message Intelligence product recently assisted a large pharmaceutical company mine information hidden in reddit posts to better understand the off-label usage of its products.

Marketing Technology News: MarTech Interview with Rick Zhuang, CTO at Firework

This new breed of intelligent semantic solution is based on a revolutionary representation of text called a semantic fingerprint. It preserves all senses and contexts of terms and enables disambiguation at a very granular level. A solution like Message Intelligence can help mine social media information regarding a customer’s intent and behavior, regardless of a social media posts’ brevity and the accrued usage of abbreviations, slang language and misspellings. It represents a huge opportunity for marketeers, since social media is an essential information source on customers’ habits and satisfaction.

As martech platforms upgrade their NLP capacities to the most recent technology advancements, they will be able to address other use cases like monitoring text streams in real time and triggering alerts when product mention spikes occur. Most importantly, these real time text streams and product mention alerts will be immediate, enabling marketing teams to craft customized offers and promotions in real time as well as react quickly to potential threats.

Last but not least, the next generation of martech platforms will leverage NLP to generate insights from documents containing a lot of strategic information. Identifying market trends from proposal requests (RFPs) is one example, as our latest Message Intelligence implementation with a publishing company indicated. Another interesting application of meaning-based text processing is to systematically search through scientific and legal documentation to verify the indisputability of marketing claims – a major challenge in the context of consumer protection laws and industry self-regulatory guidelines.

Marketing Technology News: The Hidden Value Developer Relations Teams Provide