How AI can help to understand your customers - Hyntelo

Nowadays a key factor for the success of companies is to understand their customers: what they think and feel. This is crucial when launching a new product or service, but also when improving an existing one. Collecting and analyzing customers’ feedback during their journey with the company is absolutely crucial to better tailor the experience with the product or service and meet customers’ needs.

In the past, companies could rely only on surveys and interviews to collect customers’ feedback and on the gut feeling of marketing teams to analyze it. Today, with the increasing digitization, the touchpoints between companies and customers have multiplied. Therefore, companies have now two kinds of customer feedback data that they measure, store, and analyze: structured and unstructured data.

• Structured data is information that is clearly defined and easy to report on. It is the kind of data coming from a survey and can be organized in a spreadsheet: name, location, age, and rating (3 out of 5 stars, for example, or a 10 for “most satisfied” versus a 1 for “least satisfied”).
• Unstructured data consists of text, although it can also include other media such as audio, photos, or videos. Unstructured data can be captured in an email, the “additional comments” section of a survey, voice recordings of customer interactions, a post on a customer review site, social media, call center notes, chat transcripts, and dozens of other sources.
  
  

In this context where a huge amount of data is available to be analyzed, marketing teams struggle to cope with it manually. This is where natural language processing (NLP) comes in. NLP is a discipline that brings together linguistics, computer science and artificial intelligence with the aim of understanding the content of documents, including the contextual nuances of the language within them.

The NLP algorithms most commonly used to understand customer feedback are topic modeling and sentiment analysis.

Topic modeling 

Topic modeling (or topic extraction) is an NLP technique that allows the machine to extract meaning from text by identifying recurrent abstract themes or topics represented by the most relevant keywords.

Four main methods exist for topic modeling:

1. A probabilistic model called LDA or Latent Dirichlet Allocation
2. A Linear-algebraic model called NMF or Non–negative Matrix Factorization
3. Top2Vec based on the doc2vec model and clustering techniques
4. BERTopic that leverages transformers and c-TF-IDF

Topic modeling is a very powerful technique. It is an unsupervised method able to analyze a large number of documents and automatically discover the main concepts without having prior knowledge on the subject. Moreover, since customer feedback continuously evolves, new topics develop while others die, topic modeling is a dynamic approach that allows reflecting these changes as close as possible to real-time interest change, by creating new categories and merging old ones.

Sentiment analysis

Sentiment analysis (or opinion mining) is an NLP technique that focuses on the polarity of a text (positive, negative, neutral) but it also goes beyond polarity to detect specific feelings and emotions (angry, happy, sad, etc.), urgency (urgent, not urgent) and even intentions (interested v. not interested).

Many emotion detection systems use lexicons (i.e. lists of words and the emotions they convey) or complex machine learning algorithms.

By combining topic modeling with sentiment analysis to analyze customers' feedback companies can:

• follow how the sentiment related to a certain topic evolves over the time
• discover why customers are happy or unhappy at each stage of the customer journey.
• identify what are the main pain points related to a product
• monitor customer perceptions in real-time
• prioritize areas for action

These are only a few examples of the big potential NLP has in enhancing a company’s understanding of its customers. In the not too distant future, AI could be so empathetic and intuitive to anticipate customer needs and feelings.

Product review vector created by vectorjuice -www.freepik.com