Why Sentiment Analysis is not good enough to move towards a new level of understanding

By Tom Krupa, On 7th May 2019

Reviews are the richest source of information. They have a lot of relevant information and they are a validation made by real users about current hotel/business. And yet, for computers, they are still undiscovered. Today’s technology allows only for shallow understanding. It can categorize reviews but to fully understand a review you still need to read it. One by one. If you have 1000 reviews about a specific place then you pick a few or spend hours before going on a “perfect vacation”. If we could find a new level of understanding that could capture even 90% of relevant information coded in a review and display it to the user in a way that he could understand, then we gain a lot. Imagine, that any user, in 3 minutes could read, not 5 reviews about the specific place, but 5000. Imagine, that an algorithm could compare thousands of hotels, read all of the reviews and provide you with a short list of ones that match Your personal needs. To do that, we need to go deeper than the Sentiment Analysis.

Sentiment Analysis was the first widely adapted model of language understanding. It was widely used in all domains: e.g. hotels reviews, product reviews, airlines, brand monitoring and even to predict the stock market. It was easy to understand, easy to scale, easy to present on charts, timelines and not so hard to test if it works properly. You do not need sophisticated knowledge to say if it is a positive or negative opinion. Most of the time. But in some cases, it is not that clear. There is a famous saying, that even people agreement has its limits and it is 82% correct (when testing Sentiment of a text). But let’s go deeper here, why is that? Why people can’t decide among themselves if something is positive or negative? In my opinion, the model is a mathematical description of the world, and it is very simple. It is a simple one-dimension description of the text. But the language is not that simple. The world is not black and white. It has colors. Lots of it. Like a rainbow. If we are going to build more clever language understanding application, we need to make a more sophisticated model of understanding. We need to find a new, deeper way of describing information. I am not telling about implementing grey color into black and white one-dimensional description (very positive, slightly positive, neutral, slightly negative, very negative) or mathematically speaking {-2,-1,0,+1, +2} instead of {-1,0+1}. I am telling about implementing more dimensions. Semantic dimensions.

Wait! But there is Aspect-based Sentiment Analysis. This is a more relevant description of the world! It is true that we can detect more objects/aspects and tell the sentiment about every object. But in terms of description, I would disagree. It is still black and white scale. To go deeper, we need to change not only the resolution of objects (aspects) that we are describing but also the description. Let me explain using these simple examples:

You cannot describe Breakfast just telling if it is positive or negative.

Imagine, that people would talk in this way:
- How was Your Breakfast?
- Thank You, It was slightly positive:)
- And how was Your day?
- Neutral:)

Next phase is to create a new way of description. Multidimensional description. If we have a semantic description which is human-like form, people annotation would not be 82% correct but close to 100% correct (or 99% to leave a little bit of magic to the language). The 82% annotation agreement means that the model is broken and even experts cannot decide among themselves. For one person small portions might be neutral or even positive, for another negative. We need to get our hands dirty in Semantics/Pragmatic description of the world. But, I would like to call it a Human-like description of the world, not to stuck with scientific theories and debates. If You would have the human-like description, not a mathematical one, there will be no disagreements. You could detect more human-relevant values about Breakfast: tasty/not tasty, big portions/small portions, free/extra paid, and there would be no problem to describe given examples:

You could, of course, value mathematically tasty +1, not tasty -1, big portions +1 etc. however description should be semantically driven, not model-driven. The downside of this approach, it is much more work to do and solutions are domain specific. You cannot make one model of description (positive/negative) and apply it to every object/aspect and just change keywords which they are referring to. For example, You cannot describe Bed: not tasty, or Room: small portions. You need to go one aspect by one and describe each of them separately. Or not even aspect but to think broader every bit of knowledge that matters inside the review. But in my opinion, it is worth it. And once you have done this model, everything later is much much easier. If you do that, a lot of new possibilities arise. And frankly, I do not see the other way. If we are going towards real language understanding, we need to get to the point, where you transform unstructured text into structured knowledge. This structured description of knowledge should capture most of the information coded in the text. And if You achieve that, then the magic happens. Then you could perform the same basic operations on a text as you do on numbers. You could add/subtract/compare unstructured texts (e.g. reviews, tips) the same as you compare structured data (e.g. star rating). You could find similarities between reviews and match places on a new layer of understanding. You could deliver a human-like quality of analysis automatically for a fraction of a cost. You could provide a new interface of super fast reading reviews for mobile, not one by one, but by the thousands. You could compare millions of places from all over the world, find similar places you have been before. Or just select your preferences and let the computer do the dirty job. You could build a real semantic search on reviews or any other text. What is more, most of the machine learning algorithms will work better on pre-organized knowledge.

We developed Language Understanding API designed for hotel and other travel reviews (hostel, apartment, restaurant, airline, food delivery) which provides a human-like level of understanding. We feel that we moved beyond Sentiment Analysis. We call it Semantic Analysis. If You want to see our API in action, check our demo.

By Tom Krupa, On 7th May 2019