NLP, Natural Language Processing, is an area of Computer Science and AI, that is concerned with the interactions between computers and human languages. It is mostly used to apply ML models to text and language: teaching a machine to understand what is written and spoken, and the way humans learn and use language. Here is a bit of history about NLP, when and how it started, Alan Turing and his published paper, etc.
Have you ever Googled something and gotten the response, “did you mean ___ ?” That’s NLP schooling itself to learn users’ intent despite mistakes or common language styles that baffle most software’s search algorithms.
While computers are good at working with structured data like spreadsheets and tables, it is hard for them to understand human language and extract meaning from a text or a voice. Here comes NLP that is allowing to overcome these challenges and difficulties to understand unstructured data, which is hugely increasing in size every day. NLP is helping to analyze and draw insights from this data contained in emails, videos, and other unstructured material like Twitter, etc.
With the machine understanding the meaning of a sentence, understanding what a human is writing and saying, online translators are more useful, chatbots and virtual assistants are more helpful, Google is suggesting the finishing of a sentence, Siri and Alexa were born, filtering spam emails is more accurate, the customer’s opinion behind his tweet/post is more transparent, etc.
For the machine to achieve its goal in understanding the meaning of a text, NLP considers many steps and rules: splitting a paragraph into sentences and then into words to be checked, neglecting words like “the” and “a”, considering words ending with “s” as the plural of the same word without an “s” at the end, defining the importance of a word by the number of times it appears, pre-defining emotional words like “happy” and “sad”, etc.
On the other hand, fully understanding everything in every text is something only humans can do. Typos, rumor sentences, and many other cases are too hard for the machine to analyze: the ball doesn’t fit in the bag because it is too big, is about the ball being big not the bag, but, is this what the machine will understand?
Detecting fake news is an important scenario that uses NLP. It will be discussed deeply in later-on posts.
Computer Engineer • Entrepreneur • Blogger