Years back, movies introduced us to the hi-tech, irresistible future where machines would be doing nearly everything what a human does. This future still seemed too distant as “machines writing intelligent letters” or even “machines understanding human language to carry out technical tasks” was too much to digest for a generation working on writing lines of code to establish an interaction between 2 applications.
The Age of AI seems to have just fast tracked things a bit. We are seeing many people researching, innovating and applying the technologies to carry out some tasks which were considered impossible just some years back.
This article is about one of such incredible language model. GPT3.
Want to design a HTML page real quick from just the requirement specifications?
How about having some expert to pull out and fill Company information data from his vast knowledge base and a thorough understanding of data. Seems we need an expert in this field?
Fascinating?
Before diving into what all things are achievable with this, first, lets understand what a language model is.
For example, if we had been hearing or reading a lot of negative stuff about Donald Trump, and we understand he has lost the elections. We infer : “This was bound to happen. With so much wrong policies implemented, he was not going to win.”
On the other hand, if we had been reading and hearing only positive stuff about Mr Trump and we hear about him loosing the elections, we infer : “Probably something went wrong with the campaign. The Democrats must have instigated the people.” The point is we make complete bipolar inferences based on what data we were fed.
Thus, on reading or hearing many such phrases inside lines and paragraphs, we develop our own unique way of interpreting new text (news) about a topic presented to us. it also gives us the ability to write about the topic on our own. This unique way of interpreting new text or learning to write something forms a unique model for every one of us which is built in our brains. Also the amount of data we read or hear gradually sharpens our understanding about any topic.
An AI language model is not too different. First it is fed with structured data which helps it to learn and understand certain words, phrases and emotions associated with them. Structured data, here, is manually classified data. In simple terms, this data is carefully classified by humans to teach the machine which word or phrase means what and which emotion is associated with it. This is also popularly called supervised learning.
As a model’s capability depends on the amount of data it is fed and since the amount of manually structured data is limited, the intelligence of this model is limited. It still can make certain inferences based on its learning, but it has limitations.
That is why it is fed with a huge amount of unstructured data. This unstructured data can be any data just available over the internet like wiki pages, forums etc. The interesting thing for this type of data is that, the machine is free to interpret it on its own. So, this form of learning is often termed as unsupervised. It makes inferences based on the past unstructured data, the article conclusive summary and the structured data it was fed earlier. Well, actually many other factors, but this is for the sake of understanding. So. just like a human brain we have a machine trained to infer on its own, predict the next words and interpret what we are talking to it.
In simple terms, all these learning which impart those super capabilities to a machine by which it can understand, predict text and answer questions which form its intelligence are stored in a file which we call as a “Language Model” file. Just like a human being, the amount of data it is fed to about a topic makes it better at responding or comprehending texts about that topic. Predefined language model files are huge in size and hence they expose their API such that anyone can use this model in their code.