Breaking Down the Magic: A Casual Chat About How ChatGPT Actually Thinks

If you have ever had an extended conversation late at night with an AI system, discussing everything from how to write a professional email to your manager. Or even asking for a recipe for the newest version of the world’s best-known sambal dish, you have likely reached the point where you say: “How does he know this? It feels like [insert someone who is extremely intelligent], but really it is just an incredibly advanced math formula. When we consider the technology architecture of ChatGPT explained at a level that average users will understand, the best way to visualize this technology would be to stop thinking of it as a database or encyclopedia of information. It doesn’t store information and ‘know’ it in the same manner that we do. It is more like being the world’s most advanced, powerful, and efficient smartphone autocompleting function for words and sentences.

Why does it feel like AI “thinks” before it speaks?

When you’re driving around Kuala Lumpur and see a car with its indicator on, your brain already knows it’s about to turn. That is to say, I don’t need to refer to the manual or read the laws, I’m just assuming you’ve done it before. This is essentially how ChatGPT works. Model was trained with lots of information on the web (blog posts, news articles, novels, posts on Reddit). So it can learn how likely a given piece of information (prompt) is to have certain things happen next. For example, a neural network can predict, after the above sentence, “will the next word be hot, or humid?” or “will the next word be snowy?” with 90% accuracy for hot, and 0.01% accurate for snow.

Rather than relying on a database of previous answers, ChatGPT employs an inference methodology every time you submit a prompt. This means the neural network is running through one-by-one analyzing all of the words in your question based on what you provided (i.e., the given input) and looking for the most appropriate response (the output) by utilizing the patterns it learned during the training process.

ChatGPT technology architecture explained through Transformers

Model architecture, which serves as one of the core components of GPT. Prior to 2017, AI was very “blurry”; AI still forgot the first half of a longer sentence while it was working on the second half. What makes Transformers revolutionary is that they allow the model to consider all the words in the sentence simultaneously. In other words. Prior to Transformers — reading a book word-for-word as if you were a young child. With a Transformer — scanning the complete surface area of the entire page to instantaneously understand what “it” refers to in the previous statement as “Durian.”

The attention mechanism in GPT makes this possible, which is similar to a spotlight behind the scenes. The attention mechanism identifies which other pertinent words, or words of interest, are included in your prompt while the AI is generating the next word in its vocabulary. As an example, if you asked the question “How do I get my car repaired?”. In this instance, and many others, the “spotlight” of the attention mechanism is directed toward the word “car”.”fix,” “car,” and “engine” to make sure the answer isn’t about fixing a broken heart or a computer software bug.

How the ChatGPT technology architecture explained the concept of “Tokens”

You might hear about “tokens” when people talk about technology. Think of tokens as ‘Lego’ bricks for language. Tokenizing in a language model is the way that AI breaks your sentence down into smaller sections. Sometimes tokens are whole words (such as ‘eaten’), while other tokens are one or more letters (such as ‘ing’). In other words, when AI sees the word ‘Apple’, it doesn’t really see that word; instead, it sees some unique identifier that refers to that particular Word.

Because of this, AI often has difficulty performing exact calculations or spelling because it is looking at “The Token Patterns” instead of just looking at the letters of the word. Token-Based Generative AI as explained above explains why this model works very quickly. It does not need to “understand” the essence of a word. It is just rearranging Mathematics Representations (Vectors) that exist in multiple dimensions.

The “Gym Session” for AI: The training process ChatGPT model

What led it to become so incredibly intelligent? This wasn’t just something that occurred spontaneously. The data training pipeline for ChatGPT focuses on two areas. Pre-training: The artificial intelligence reads the Internet in its entirety. It is similar to a student being trapped in a library for an entire year. This is a crucial component for the AI in its ability to learn grammar, facts, and code. This is in addition to establishing the overall architecture of the AI’s large language model.

Fine-tuning (RLHF): The second component of the training process is the “fluffing” or polishing stage. During this phase humans will evaluate the answers from the AI’s output. Humans assign the AI’s output either an excellent or bad rating based on whether or not the response was helpful to them or was rude in nature. The process described is called “Reinforcement Learning from Human Feedback“. Therefore, it can be said that ChatGPT’s output feels much more human-like and useful when compared to those attempts from an AI produced in past times.nnoying chatbots you find on banking websites that can only answer three fixed questions.

The ChatGPT technology architecture explained for daily use

Once we recognize the use of probability and patterns in the ChatGPT neural network, our ability to implement it increases. We know ChatGPT functions by way of predictions based on how much ‘context’ we provide it with to accurately predict responses. When you’re saying, “write a caption,” you’re forcing ChatGPT to predict results without any prior knowledge. By providing ChatGPT with additional tokens, such as; “Write me a caption for a café in Penang that sells sourdough to young adults with a chill and MY style tone,” you aren’t just telling ChatGPT what you want. You’re actually providing the ‘attention’ that the attention mechanism requires to generate an output that matches my personality.

Many companies like TechPulse are beginning to realize how this new technology is changing the way we operate in Asia. This will not be a replacement to employment. Rather, it will allow employees to work 10x more productively than before if we take the time to figure out the ‘logic’ behind the ‘how’. Whether in PJ as a student or SG as an entrepreneur, knowing the ‘why’ with the ‘how’ is what separates you from everyone else.