ChatGPT: How Does It Work Internally?

Launched by OpenAI on November 30, 2022, a new conversational assistant is making a buzz on the Internet with the incredible quality of the answers it provides.

Photo by Romain Vignes on Unsplash

This chatbot is based on the language model (LLM for Large Language models) GPT-3 or, more precisely, on its version 3.5. ChatGPT is in fact an adaptation of InstructGPT, which was launched in January 2022 but did not make the same impression at the time. Probably due to the difficulty of accessing it and possibly due to the model being 100x smaller than ChatGPT.

ChatGPT owes its incredible success to its ability to automatically generate text similar to that of a human being and its ability to take into account the context of a conversation while avoiding the shortcomings of its ancestors, such as Tay from Microsoft or Galactica from Meta.

Tay had become racist and xenophobic in 24 hours. Galactica was generating nonsense and misinformation and could give its opinion on racism in a very convincing way. Tay was shut down within…