Microgpt – Understand How LLMs Work

Recently Andrej Karpathy released microgpt, which is the essential functionality of LLMs.

If you are interested in knowing how ChatGPT works, this is for you.

Here is the main post:

https://karpathy.github.io/2026/02/12/microgpt

Github Gist:

https://gist.github.com/karpathy/8627fe009c40f57531cb18360106ce95

Also, it’s worth checking out the comments on the Github link, since people have added some interesting things there, for example, this guy made an entire illustration of the whole process:
https://ko-microgpt.vercel.app/

Finally, you can check out the YouTube channel here to learn more: https://www.youtube.com/@AndrejKarpathy