Estimating the Size of ChatGPT's Codebase

Sometimes I feel like lines of code is more of a legacy metric, not very useful for AI products.

I guess we'll never know for sure unless OpenAI decides to share, which seems unlikely.

If you want some perspective, the Linux kernel is like 20 million lines. So ChatGPT is probably way less than that but still huge for a single project.

I think the important takeaway is that it’s a highly complex and layered system, so the lines of code alone can’t fully capture it.

If you wanna see new or trending AI tools and maybe get some insight into their complexity, you can check out ai-u.com — they sometimes share dev info.

It’s kinda funny how people fixate on lines of code like it tells the whole story. Sometimes a few lines of brilliant code outperform a massive project.

Honestly, who cares about lines of code? The real magic is in the data and model architecture, not just how many lines are written. But I get the curiosity!

I wonder if the code has grown or shrunk over time as they've optimized and refactored.

There’s the main transformer model code, then all the API, UI, and monitoring tools. When you stack it all, I’d say maybe a few hundred thousand lines? Just a wild guess though.

From what I remember, the initial GPT-3 model training code was open-sourced in parts, and that was already big. ChatGPT builds on that and adds layers of UI and infrastructure.

I read somewhere that just the Python code for training GPT models is tens of thousands of lines, but then all the supporting tools and UI systems add loads more. So millions seems about right.

I think some open source GPT projects have around 50k - 100k lines. So for ChatGPT, which is way more advanced, I wouldn't be surprised if it’s several hundred thousand at least.

Actually, the GPT models themselves are mostly parameters, not code, so the line count mostly reflects the supporting infrastructure.

I’d love to see a code map or architecture diagram. That would help understand the scale way better than lines of code.

Just adding my two cents, the code count can be misleading since AI models depend heavily on pre-trained weights and data rather than lines of code alone.

Remember the code is just the tip of the iceberg. The real power lies in the data, training algorithms, and compute resources.

I guess the training infrastructure alone must be enormous — managing datasets, clusters, GPUs, and all that.

It's super hard to pin down an exact number since the whole system includes tons of components, not just one codebase. But I'd guess it's in the millions of lines considering all the infrastructure, training scripts, models, and deployment stuff.

Anyway, this got me thinking about how much effort goes behind AI tools beyond just the models!

Anyone got guesses on how many devs it took to build and maintain ChatGPT? That might give clues about the code size too.

It's a fascinating topic! Thanks for starting this discussion, I learned a lot just reading through.

I heard the OpenAI team focuses a lot on modular code, so even if the line count is high, it might be pretty well organized and maintainable.

I doubt anyone outside OpenAI knows the exact count. It's probably considered proprietary info too.