AI Under the Hood: A Free Download You Still Can't Run (Part 2)
One of the largest AI models in the world is free to download.
You almost certainly can't run it.
WHAT A PARAMETER IS
An AI model is mostly a huge set of numbers called parameters. During training, the model reads trillions of tokens and nudges those numbers, again and again, to get better at predicting the next piece of text. In finance terms, think of them as the coefficients in an enormous formula.
(Disclaimer: this is the finance-friendly version. To any engineers or techies reading: yes, I know it’s nonlinear, there’s backpropagation, and pretraining isn’t the whole story. You may experience mild discomfort reading this version, but symptoms should pass by the next paragraph.)
Kimi K3, released by Moonshot AI, has about 2.8 trillion of them. It's the largest "open-weight" model released so far, meaning anyone can download it.
THE CATCH
However, free to download isn't free to run (all figures approximate):
• Even compressed, its parameters take roughly 1.4 TB of memory. • It uses only about 104 billion parameters for each token, but all 2.8 trillion still have to be loaded. • Running it yourself is estimated at $500,000 to $2 million in hardware.
AT WORK
Picture an IT director who hears "the model is free" in a budget meeting. The license costs nothing. The servers to run it are a capital request.
That's why almost every company uses AI through an app or a paid service. The real question, then, is never "is it free?" It's "what does it cost to run?"
NEXT
Overall, open weights are not the same as accessible.
If you found this helpful, give it a like and share it with someone who needs it.
Want the case for higher education in an AI workplace? Follow my companion series, The Educated Edge.
Next up, Part 3: the AI doesn't plan its answer. It guesses one piece at a time.
#AIUnderTheHood #AI #OpenSource #HowAIWorks
You almost certainly can't run it.
WHAT A PARAMETER IS
An AI model is mostly a huge set of numbers called parameters. During training, the model reads trillions of tokens and nudges those numbers, again and again, to get better at predicting the next piece of text. In finance terms, think of them as the coefficients in an enormous formula.
(Disclaimer: this is the finance-friendly version. To any engineers or techies reading: yes, I know it’s nonlinear, there’s backpropagation, and pretraining isn’t the whole story. You may experience mild discomfort reading this version, but symptoms should pass by the next paragraph.)
Kimi K3, released by Moonshot AI, has about 2.8 trillion of them. It's the largest "open-weight" model released so far, meaning anyone can download it.
THE CATCH
However, free to download isn't free to run (all figures approximate):
• Even compressed, its parameters take roughly 1.4 TB of memory. • It uses only about 104 billion parameters for each token, but all 2.8 trillion still have to be loaded. • Running it yourself is estimated at $500,000 to $2 million in hardware.
AT WORK
Picture an IT director who hears "the model is free" in a budget meeting. The license costs nothing. The servers to run it are a capital request.
That's why almost every company uses AI through an app or a paid service. The real question, then, is never "is it free?" It's "what does it cost to run?"
NEXT
Overall, open weights are not the same as accessible.
If you found this helpful, give it a like and share it with someone who needs it.
Want the case for higher education in an AI workplace? Follow my companion series, The Educated Edge.
Next up, Part 3: the AI doesn't plan its answer. It guesses one piece at a time.
#AIUnderTheHood #AI #OpenSource #HowAIWorks