How Does an AI Model Work? Think of it as a Box

Model, LLM, parameters, training, inference. These words show up in every meeting now, and they sound heavier than they are. When I started running AI workloads on the platform side, I wanted one mental picture that made all of them click. This is the one I use: a box.

What is a model?

A model is a box. Something goes in, something comes out.

Input → Box → Output

All the interesting stuff happens in the middle. The box runs some operations on your input, and those operations are shaped by a set of factors. Put the operations and the factors together and you have what people call "the model".

Boxes you already use

The input can be numbers, text or images. You use all three kinds every day without thinking about it.

• Numbers in, a number out. The arrival time in your maps app. Start and destination go in, the box weighs distance, time of day and traffic, and "25 minutes" comes out.
• Text in, text out. The next-word suggestion on your phone keyboard. Chat assistants do exactly this, just at a much bigger scale.
• Image in, a label out. Face unlock. A photo of your face goes in, and the box answers one question: owner or not?

Same pattern every time. Input, box, output.

What is inside the box?

Not a person. Not a giant database of stored answers. Just math.

Math only works on numbers, so everything gets converted first. Text is split into small pieces called tokens, and each token turns into a list of numbers. An image becomes a grid of numbers, one per pixel. Those numbers then go through layer after layer of multiplication and addition.

So what decides how that math behaves? Those "factors" from earlier. They're called parameters, or weights.

Picture a sound engineer's mixing board, except this one has billions of tiny dials. Each dial nudges the input a little as it passes through. Where all the dials sit decides what comes out the other end.

How do the dials get set?

Nobody sits there turning billions of dials by hand. The box learns their positions, and that process is training.

Take next-word prediction. We show the box a sentence with the last word hidden:

"The sun rises in the ___"

The first guess is basically random. We compare it with the right answer, "east", and measure how far off it was. Then every dial gets nudged a tiny bit in the direction that would have made the guess better. Repeat that billions of times over a huge pile of text, and the dials slowly settle into positions that work.

Think of tuning an old radio. You turn the knob until the static clears. Training is the same thing with billions of knobs and a computer doing the turning.

Once training is done, the dials are frozen. Every time you use the model after that, it's called inference: new input goes through the box, dials stay exactly where they are.

So what is an LLM?

LLM means Large Language Model. Text goes in, text comes out. "Large" is about the number of dials, which runs into the billions.

Here's the flow when you send it a message:

1. Your text is split into tokens, and the tokens become numbers.
2. The box scores every possible next token by how likely it is.
3. One token gets picked and added to the text.
4. The new, slightly longer text goes back in, and the loop runs again, one token at a time, until the answer is done.

That loop is why answers stream onto your screen word by word. Under the hood it's your phone's next-word suggestion, scaled up massively and trained on far more text. Scale is what lets it explain, summarize and write code.

Why this matters for AI infrastructure

This is the part I care about most, because it's where my day job comes in.

Those billions of dials are just numbers, and numbers have to live in memory. Quick math: a 7-billion-parameter model at 2 bytes per parameter needs about 14 GB just to hold the dials, before it serves a single request. Then every token needs a mountain of multiplication, which is exactly what GPUs are built for.

Looking at it from the ops and security side, the box picture gives you a useful checklist:

• The dials are a file. Model weights are an artifact like any container image. Pull them from a trusted source, pin the version, verify the checksum, and control who can download them.
• Inputs are untrusted. Whatever goes into the box can steer what comes out. Treat prompts like any other user input and don't hand the output straight to something with production credentials.
• Every token costs money. The loop runs once per token, so longer prompts and longer answers mean more GPU time. That's a cost line and a capacity-planning problem.
• Outputs need an audit trail. Since the box can be confidently wrong, log what went in and what came out, so you can trace a bad answer back later.

So the box isn't magic. It needs memory, compute and guardrails, and running it reliably, securely and without burning money is what AI infrastructure is about. That's what I'll write about next, starting with what actually fills up a GPU's memory when you send a request.

Key takeaways

• A model is a box that turns an input into an output.
• Inside is plain math, controlled by billions of dials called parameters.
• Training sets the dials. Inference uses them, frozen. An LLM runs that box in a loop, one token at a time.
• On the platform side, treat weights like any other artifact, inputs as untrusted, and tokens as cost.

Try this today: pick a model you use and find its parameter count. Multiply by 2 bytes and you'll know roughly how much GPU memory it needs just to load. It's a quick way to start reading model cards like an infra engineer.

If something here didn't land, or you want me to go deeper on one part, ping me on LinkedIn or X and I'll pick it up in a future post.