Latency
Level 1
The delay between a request and its response.
Term 3 of 6 in Building AI systems
In plain language
Latency is felt, not measured, by users. AI features are unusually sensitive to it because the model has to generate an answer piece by piece rather than look one up.
Think of it like this
The pause between ordering coffee and holding it.
Why it matters
Latency is a real product decision. A smarter, slower model can lose to a faster, simpler one.
Next terms in Building AI systems
- Frontend and backendThe frontend is what the user sees; the backend is the part that runs on servers.
- DeploymentMaking a new version of software live for real users.
- EvaluationSystematically testing whether an AI system's output is actually good.
- GuardrailsRules and checks that constrain what an AI system may say or do.