The time between sending a request and receiving the response, whether to an AI model, a page or a server.
Latency is the waiting time. On websites, it is the gap between tapping a button and seeing the result. In AI, it depends on model size, prompt length, answer length and the distance between the user and the server. Even fractions of a second add up when they repeat at every step.
Users judge apps by speed. A slow checkout page or an assistant that takes too long to answer can push customers away. That is why server proximity matters, one reason for in-region hosting, and why choosing a model of the right size matters too. Practical tip: for AI replies, stream text as it is generated instead of waiting for the full answer, which makes waiting feel shorter. Trim instructions and context, and measure speed from devices and networks similar to your customers'.