Tagged
Architecture
Tokens, context windows, and why your bill is what it is
The unit of billing for language models is not the request or the word. Once you understand tokens and how a context window fills up, the invoice stops being a surprise.
Queues are not optional
If your request handler calls somebody else’s API, you have already built a distributed system. The queue is how you stop their bad day from becoming your bad day.
When not to use a model
Half the AI features I have been asked to build should have been a regular expression, a lookup table, or a better form. Here is how to tell before you spend the sprint.