So far the model has been answering in its default mood. But the request body accepts a few extra fields next to messages that change how the model picks its words. Three are worth knowing early: temperature, max_tokens, and top_p.

Temperature, kind of randomness dial:

Remember from the How LLMs Work post: the model never knows the next token, it holds a ranked list of candidates with probabilities. Temperature decides how that list gets used.

At 0, the model almost always grabs the top candidate. Same question, same answer, run after run. As you raise it, lower-ranked tokens start getting picked, and the output gets more varied. Most OpenAI-compatible APIs default to 1 and accept values from 0 to 2.

See it yourself:

import requestsurl = "https://algoholia.com/api/free-llm/v1/chat/completions"for temp in [0, 0.7, 1.5]:    payload = {        "messages": [{"role": "user", "content": "Give me one catchy title for a blog about Python type hints"}],        "temperature": temp,    }    response = requests.post(url, json=payload, timeout=30)    print(f"temp {temp} -> {response.json()['choices'][0]['message']['content']}")

Run it three or four times. The temp 0 line barely changes between runs. The 1.5 line is a lottery, sometimes a good title, sometimes modern art. Neither is "better", they are for different jobs.

max_tokens, the budget cap

This one caps how many tokens the model may generate in its reply. Sounds harmless until you see what it does:

payload = {    "messages": [{"role": "user", "content": "Explain FastAPI in detail"}],    "max_tokens": 30,}response = requests.post(url, json=payload, timeout=30)choice = response.json()["choices"][0]print(choice["message"]["content"])print(choice["finish_reason"])

The reply stops mid-sentence, and finish_reason says "length" instead of the usual "stop". The model does not write a shorter answer to fit the budget, it writes its normal answer and gets cut off. So if you want short output, ask for short output in the prompt, like our two-sentence rule in the summarize endpoint. Use max_tokens as a safety net so a rambling model cannot burn tokens forever.

top_p, the other chaos dial

top_p also controls randomness, from a different angle: instead of reshaping probabilities, it tells the model to only consider the smallest set of top tokens whose probabilities add up to p. With top_p: 0.1, only the elite few candidates survive. The practical advice is short: tune temperature or top_p, not both. Pick temperature, it is easier to reason about, and leave top_p alone until you have a reason not to.

Back to our Blog

Our app now wants two moods at once. Summaries should be boring and reliable. Title brainstorming should be wild. Same model, different dials:

def call_llm(payload):    response = requests.post(LLM_URL, json=payload, timeout=30)    if response.status_code != 200:        raise HTTPException(status_code=502, detail="The LLM did not respond, try again")    return response.json()["choices"][0]["message"]["content"]@app.post("/blog/summarize")def summarize_blog(blog: Blog):    payload = {        "messages": [            {"role": "system", "content": "You are an editor for a tech blog. Write summaries in exactly two sentences, plain language, no marketing fluff."},            {"role": "user", "content": blog.content}        ],        "temperature": 0.2,        "max_tokens": 150,    }    return {"title": blog.title, "summary": call_llm(payload)}@app.post("/blog/title-ideas")def title_ideas(blog: Blog):    payload = {        "messages": [            {"role": "user", "content": f"Suggest 5 catchy titles for this blog:\n\n{blog.content}"}        ],        "temperature": 1.3,    }    return {"ideas": call_llm(payload)}

Let's understand this:

1. The LLM call moved into a call_llm helper. Both endpoints were about to repeat the same request and error handling, and we are too lazy to write things twice.

2. /blog/summarize runs cold at 0.2 with a max_tokens safety net. Facts in, same facts out, every time.

3. /blog/title-ideas runs hot at 1.3 with no cap, because creative output is the whole point there.

A rough guide for picking temperature: 0 to 0.3 for anything with a right answer (summaries, extraction, classification), around 0.7 for a natural chat feel, 1 and above for brainstorming. Past 1.5, you are doing it for entertainment. 🎲

One thing we keep ignoring: our chat history from last post grows every turn, and every token of it gets sent, billed, and processed on every request. There is also a hard ceiling on how much the model can even accept. That ceiling is called the context window, and it is next.