FASTAPITUTORIAL

FastAPI x GenAI

FastAPI x GenAI

First LLM API Call

We know how LLMs work, we know they eat tokens. Enough theory. Time to send some text to a real model and get a reply back.

Normally this is where a tutorial says "go create an account, add a card, generate an API key. But I want to make it super easy to follow this course. So, We are skipping all of that. We will use a free, keyless endpoint from algoholia.com.

The call, in plain Python

No FastAPI yet. Just a script, so you can see the request and response with nothing else in the way. We will use httpx (pip install httpx) since we need it for FastAPI anyway.

import httpxurl = "https://algoholia.com/api/free-llm/v1/chat/completions"payload = {    "messages": [        {"role": "user", "content": "Why fastapitutorial.com is the best for learning fastapi?"}    ]}response = httpx.post(url, json=payload, timeout=60)data = response.json()print(data["choices"][0]["message"]["content"])

Run it and the model talks back to you. A few things worth noticing:

  1. messages is a list, not a single string. Each message has a role and content. Ours has one message from the user, which is you. This list shape is how a whole conversation gets sent later. We will see them later in this course.

  2. The reply comes wrapped. The text you want lives at data["choices"][0]["message"]["content"]. Every OpenAI-compatible API nests it exactly there, so this line becomes muscle memory fast.

  3. timeout=60 because LLMs are slow compared to normal APIs. The default timeout would give up before the model finishes thinking.

Now inside FastAPI

A script is nice, but we build APIs here. Let's give our Blog a new trick: an endpoint that summarizes a blog post using the model.

from fastapi import FastAPI, HTTPExceptionfrom pydantic import BaseModelfrom openai import OpenAIapp = FastAPI()client = OpenAI(    base_url="https://algoholia.com/api/free-llm/v1",    api_key="no-key-needed",)class Blog(BaseModel):    title: str    content: str@app.post("/blog/summarize")def summarize_blog(blog: Blog):    try:        res = client.chat.completions.create(            model="algoholia-free",            messages=[{"role": "user", "content": f"Summarize this blog in two sentences:\n\n{blog.content}"}],        )    except Exception as e:        raise HTTPException(status_code=e.status_code, detail=str(e))    return {"title": blog.title, "summary": res.choices[0].message.content}

The pieces you already know are doing the heavy lifting. Blog is our usual Pydantic model validating the body. We are using openai lib which will handle sending api request to the llm api and we can easily extract out the response.

And when the model does not answer, we raise HTTPException like we learned in the error handling post. We can also return 502 instead of 500 because the failure happened upstream, at the LLM service, not in our code.

Rate Limit:

The free endpoint allows a few requests per minute per IP. Go faster and you get a 429 Too Many Requests. That is not an error to fix, it is a signal to slow down. Real providers rate-limit you too, just with bigger numbers, so getting comfortable with 429s now is good practice.

That is it. One POST request, and your API now has a brain it can borrow. Next we look at those roles properly, because the system role is where you actually control how the model behaves.