9. Asynchronous programming for FastAPI

Know what, I have kept 🍕pizza in the oven, and meanwhile trying to write this article!
Oh, but that's not what you might be interested in. You are here to study asynchronous programming.

Let's first understand synchronous programming. It can be understood as a sequential way of processing things. You might have studied several other blogs too and may already have a theoretical understanding of it. So, let's cover the practical aspects of asynchronous programming.

Let's say these 3 requests hit our server. This is how synchronous frameworks will process the request.

Request 1: It requires talking to a mail server and sending the forgot password email.
Request 2: It requires talking to the database to search for the blog with id 23.
Request 3: It is a simple request that just adds the numbers 3 and 4 and returns the result.

You might have observed a problem with this approach. If requests 1 and 2 are too large then though request 3 is small, It starves for its chance. Say, we attempt to send 100 newsletter emails in request 1, In that case, request 3 will starve to connection time expired error!  Let me quickly check if the pizza🍕 is ready or not.

It will take some more time, So, we were discussing synchronous programming and saw that sequential tasks may lead to starvation. To improve the performance of synchronous tasks we need to understand exactly why the problem arises. Let's take request 1 and expand it to have a holistic view.
Let's review what's happening. Request 1 is by some website user, who has filled our forgot password form in the frontend and sent the request. Majorly the below operations are performed:

• FastAPI does some setup-related tasks and figures out which endpoint/path operation this request should hit. Since most of these tasks are done in the CPU, These are extremely fast operations. My CPU's current speed shows 3GHz which means that it can process 1 micro-operation in nanoseconds. (10^-9 sec)
• We check if the email entered exists in the database or not. If a user with that email exists, then we are going to send an email. This involves talking to the database on a hard disk, Harddisks are super slow when compared to CPUs. The average seek time is in milliseconds (10^-3 sec). This looks like a problematic thing as it is around multiple 100 thousand times slower than the CPU. The CPU is idle during this time.
• We generate a unique token, that we will inject into forgot password email to identify the user. Token generation is done in CPU so, no issues, it's quite fast.
• The next operation is talking to a mail server and waiting for its response. This is even slower than talking to the database. It involves network calls and it may take few seconds. Again CPU is idle for this time.
• Finally, we send a response to the website user that includes this message "We have successfully sent an email".

I know I am lazy😁 however, If I assume myself to be the CPU, even I would be irritated. This is such a waste of resources. We could have at least processed some parts of request 2 and request 3. This is the crux of asynchronous programming. We try to utilize the time for these I/O-based tasks. Had we adopted asynchronous programming the above situation would look something like this.

The above example of concurrent processing is possible by the use of asynchronous programming. Enough theory, let's send 10 requests to an URL endpoint and see the difference ourselves.

# Synchronous Program

import time
import requests
def main():
    request_count = 10
    url = "https://httpbin.org/get"
    session = requests.Session()
    for i in range(request_count):
        print(f"making request {i}")
        resp = session.get(url)
        if resp.status_code == 200:
            pass

start = time.time()
main()
end = time.time()
print("Time elapsed: ",end-start)
# Output: Time elapsed 4.87 sec

This is an asynchronous version of the same program. It makes use of aiohttp to make non-blocking API calls.

import time
import asyncio
import aiohttp

async def make_request(session, req_n):
    url = "https://httpbin.org/get"
    print(f"making request {req_n}")
    async with session.get(url) as resp:
        if resp.status == 200:
            await resp.text()

async def main():
    request_count = 10
    async with aiohttp.ClientSession() as session:
        await asyncio.gather(
            *[make_request(session, i) for i in range(request_count)]
        )

loop = asyncio.get_event_loop()
start = time.time()
loop.run_until_complete(main())
end = time.time()
print("Time elapsed: ",end-start)

#Output: Time elapsed 1.37 sec

The difference is quite evident, the async version took just 1.37 seconds while the synchronous version took 4.87 seconds. Notice that we sent just 10 requests, If we send 1000s of requests, the difference would be more evident. However, I suggest not to overload someone's server by sending so many requests. They may also rate-limit or block your IP.

Take it with a pinch of salt: Try to use the asynchronous code, async drivers, and in general asynchronous programming to better utilize the resources.

Oops, Let me check if the pizza🍕 is ready!