FastAPI async and sync I/O operations.

  

FastAPI can effectively handle both async and sync I/O activities behind the hood.

  • FastAPI uses threadpool sync routes, and blocking I/O activities do not prevent the event loop from processing tasks.

  • Otherwise, if the route is defined async, it is called on a regular basis using await, and FastAPI expects you to perform only non-blocking I/O activities.

  • The downside is that if you break that trust and perform blocking operations within async routes, the event loop will be unable to process the next jobs until that blocking operation is completed. 

 

import asyncio
import time

@router.get("/terrible-ping")
async def terrible_catastrophic_ping():
    time.sleep(10) # I/O blocking operation for 10 seconds
    pong = service.get_pong()  # I/O blocking operation to get pong from DB
    
    return {"pong": pong}

@router.get("/good-ping")
def good_ping():
    time.sleep(10) # I/O blocking operation for 10 seconds, but in another thread
    pong = service.get_pong()  # I/O blocking operation to get pong from DB, but in another thread
    
    return {"pong": pong}

@router.get("/perfect-ping")
async def perfect_ping():
    await asyncio.sleep(10) # non-blocking I/O operation
    pong = await service.async_get_pong()  # non-blocking I/O db call

    return {"pong": pong}

 

What happens when we call:

  1. GET /terrible-ping
    1. FastAPI server receives a request and starts handling it
    2. Server's event loop and all the tasks in the queue will be waiting until time.sleep() is finished.
      1. Server thinks time.sleep() is not an I/O task, so it waits until it is finished
      2. Server won't accept any new requests while waiting.
    3. Then, event loop and all the tasks in the queue will be waiting until service.get_pong is finished
      1. Server thinks service.get_pong() is not an I/O task, so it waits until it is finished
      2. Server won't accept any new requests while waiting
    4. Server returns the response.
      1. After a response, server starts accepting new requests.
  2. GET /good-ping
    1. FastAPI server receives a request and starts handling it
    2. FastAPI sends the whole route good_ping to the threadpool, where a worker thread will run the function
    3. While good_ping is being executed, event loop selects next tasks from the queue and works on them (e.g. accept new request, call db)
      1. Independently of main thread (i.e. our FastAPI app), worker thread will be waiting for time.sleep to finish and then for service.get_pong to finish
      2. Sync operation blocks only the side thread, not the main one.
    4. When good_ping finishes its work, server returns a response to the client
  3. GET /perfect-ping
    1. FastAPI server receives a request and starts handling it
    2. FastAPI awaits asyncio.sleep(10)
    3. Event loop selects next tasks from the queue and works on them (e.g. accept new request, call db)
    4. When asyncio.sleep(10) is done, servers goes to the next lines and awaits service.async_get_pong
    5. Event loop selects next tasks from the queue and works on them (e.g. accept new request, call db)
    6. When service.async_get_pong is done, server returns a response to the client


The second limitation is that actions that are non-blocking awaitables or are dispatched to the thread pool must be I/O intensive (e.g., open file, database call, external API request). 

  • Awaiting CPU-intensive tasks (e.g. heavy calculations, data processing, video transcoding) is worthless since the CPU has to work to finish the tasks, while I/O operations are external and server does nothing while waiting for that operations to finish, thus it can go to the next tasks.
  • Running CPU-intensive tasks in other threads also isn't effective, because of GIL. In short, GIL allows only one thread to work at a time, which makes it useless for CPU tasks.
  • If you want to optimize CPU intensive tasks you should send them to workers in another process.


Related StackOverflow questions : 

Comments

Post a Comment