Blocking the Event Loop: Finding Latency Spikes in Async Python Pipelines

Overview

An async service handles most requests quickly, but every so often p99 latency jumps, health checks time out, or a websocket misses a heartbeat. CPU is not pegged, and nothing is obviously slow. The usual culprit is a synchronous call that holds the event loop thread and prevents every other coroutine from running.

1. Why One Blocking Call Hurts Everyone

The event loop runs coroutines cooperatively on a single thread. A coroutine yields control only at an await. If it calls time.sleep, a synchronous HTTP client, a large JSON parse, a regex on a big input, or a blocking database driver, nothing else runs until that call returns.

Latency for every other task grows by the blocked duration, which makes the spikes look random and unrelated to the slow code.

import asyncio
import time


async def bad() -> None:
    time.sleep(2)  # blocks the whole loop


async def good() -> None:
    await asyncio.sleep(2)  # yields to the loop

2. Common Offenders

• requests, synchronous database drivers, and file or network I/O without an async API.
• time.sleep, and locks from threading held for long.
• CPU-heavy work: compression, hashing large data, image or PDF processing, big json.loads, pandas operations.
• Synchronous SDK clients used inside async def handlers.
• Logging handlers that write to slow sinks synchronously.

3. Find It With Asyncio Debug Mode

Debug mode logs callbacks and tasks that take too long. Enable it with PYTHONASYNCIODEBUG=1, python -X dev, or asyncio.run(main(), debug=True).

import asyncio
import logging

logging.basicConfig(level=logging.WARNING)


async def main() -> None:
    loop = asyncio.get_running_loop()
    loop.slow_callback_duration = 0.05  # warn above 50 ms
    await run_pipeline()


asyncio.run(main(), debug=True)

The log will say something like "Executing task took 0.8 seconds" along with the task, which points you at the code. Debug mode adds overhead, so use it in development or a staging replica, not as a permanent production setting.

You can also measure loop lag in production by scheduling a periodic task that sleeps for a fixed interval and records how late it wakes. Rising lag is a direct signal that something is blocking.

4. Move Blocking Work Off the Loop

For blocking I/O, call it in a worker thread. asyncio.to_thread (Python 3.9 and later) is the simplest option:

import asyncio
import requests


def fetch_sync(url: str) -> bytes:
    return requests.get(url, timeout=10).content


async def fetch(url: str) -> bytes:
    return await asyncio.to_thread(fetch_sync, url)

For more control over pool size, use loop.run_in_executor with your own executor. The default thread pool has a limited number of workers, so a flood of blocking calls can queue up. Size and monitor it.

If you can, prefer a native async library, such as httpx.AsyncClient or an async database driver, over wrapping a sync one.

5. CPU-Bound Work Needs a Process, Not a Thread

Threads do not escape the GIL for pure-Python CPU work, and a long computation still competes with the loop thread for it. For heavy CPU tasks, use a ProcessPoolExecutor through run_in_executor, or a separate worker service with a queue. Remember that arguments and results must be picklable, and that process startup and transfer have a cost. Check current free-threaded Python support before relying on it as an alternative.

6. Why asyncio.sleep(0) Masks the Problem

Inserting await asyncio.sleep(0) inside a long loop makes it yield control once per iteration. It can reduce the worst-case pause, and it is a legitimate way to split a long cooperative computation. But it does not fix a single blocking call, because the blocking call itself never reaches the await. It also adds scheduling overhead and makes latency depend on how many other tasks are ready.

Treat it as a stopgap for chunking pure-Python loops, not as a replacement for moving blocking work to a thread or process. Prefer processing in bounded batches, offloading the batch, and awaiting the result.

7. Verify the Fix

After the change, compare loop-lag and p99 latency under realistic load. Confirm that executor queues do not grow, and add a test or log assertion so a new synchronous call does not creep back in. Linters such as the ASYNC rules in Ruff can flag some blocking calls inside async functions.

Conclusion

Latency spikes in an async service usually come from synchronous work on the loop thread. Detect it with debug mode and loop-lag metrics, move blocking I/O to threads, send CPU work to processes, and use sleep(0) only as a minor yielding tool.

If your async pipeline has unexplained tail latency, I can help profile it and restructure the blocking paths. Contact me.

Additional Resources

• Python docs: Developing with asyncio
• Python docs: asyncio.to_thread and executors
• Python docs: concurrent.futures