Async Python for Experienced Developers: CPU-Bound vs IO-Bound Work
- Author
- Vishal Maurya
- Published on
- Reading time
- 4 min read
Overview
You know concurrency from other languages, but Python has its own wrinkles: the GIL, async/await that does not run things in parallel, and three different tools that all say "concurrent". Picking the wrong one gives you code that is more complex and no faster.
This guide gives a decision framework based on what your workload spends its time doing.
1. Classify the Work First
- IO-bound: most time is spent waiting on networks, disks, databases, or other services. CPU is mostly idle.
- CPU-bound: most time is spent computing in Python code: parsing, transforming, hashing, numeric loops, image processing.
Profile before deciding. A request that "feels slow" may be waiting on a database call, not computing. Tools such as cProfile, py-spy, or simple timing around calls tell you quickly.
2. GIL Basics in One Paragraph
In the standard CPython build, the Global Interpreter Lock lets only one thread execute Python bytecode at a time. Threads still help when they wait on I/O, because the lock is released during blocking operations, and many C extensions release it during heavy work. But pure-Python CPU work in several threads does not run in parallel. Free-threaded builds, which remove the GIL, exist in recent Python versions, but support is still evolving and third-party extension compatibility varies, so check the current documentation and your dependencies before relying on them.
3. asyncio: Many IO Waits on One Thread
asyncio multiplexes many tasks on a single thread, switching at await points. It is a good fit for large numbers of concurrent network operations, such as an API gateway, web scraper, or service calling many backends. It needs async-aware libraries, and it will not speed up computation.
import asyncio
import httpx
async def fetch_all(urls: list[str]) -> list[int]:
sem = asyncio.Semaphore(20)
async with httpx.AsyncClient(timeout=10) as client:
async def one(url: str) -> int:
async with sem:
resp = await client.get(url)
return resp.status_code
return await asyncio.gather(*(one(u) for u in urls))
Note the semaphore. Unbounded concurrency overloads the remote service and your own memory.
4. Threads: IO With Blocking Libraries
If your libraries are synchronous, a ThreadPoolExecutor gives you IO concurrency without rewriting everything. It suits moderate concurrency, such as dozens of parallel calls, and also works as a bridge from async code via asyncio.to_thread.
Threads share memory, so protect shared mutable state with locks or queues. They are a poor choice for thousands of concurrent connections because each thread has overhead.
5. Multiprocessing: CPU-Bound Work
For heavy Python computation, use processes so each has its own interpreter and GIL.
from concurrent.futures import ProcessPoolExecutor
def score(chunk: list[int]) -> int:
return sum(x * x for x in chunk)
def main(data: list[list[int]]) -> int:
with ProcessPoolExecutor() as pool:
return sum(pool.map(score, data))
if __name__ == '__main__':
main([[1, 2, 3], [4, 5, 6]])
Processes cost more: startup time, memory per process, and pickling of arguments and results. Send chunks big enough to amortize that overhead. The __main__ guard matters on platforms that spawn processes.
Also consider whether a vectorized library, such as NumPy, or a compiled extension can do the work faster in a single process.
6. Mixing Them in One Service
A typical web service is mostly IO-bound, with an occasional heavy computation. Keep the request path async, and push the heavy part elsewhere:
- Short blocking calls:
asyncio.to_thread. - Heavy CPU: a process pool, or a task queue with separate workers.
Running CPU work directly inside an async def blocks the loop and delays every other request.
7. Quick Decision Table
- Thousands of concurrent network calls, async libraries available: asyncio.
- Dozens of concurrent calls, sync libraries: threads.
- Parallel Python computation: processes.
- Mixed workload: async front, offloaded blocking or CPU work.
- Single heavy numeric task: try vectorization first.
Conclusion
Match the tool to where the time goes. Asyncio and threads overlap waiting, processes add real CPU parallelism, and none of them fixes an algorithm that does too much work. Measure first, then pick the simplest model that meets your need.
If you need help designing the concurrency model of a Python service or tracking down its bottleneck, I can help. Contact me.