Skip to Content
Course content

213: Async I/O with aiohttp

Click on the "Edit" button in the top corner of the screen to edit your slide content.

Imagine you're at a busy food court. If you were a "synchronous" person, you'd walk up to the burger stall, order a burger, and stand there staring at the cook until it's finished. Only once the burger is in your hand would you walk over to the drink stall and order a soda, again waiting in silence until it's poured. You've spent ten minutes waiting, but most of that time you weren't actually doing anything; you were just waiting for someone else to finish a task.

Async I/O is like using the buzzer system. You order your burger, they give you a buzzer, and you immediately head over to order your soda. While the kitchen is working on both, you can check your emails or find a table. When a buzzer goes off, you go grab the food. You've parallelized the waiting, not the cooking.

In Python, aiohttp is that buzzer system for the web. When you make an HTTP request, your CPU isn't doing much—it's just waiting for a server thousands of miles away to send packets back. aiohttp lets you fire off ten requests and then "sleep" until the responses start trickling back in, rather than freezing your entire program for every single request.

Stopping the "Wait-and-See" Cycle

You've likely used the requests library before. It's great, but it's blocking. If you need to fetch data from 50 different URLs, requests will do them one by one. If each takes one second, you're stuck for 50 seconds. With aiohttp, those 50 requests happen almost concurrently, and the total time is usually just slightly longer than the single slowest request.

Here is how I usually structure a basic concurrent fetch. Notice that we don't just call a function; we use a ClientSession. I see a lot of beginners creating a new session for every single request—don't do that. It's expensive. Create one session and reuse it for everything.

import asyncio
import aiohttp
import time

async def fetch_user_data(session, user_id):
    url = f"https://jsonplaceholder.typicode.com/users/{user_id}"
    # The 'async with' ensures the connection is released back to the pool
    async with session.get(url) as response:
        # .json() is also an awaitable operation because it's reading the response body
        return await response.json()

async def main():
    # Create one session to be used for all requests
    async with aiohttp.ClientSession() as session:
        tasks = []
        for user_id in range(1, 11): # Fetching 10 users
            tasks.append(fetch_user_data(session, user_id))
        
        # asyncio.gather is the magic part—it kicks off all tasks and waits for all to finish
        results = await asyncio.gather(*tasks)
        
        for user in results:
            print(f"Fetched: {user['name']}")

# Standard way to run the top-level entry point
asyncio.run(main())

The Session Trap and Context Managers

You'll notice the async with blocks. I can't stress this enough: network connections are finite resources. If you open thousands of connections without closing them, you'll hit the OS limit and your program will crash with a "Too many open files" error. aiohttp uses asynchronous context managers to handle this cleanup for you automatically.

Another thing to keep in mind: aiohttp doesn't magically make the remote server faster. If you hit a small API with 1,000 concurrent requests in one second, you'll likely get a 429 Too Many Requests error (or a temporary IP ban). In a real production environment, I'd suggest using a asyncio.Semaphore to limit the number of concurrent requests to something reasonable, like 10 or 20.

Handling the "Slowest Link"

When you use asyncio.gather, your code waits for the last request to finish before moving forward. Sometimes, one server is lagging, and you don't want your whole app to hang because of one slow API. In those cases, I prefer asyncio.as_completed. It yields results as soon as they come in, allowing you to process the "fast" data while the "slow" data is still in flight.




📋 Practical Task

Build a Concurrent Cryptocurrency Price Tracker

Your goal is to build a script that fetches the current price of multiple cryptocurrencies simultaneously using the CoinGecko API. Instead of waiting for each price one by one, you will implement a concurrent fetcher.

Requirements:

  • Create a list of at least 5 coin IDs (e.g., 'bitcoin', 'ethereum', 'cardano', 'solana', 'polkadot').
  • Use a single aiohttp.ClientSession to make requests to the following endpoint for each coin: https://api.coingecko.com/api/v3/simple/price?ids={coin_id}&vs_currencies=usd.
  • Use asyncio.gather to trigger all requests concurrently.
  • Print the results in the format: "Coin: [Name] | Price: $[Price]".
  • Ensure you use async with for both the session and the response to prevent memory leaks.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.