Python asyncio Module

The asyncio module in Python provides a framework for writing single-threaded concurrent code using coroutines, enabling asynchronous programming. It is particularly useful for I/O-bound and high-level structured network code.

Table of Contents

1. Introduction
2. Key Concepts
   • Coroutines
   • Tasks
   • Event Loop
   • Futures
   • Awaitables
3. Key Classes and Functions
   • asyncio.run
   • asyncio.create_task
   • asyncio.gather
   • asyncio.sleep
   • asyncio.Future
   • asyncio.Queue
   • asyncio.Lock
   • asyncio.Event
   • asyncio.Semaphore
4. Examples
   • Basic Coroutine Example
   • Running Multiple Coroutines
   • Using asyncio.Future
   • Using asyncio.Queue
   • Synchronization Primitives
5. Real-World Use Case
6. Conclusion
7. References

Introduction

The asyncio module provides a framework to write asynchronous programs in Python. It uses coroutines, which allow you to write code that performs non-blocking operations, making it suitable for I/O-bound and high-level network applications.

Key Concepts

Coroutines

Coroutines are special functions that can pause their execution and allow other functions to run. They are defined with async def and use await to yield control.

Tasks

Tasks are used to schedule the execution of coroutines. They run concurrently within the event loop.

Event Loop

The event loop is the core of every asyncio application. It runs asynchronous tasks and callbacks, performs network I/O operations, and runs subprocesses.

Futures

Futures are objects that represent the result of an asynchronous operation. They are used to handle results or exceptions of a coroutine.

Awaitables

Awaitables are objects that can be used with await expression. Coroutines, Tasks, and Futures are all awaitables.

Key Classes and Functions

asyncio.run

Runs the provided coroutine and returns the result.

import asyncio

async def main():
    print('Hello, World!')

asyncio.run(main())

asyncio.create_task

Schedules the execution of a coroutine as a Task.

import asyncio

async def my_coroutine():
    await asyncio.sleep(1)
    print('Coroutine executed')

async def main():
    task = asyncio.create_task(my_coroutine())
    await task

asyncio.run(main())

asyncio.gather

Runs multiple coroutines concurrently and gathers their results.

import asyncio

async def foo():
    await asyncio.sleep(1)
    return 'foo'

async def bar():
    await asyncio.sleep(1)
    return 'bar'

async def main():
    results = await asyncio.gather(foo(), bar())
    print(results)

asyncio.run(main())

asyncio.sleep

Suspends the coroutine for a specified amount of time.

import asyncio

async def main():
    print('Sleeping for 1 second...')
    await asyncio.sleep(1)
    print('Awake!')

asyncio.run(main())

asyncio.Future

A Future represents the result of an asynchronous operation.

import asyncio

async def set_future_value(fut):
    await asyncio.sleep(1)
    fut.set_result('Future is done!')

async def main():
    fut = asyncio.Future()
    await asyncio.create_task(set_future_value(fut))
    print(await fut)

asyncio.run(main())

asyncio.Queue

A FIFO queue for communication between coroutines.

import asyncio

async def producer(queue):
    for i in range(5):
        await queue.put(i)
        await asyncio.sleep(1)

async def consumer(queue):
    while True:
        item = await queue.get()
        print(f'Consumed {item}')
        queue.task_done()

async def main():
    queue = asyncio.Queue()
    await asyncio.gather(producer(queue), consumer(queue))

asyncio.run(main())

asyncio.Lock

A lock object to synchronize access to shared resources.

import asyncio

lock = asyncio.Lock()

async def worker(n):
    async with lock:
        print(f'Worker {n} acquired lock')
        await asyncio.sleep(1)
        print(f'Worker {n} released lock')

async def main():
    await asyncio.gather(worker(1), worker(2))

asyncio.run(main())

asyncio.Event

An event object to notify coroutines.

import asyncio

event = asyncio.Event()

async def waiter():
    print('Waiting for event...')
    await event.wait()
    print('Event received!')

async def main():
    asyncio.create_task(waiter())
    await asyncio.sleep(2)
    event.set()

asyncio.run(main())

asyncio.Semaphore

A semaphore to control access to a shared resource.

import asyncio

semaphore = asyncio.Semaphore(2)

async def worker(n):
    async with semaphore:
        print(f'Worker {n} acquired semaphore')
        await asyncio.sleep(1)
        print(f'Worker {n} released semaphore')

async def main():
    await asyncio.gather(worker(1), worker(2), worker(3))

asyncio.run(main())

Examples

Basic Coroutine Example

import asyncio

async def say_hello():
    print('Hello')
    await asyncio.sleep(1)
    print('World')

asyncio.run(say_hello())

Output:

Hello
World

Running Multiple Coroutines

import asyncio

async def say(text, delay):
    await asyncio.sleep(delay)
    print(text)

async def main():
    await asyncio.gather(say('Hello', 1), say('World', 2))

asyncio.run(main())

Output:

Hello
World

Using asyncio.Future

import asyncio

async def set_future(future):
    await asyncio.sleep(1)
    future.set_result('Done!')

async def main():
    future = asyncio.Future()
    await asyncio.create_task(set_future(future))
    result = await future
    print(result)

asyncio.run(main())

Output:

Done!

Using asyncio.Queue

import asyncio

async def producer(queue):
    for i in range(5):
        await queue.put(i)
        print(f'Produced {i}')
        await asyncio.sleep(1)

async def consumer(queue):
    while True:
        item = await queue.get()
        print(f'Consumed {item}')
        queue.task_done()

async def main():
    queue = asyncio.Queue()
    await asyncio.gather(producer(queue), consumer(queue))

asyncio.run(main())

Output:

Produced 0
Consumed 0
Produced 1
Consumed 1
Produced 2
Consumed 2
Produced 3
Consumed 3
Produced 4
Consumed 4

Synchronization Primitives

import asyncio

async def worker(lock, n):
    async with lock:
        print(f'Worker {n} acquired lock')
        await asyncio.sleep(1)
        print(f'Worker {n} released lock')

async def main():
    lock = asyncio.Lock()
    await asyncio.gather(worker(lock, 1), worker(lock, 2))

asyncio.run(main())

Output:

Worker 1 acquired lock
Worker 1 released lock
Worker 2 acquired lock
Worker 2 released lock

Real-World Use Case

Asynchronous Web Scraping

import asyncio
import aiohttp

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

async def main():
    urls = [
        'http://example.com',
        'http://example.org',
        'http://example.net'
    ]
    tasks = [asyncio.create_task(fetch(url)) for url in urls]
    results = await asyncio.gather(*tasks)
    for result in results:
        print(result)

asyncio.run(main())

Output:

<!doctype html>...
<!doctype html>...
<!doctype html>...

Conclusion

The asyncio module in Python provides a powerful framework for writing asynchronous code. It is particularly useful for I/O-bound and high-level structured network code, enabling efficient concurrency and simplifying the management of asynchronous operations.

References

• Python asyncio module documentation