An equivalent of generators in asyncio is a coroutine - a function whose execution can be suspended and resumed. As we can see, generators perfectly fit the definition of coroutines, with one exception: a coroutine has the awaitable property - the ability to be used with the await operator.
To run coroutines, we need an Event Loop - a crucial component in asyncio that executes asynchronous tasks. The asyncio package provides a comprehensive set of low-level methods and functions for event loops. However, it's important to note that for most practical purposes, using the high-level function asyncio.run() is more than sufficient.
https://docs.python.org/3/library/asyncio-eventloop.html#event-loop
Let's consider a simple example:
import asyncio
async def cor1():
print('cor1')
asyncio.run(cor1())All attempts to run the coroutine cor1() as a regular function will result in an error: RuntimeWarning: coroutine 'cor1' was never awaited. This indicates that running a coroutine means notifying the event loop that the coroutine is ready to be executed by using the await operator. In other words, by using the await cor1() construction, we are informing the event loop something along the lines of: "We are ready to execute the coroutine cor1() at this point in the program. Please do it at the earliest opportunity".
In our previous example, the asyncio.run() function already implicitly contains this operator, so the code runs without an error. However, we can explicitly run a coroutine from another coroutine:
import asyncio
async def cor1():
print('cor1')
async def main():
await cor1()
print('The cor1 coroutine has been started.')
asyncio.run(main())Let's make the code more complex and try to run two coroutines in parallel. To do this, we will set different delays in each coroutine and observe how well asyncio handles this task.
By the way, note that the delay in our example is not taken from the time package but from the asyncio package. The explanation is as follows: in principle, we can use any function inside a coroutine. However, if we want to avoid blocking within the coroutine, this function must have the awaitable property.
import time
import asyncio
from my_deco import async_time_counter
async def cor1(start):
print(f'cor1 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(1)
print(f'cor1 after with delay: {time.time() - start:0.0f}')
async def cor2(start):
print(f'cor2 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(2)
print(f'cor2 after with delay: {time.time() - start:0.0f}')
@async_time_counter
async def main():
start = time.time()
await cor1(start)
await cor2(start)
print(f'and all with delay: {time.time() - start:0.0f}')
asyncio.run(main())As we can see, the trick didn't work out - the coroutines executed sequentially instead of in parallel:
======== Script started ========
cor1 before with delay: 0
cor1 after with delay: 1
cor2 before with delay: 1
cor2 after with delay: 3
and all with delay: 3
======== Script execution time: 3.00 ========This happened because, for parallel execution of coroutines, it is not enough for all the functions inside the coroutines to be awaitable objects. The coroutines themselves need to be passed to the event loop not as coroutines but as tasks. There are several ways to do this.
Firstly, you can do it explicitly by converting the coroutine into a task using the asyncio.create_task() function:
import time
import asyncio
from my_deco import async_time_counter
async def cor1(start):
print(f'cor1 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(1)
print(f'cor1 after with delay: {time.time() - start:0.0f}')
async def cor2(start):
print(f'cor2 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(2)
print(f'cor2 after with delay: {time.time() - start:0.0f}')
@async_time_counter
async def main():
start = time.time()
task1 = asyncio.create_task(cor1(start))
task2 = asyncio.create_task(cor2(start))
await task1
await task2
print(f'and all with delay: {time.time() - start:0.0f}')
asyncio.run(main())As you can see, congratulations are in order - now the overall script execution time has been reduced to the time taken by the "longest" coroutine:
======== Script started ========
cor1 before with delay: 0
cor2 before with delay: 0
cor1 after with delay: 1
cor2 after with delay: 2
and all with delay: 2
======== Script execution time: 2.00 ========Secondly, you can achieve the same result implicitly using the fantastic function asyncio.gather(), which automatically converts all the coroutines passed as arguments into tasks:
import time
import asyncio
from my_deco import async_time_counter
async def cor1(start):
print(f'cor1 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(5)
print(f'cor1 after with delay: {time.time() - start:0.0f}')
async def cor2(start):
print(f'cor2 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(2)
print(f'cor2 after with delay: {time.time() - start:0.0f}')
@async_time_counter
async def main():
start = time.time()
"""
From tutorial:
awaitable asyncio.gather(*aws, return_exceptions=False)
"If any awaitable in aws is a coroutine, it is automatically scheduled as a Task."
https://docs.python.org/3/library/asyncio-task.html#asyncio.gather
"""
await asyncio.gather(cor1(start), cor2(start))
print(f'and all with delay: {time.time() - start:0.0f}')
asyncio.run(main())And thirdly, in Python version 3.11, there is a more elegant way to convert coroutines into tasks using asyncio.TaskGroup():
import time
import asyncio
from my_deco import async_time_counter
async def cor1(start):
print(f'cor1 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(5)
print(f'cor1 after with delay: {time.time() - start:0.0f}')
async def cor2(start):
print(f'cor2 before with delay: {time.time() - start:0.0f}')
await asyncio.sleep(2)
print(f'cor2 after with delay: {time.time() - start:0.0f}')
@async_time_counter
async def main():
start = time.time()
async with asyncio.TaskGroup() as tg:
task1 = tg.create_task(cor1(start))
task2 = tg.create_task(cor2(start))
print(f'and all with delay: {time.time() - start:0.0f}')
asyncio.run(main())
Thus, we have discovered that the third crucial element of the asyncio package is the Task. Tasks enable coroutines to be executed concurrently, or more precisely, quasi-parallel.
https://docs.python.org/3/library/asyncio-task.html#:~:text=a%20coroutine%20function.-
You can learn more about all the details of this topic from this video (Russian Voice):
