Skip to content

bpo-31861: Provide aiter and anext builtins #23847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Mar 23, 2021
Merged

bpo-31861: Provide aiter and anext builtins #23847

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
44b2a60
bpo-31861: Add operator.aiter and operator.anext
jab Aug 24, 2018
142523b
address comments from first review
jab Sep 7, 2018
3ebce05
Address new review comments + rebase
jab Nov 30, 2020
2f8df99
improve tests
jab Dec 4, 2020
ad12116
Sketch out async iterator in C
lordmauve Dec 4, 2020
ce35092
Implement aiter() and anext() using sync methods only
lordmauve Dec 7, 2020
f9dc183
Add basic aiter built-in [WIP]
justin39 Dec 7, 2020
086bb79
Start aiter implementation
justin39 Dec 9, 2020
29ef712
Get test_asyncgen tests passing
justin39 Dec 10, 2020
331e80e
Fix awaitable iternext
justin39 Dec 10, 2020
95879d8
Add anext builtin
justin39 Dec 11, 2020
5b64589
Use stop iter functions for anext
justin39 Dec 14, 2020
6e145db
Use stop iter functions in aiter
justin39 Dec 14, 2020
2fd4be6
Note about implementing __reduce__ for aiter
justin39 Dec 16, 2020
430dd59
Refactor aiter and anext type names
justin39 Dec 16, 2020
4266e65
Add documentation for aiter()
justin39 Dec 16, 2020
ecbedc7
Update documentation for aiter and anext
justin39 Dec 18, 2020
5fa3812
Clean up docs and formatting
justin39 Dec 18, 2020
0f9c814
Remove async iterator code from operator.py
justin39 Dec 18, 2020
8160c82
Cleanup formatting
justin39 Dec 18, 2020
fa8b12a
Fix test_builtins_have_signatures + misc. cleanups
jab Dec 18, 2020
a2242d9
Use PyErr_SetNone now that we can
jab Dec 18, 2020
3ce5675
whitespace
jab Dec 18, 2020
0038cde
cosmetic fixes
jab Dec 19, 2020
06019ea
Add null check and use StopAsyncIteration when aiter is exhausted
justin39 Dec 19, 2020
0cc00f2
Fix AC definition and resulting signature
justin39 Dec 21, 2020
8b8a689
Fix comparison to NULL instead of Py_None
justin39 Dec 21, 2020
894600d
Revert None deafult for aiter/anext and add whatsnew entry
justin39 Dec 21, 2020
e687d88
Merge branch 'master' of https://github.com/python/cpython into justi…
justin39 Feb 26, 2021
259af97
Delint tests + docs (fixes CI), alphabetize names.
jab Mar 20, 2021
97d06e5
Merge branch 'master' into justin39/aiter-c
jab Mar 20, 2021
dd7c02d
Fix code style and comments.
jab Mar 20, 2021
2fbdd5b
Remove 2-arg variant of aiter.
jab Mar 22, 2021
009118c
Fix use of clinic.
jab Mar 22, 2021
71fede3
Address some feedback from Guido's review.
jab Mar 22, 2021
4a51ace
No longer need to exclude aiter from test_builtins_have_signatures.
jab Mar 22, 2021
108e4a3
Improve aiter() docs.
jab Mar 22, 2021
042f596
Fix typo.
jab Mar 22, 2021
6ee8824
Add test_aiter_idempotent().
jab Mar 23, 2021
6f50ef8
Remove public API added for anext.
jab Mar 23, 2021
ef40fb7
Slightly improve wording
gvanrossum Mar 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions Doc/glossary.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,6 @@ Glossary
:ref:`the difference between arguments and parameters
<faq-argument-vs-parameter>`, and :pep:`362`.

asynchronous callable
Any callable that returns an :term:`awaitable`. Examples include
:term:`coroutine functions <coroutine function>` and the built-in
:func:`anext` function.

asynchronous context manager
An object which controls the environment seen in an
:keyword:`async with` statement by defining :meth:`__aenter__` and
Expand Down
14 changes: 6 additions & 8 deletions Doc/library/functions.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,15 @@ are always available. They are listed here in alphabetical order.

.. function:: aiter(async_iterable)

Return an :term:`asynchronous iterator`. This is the async variant
of the :func:`iter` builtin, and behaves similarly.
Equivalent to calling ``x.__aiter__()``.

*async_iterable* must be an :term:`asynchronous iterable`,
and :func:`aiter` returns an asynchronous iterator for it.
``aiter(aiter(x))`` is the same as ``aiter(x)``.
(``aiter(x)`` itself has an ``__aiter__()`` method that returns ``self``.)

Unlike the :func:`iter` builtin, :func:`aiter` has no 2-argument variant.
Often, this variant can be replaced with assignment expressions::
Formally, given an :term:`asynchronous iterable`,
return an :term:`asynchronous iterator`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better! Here is how I would rearrange this:

Return an :term:`asynchronous iterator`
for an :term:`asynchronous iterable`. 
Equivalent to calling ``x.__aiter__()``.

``aiter(x)`` itself has an ``__aiter__()`` method that returns ``x``,
so ``aiter(aiter(x))`` is the same as ``aiter(x)``.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated in the latest revision, thanks!

(((
Come to think of it, iter also has the iter(iter(x)) == iter(x) property, which is not currently mentioned in the iter docs. The iter docs have a lot more work to do though, to cover the 1- and 2-arg variants (and they're already doing this very well).

And now that I'm looking at those again, I notice the only code example there is for the 2-arg variant:

from functools import partial
with open('mydata.db', 'rb') as f:
    for block in iter(partial(f.read, 64), b''):
        process_block(block)

If this pattern is now obsoleted by assignment expressions[1], is it worth updating the iter docs to (1) mention the iter(iter(x)) == iter(x) property, and (2) remove the obsoleted example code? If so, happy to submit a separate PR for that.

[1] as in the following:

with open('mydata.db', 'rb') as f:
    while block := f.read(64):
        process_block(block)

)))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to personally get into the weeds about the iter() docs, sorry. Something for the docs WG perhaps.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's worth updating the docs to promise iter(iter(x)) == iter(x) as it isn't a specific benefit to the user beyond logically doing the right thing.

Copy link
Contributor Author

@jab jab Mar 23, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that.

@gvanrossum, do you think @brettcannon's rationale applies equally to the aiter docs too? I did add a test_aiter_idempotent() for this in 6ee8824 to go along with the promise that the aiter docs are now making, but can remove that test along with that part of the docs if that's better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a different bar for updating the iter() docs than for the initial version of the aiter() docs. And they don't have to match precisely.

FWIW the main reason the idempotency property is important is because of the implicit [a]iter() call in a for-loop, since in

for i in x: ...

the for-loop calls iter(x), so that in

for i in iter(x): ...

the for-loop ends up calling iter(iter(x)). It's the same for async for and aiter().


while chunk := await sock.read(CHUNK_SIZE):
...
Note: Unlike :func:`iter`, :func:`aiter` has no 2-argument variant.


.. function:: all(iterable)
Expand Down
2 changes: 1 addition & 1 deletion Lib/test/test_inspect.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3861,7 +3861,7 @@ def test_builtins_have_signatures(self):
"next", "iter", "vars"}
no_signature |= needs_groups
# These have unrepresentable parameter default values of NULL
needs_null = {"aiter", "anext"}
needs_null = {"anext"}
no_signature |= needs_null
# These need PEP 457 groups or a signature change to accept None
needs_semantic_update = {"round"}
Expand Down
8 changes: 4 additions & 4 deletions Objects/abstract.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2770,10 +2770,10 @@ int
PyAiter_Check(PyObject *obj)
{
PyTypeObject *tp = Py_TYPE(obj);
return (tp->tp_as_async != NULL && \
tp->tp_as_async->am_aiter != NULL && \
tp->tp_as_async->am_aiter != &_PyObject_NextNotImplemented && \
tp->tp_as_async->am_anext != NULL && \
return (tp->tp_as_async != NULL &&
tp->tp_as_async->am_aiter != NULL &&
tp->tp_as_async->am_aiter != &_PyObject_NextNotImplemented &&
tp->tp_as_async->am_anext != NULL &&
tp->tp_as_async->am_anext != &_PyObject_NextNotImplemented);
}

Expand Down