Tests fail with pytest 8.4: empty traceback when the plugin raises an exception directly

[cjwatson]

https://bugs.debian.org/1114324 reports that pytest-trio's tests fail with pytest 8.4.2. The failure looks like this:

pytest failure report

_____________________________________________________________________________ test_skip_and_xfail ______________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-cjwatson/pytest-11/test_skip_and_xfail0')>

    def test_skip_and_xfail(testdir):
        testdir.makepyfile(
            """
            import functools
            import pytest
            import trio

            trio.run = functools.partial(trio.run, strict_exception_groups=True)

            @pytest.mark.trio
            async def test_xfail():
                pytest.xfail()

            @pytest.mark.trio
            async def test_skip():
                pytest.skip()

            async def callback(fn):
                fn()

            async def fail():
                raise RuntimeError

            @pytest.mark.trio
            async def test_xfail_and_fail():
                async with trio.open_nursery() as nursery:
                    nursery.start_soon(callback, pytest.xfail)
                    nursery.start_soon(fail)

            @pytest.mark.trio
            async def test_skip_and_fail():
                async with trio.open_nursery() as nursery:
                    nursery.start_soon(callback, pytest.skip)
                    nursery.start_soon(fail)

            @pytest.mark.trio
            async def test_xfail_and_skip():
                async with trio.open_nursery() as nursery:
                    nursery.start_soon(callback, pytest.skip)
                    nursery.start_soon(callback, pytest.xfail)
        """
        )

        result = testdir.runpytest("-s")

>       result.assert_outcomes(skipped=1, xfailed=1, failed=3)
E       AssertionError: assert {'passed': 0, 'skipped': 0, 'failed': 0, 'errors': 0, 'xpassed': 0, 'xfailed': 0} == {'passed': 0, 'skipped': 1, 'failed': 3, 'errors': 0, 'xpassed': 0, 'xfailed': 1}
E
E         Common items:
E         {'errors': 0, 'passed': 0, 'xpassed': 0}
E         Differing items:
E         {'failed': 0} != {'failed': 3}
E         {'skipped': 0} != {'skipped': 1}
E         {'xfailed': 0} != {'xfailed': 1}
E
E         Full diff:
E           {
E               'errors': 0,
E         -     'failed': 3,
E         ?               ^
E         +     'failed': 0,
E         ?               ^
E               'passed': 0,
E         -     'skipped': 1,
E         ?                ^
E         +     'skipped': 0,
E         ?                ^
E         -     'xfailed': 1,
E         ?                ^
E         +     'xfailed': 0,
E         ?                ^
E               'xpassed': 0,
E           }

/home/cjwatson/src/python/pytest-trio/pytest_trio/_tests/test_basic.py:121: AssertionError
----------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.13.7, pytest-8.4.2, pluggy-1.5.0
rootdir: /tmp/pytest-of-cjwatson/pytest-11/test_skip_and_xfail0
plugins: trio-0.8.0+dev, hypothesis-6.108.0, cov-5.0.0
collected 5 items

test_skip_and_xfail.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>     ~~~~~~~~~~~~~~~~^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_hypothesis_pytestplugin.py", line 326, in pytest_runtest_makereport
INTERNALERROR>     report = (yield).get_result()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/skipping.py", line 275, in pytest_runtest_makereport
INTERNALERROR>     rep = yield
INTERNALERROR>           ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 368, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/reports.py", line 377, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/python.py", line 1713, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/nodes.py", line 456, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>            ~~~~~~~~~~~~~~~^
INTERNALERROR>         funcargs=True,
INTERNALERROR>         ^^^^^^^^^^^^^^
INTERNALERROR>     ...<5 lines>...
INTERNALERROR>         truncate_args=truncate_args,
INTERNALERROR>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>     )
INTERNALERROR>     ^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 766, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>            ~~~~~~~~~~~~~~~~^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 1198, in repr_excinfo
INTERNALERROR>     traceback[0]._rawentry,
INTERNALERROR>     ~~~~~~~~~^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 422, in __getitem__
INTERNALERROR>     return super().__getitem__(key)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~^^^^^
INTERNALERROR> IndexError: list index out of range
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 289, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ~~~~^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 343, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/logging.py", line 801, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/terminal.py", line 688, in pytest_runtestloop
INTERNALERROR>     result = yield
INTERNALERROR>              ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 367, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/warnings.py", line 90, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/assertion/__init__.py", line 192, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/unittest.py", line 475, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/faulthandler.py", line 88, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 117, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>     ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 136, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>                    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 248, in call_and_report
INTERNALERROR>     report: TestReport = ihook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 160, in _multicall
INTERNALERROR>     _warn_teardown_exception(hook_name, teardown[1], e)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 50, in _warn_teardown_exception
INTERNALERROR>     warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5)
INTERNALERROR>     ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
INTERNALERROR> Plugin: hypothesispytest, Hook: pytest_runtest_makereport
INTERNALERROR> IndexError: list index out of range
INTERNALERROR> For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning

============================== 1 warning in 0.01s ==============================
____________________________________________________________________________ test_error_collection _____________________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-cjwatson/pytest-11/test_error_collection0')>

    def test_error_collection(testdir):
        # We want to make sure that pytest ultimately reports all the different
        # exceptions. We call .upper() on all the exceptions so that we have
        # tokens to look for in the output corresponding to each exception, where
        # those tokens don't appear at all the source (so we can't get a false
        # positive due to pytest printing out the source file).

        # We sleep at the beginning of all the fixtures b/c currently if any
        # fixture crashes, we skip setting up unrelated fixtures whose setup
        # hasn't even started yet. Maybe we shouldn't? But for now the sleeps make
        # sure that all the fixtures have started before any of them start
        # crashing.
        testdir.makepyfile(
            """
            import pytest
            from pytest_trio import trio_fixture
            import trio

            test_started = False

            @trio_fixture
            async def crash_nongen():
                with trio.CancelScope(shield=True):
                    await trio.sleep(2)
                raise RuntimeError("crash_nongen".upper())

            @trio_fixture
            async def crash_early_agen():
                with trio.CancelScope(shield=True):
                    await trio.sleep(2)
                raise RuntimeError("crash_early_agen".upper())
                yield

            @trio_fixture
            async def crash_late_agen():
                yield
                raise RuntimeError("crash_late_agen".upper())

            async def crash(when, token):
                with trio.CancelScope(shield=True):
                    await trio.sleep(when)
                    raise RuntimeError(token.upper())

            @trio_fixture
            def crash_background(nursery):
                nursery.start_soon(crash, 1, "crash_background_early")
                nursery.start_soon(crash, 3, "crash_background_late")

            @pytest.mark.trio
            async def test_all_the_crashes(
                autojump_clock,
                crash_nongen, crash_early_agen, crash_late_agen, crash_background,
            ):
                global test_started
                test_started = True

            def test_followup():
                assert not test_started

            """
        )

        result = testdir.runpytest()
>       result.assert_outcomes(passed=1, failed=1)
E       AssertionError: assert {'passed': 0, 'skipped': 0, 'failed': 0, 'errors': 0, 'xpassed': 0, 'xfailed': 0} == {'passed': 1, 'skipped': 0, 'failed': 1, 'errors': 0, 'xpassed': 0, 'xfailed': 0}
E
E         Common items:
E         {'errors': 0, 'skipped': 0, 'xfailed': 0, 'xpassed': 0}
E         Differing items:
E         {'failed': 0} != {'failed': 1}
E         {'passed': 0} != {'passed': 1}
E
E         Full diff:
E           {
E               'errors': 0,
E         -     'failed': 1,
E         ?               ^
E         +     'failed': 0,
E         ?               ^
E         -     'passed': 1,
E         ?               ^
E         +     'passed': 0,
E         ?               ^
E               'skipped': 0,
E               'xfailed': 0,
E               'xpassed': 0,
E           }

/home/cjwatson/src/python/pytest-trio/pytest_trio/_tests/test_fixture_ordering.py:193: AssertionError
----------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.13.7, pytest-8.4.2, pluggy-1.5.0
rootdir: /tmp/pytest-of-cjwatson/pytest-11/test_error_collection0
plugins: trio-0.8.0+dev, hypothesis-6.108.0, cov-5.0.0
collected 2 items

test_error_collection.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>     ~~~~~~~~~~~~~~~~^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_hypothesis_pytestplugin.py", line 326, in pytest_runtest_makereport
INTERNALERROR>     report = (yield).get_result()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR>     teardown.throw(outcome._exception)
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/skipping.py", line 275, in pytest_runtest_makereport
INTERNALERROR>     rep = yield
INTERNALERROR>           ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 368, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/reports.py", line 377, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/python.py", line 1713, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/nodes.py", line 456, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>            ~~~~~~~~~~~~~~~^
INTERNALERROR>         funcargs=True,
INTERNALERROR>         ^^^^^^^^^^^^^^
INTERNALERROR>     ...<5 lines>...
INTERNALERROR>         truncate_args=truncate_args,
INTERNALERROR>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>     )
INTERNALERROR>     ^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 766, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>            ~~~~~~~~~~~~~~~~^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 1198, in repr_excinfo
INTERNALERROR>     traceback[0]._rawentry,
INTERNALERROR>     ~~~~~~~~~^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/_code/code.py", line 422, in __getitem__
INTERNALERROR>     return super().__getitem__(key)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~^^^^^
INTERNALERROR> IndexError: list index out of range
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 289, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ~~~~^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 343, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/logging.py", line 801, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/terminal.py", line 688, in pytest_runtestloop
INTERNALERROR>     result = yield
INTERNALERROR>              ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/main.py", line 367, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/warnings.py", line 90, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/assertion/__init__.py", line 192, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/unittest.py", line 475, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 122, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ~~~~~~~~~~~~~~^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/faulthandler.py", line 88, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 117, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>     ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 136, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>                    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/_pytest/runner.py", line 248, in call_and_report
INTERNALERROR>     report: TestReport = ihook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 480, in traced_hookexec
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ~~~~~~~~~~~~~~~~~~^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_result.py", line 62, in from_call
INTERNALERROR>     result = func()
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 477, in <lambda>
INTERNALERROR>     lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
INTERNALERROR>             ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 160, in _multicall
INTERNALERROR>     _warn_teardown_exception(hook_name, teardown[1], e)
INTERNALERROR>     ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/home/cjwatson/src/python/pytest-trio/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 50, in _warn_teardown_exception
INTERNALERROR>     warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5)
INTERNALERROR>     ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
INTERNALERROR> Plugin: hypothesispytest, Hook: pytest_runtest_makereport
INTERNALERROR> IndexError: list index out of range
INTERNALERROR> For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning

I don't know this code well, but I did some digging. Looking at test_basic.py::test_skip_and_xfail, what seems to happen here is that pytest filters the traceback, finds that all the frames past the cut have __tracebackhide__ = True, and then gets an IndexError when trying to find the first such frame (traceback[0].rawentry buried in the above). The two frames past the cut are:

[<TracebackEntry /usr/lib/python3/dist-packages/pytest_trio/plugin.py:360>, <TracebackEntry /usr/lib/python3/dist-packages/_pytest/outcomes.py:205>]

There's something similar going on in test_fixture_ordering.py::test_error_collection.

I wondered whether it would be good enough to set __tracebackhide__ = False if we're about to raise an exception directly from the plugin, and indeed that seems to make the tests pass. I'll file a PR with that so you can take a look.

[jakkdl]

I think you should open an issue in pytest about this, since pytest-trio may not be the only plugin that causes this. And end-users could cause it directly themselves if they're setting __tracebackhide__ = True

At the very least pytest should fail gracefully by adding

assert traceback, "all sub-exceptions have __tracebackhide__ = True"

but the better way is probably to just ignore the tracebackhide filtering

if not traceback:
    traceback = excinfo_.traceback"

inside repr_excinfo.

It could also be a deeper issue, since we're not explicitly setting it to True ourselves.

[cjwatson]

I think you should open an issue in pytest about this, since pytest-trio may not be the only plugin that causes this.

OK, I filed pytest-dev/pytest#13734.

And end-users could cause it directly themselves if they're setting __tracebackhide__ = True

Maybe, but if so I haven't been able to reproduce that. Perhaps there's another frame in the way?

It could also be a deeper issue, since we're not explicitly setting it to True ourselves.

I don't think that's right. In all the cases where my PR added __tracebackhide__ = False, there was already a __tracebackhide__ = True earlier in the same frame.