Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
470 changes: 55 additions & 415 deletions .github/workflows/test.yml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion ci/templates/.github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- {python-version: "pypy-3.9", tox-python-version: "pypy3"}
- {python-version: "3.11", tox-python-version: "py311"}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
Expand Down
18 changes: 2 additions & 16 deletions docs/plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,5 @@ Getting coverage on pytest plugins is a very particular situation. Because of ho
entrypoints) it doesn't allow controlling the order in which the plugins load.
See `pytest/issues/935 <https://github.com/pytest-dev/pytest/issues/935#issuecomment-245107960>`_ for technical details.

The current way of dealing with this problem is using the append feature and manually starting ``pytest-cov``'s engine, eg::

COV_CORE_SOURCE=src COV_CORE_CONFIG=.coveragerc COV_CORE_DATAFILE=.coverage.eager pytest --cov=src --cov-append

Alternatively you can have this in ``tox.ini`` (if you're using `Tox <https://tox.wiki/en/latest/>`_ of course)::

[testenv]
setenv =
COV_CORE_SOURCE=
COV_CORE_CONFIG={toxinidir}/.coveragerc
COV_CORE_DATAFILE={toxinidir}/.coverage

And in ``pytest.ini`` / ``tox.ini`` / ``setup.cfg``::

[tool:pytest]
addopts = --cov --cov-append
**Currently there is no way to measure your pytest plugin if you use pytest-cov**.
You should change your test invocations to use ``coverage run -m pytest ...`` instead.
186 changes: 9 additions & 177 deletions docs/subprocess-support.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,189 +2,21 @@
Subprocess support
==================

Normally coverage writes the data via a pretty standard atexit handler. However, if the subprocess doesn't exit on its
own then the atexit handler might not run. Why that happens is best left to the adventurous to discover by waddling
through the Python bug tracker.

pytest-cov supports subprocesses, and works around these atexit limitations. However, there are a few pitfalls that need to be explained.

But first, how does pytest-cov's subprocess support works?

pytest-cov packaging injects a pytest-cov.pth into the installation. This file effectively runs this at *every* python startup:

.. code-block:: python

if 'COV_CORE_SOURCE' in os.environ:
try:
from pytest_cov.embed import init
init()
except Exception as exc:
sys.stderr.write(
"pytest-cov: Failed to setup subprocess coverage. "
"Environ: {0!r} "
"Exception: {1!r}\n".format(
dict((k, v) for k, v in os.environ.items() if k.startswith('COV_CORE')),
exc
)
)

The pytest plugin will set this ``COV_CORE_SOURCE`` environment variable thus any subprocess that inherits the environment variables
(the default behavior) will run ``pytest_cov.embed.init`` which in turn sets up coverage according to these variables:

* ``COV_CORE_SOURCE``
* ``COV_CORE_CONFIG``
* ``COV_CORE_DATAFILE``
* ``COV_CORE_BRANCH``
* ``COV_CORE_CONTEXT``

Why does it have the ``COV_CORE`` you wonder? Well, it's mostly historical reasons: long time ago pytest-cov depended on a cov-core package
that implemented common functionality for pytest-cov, nose-cov and nose2-cov. The dependency is gone but the convention is kept. It could
be changed but it would break all projects that manually set these intended-to-be-internal-but-sadly-not-in-reality environment variables.

Coverage's subprocess support
=============================

Now that you understand how pytest-cov works you can easily figure out that using
`coverage's recommended <https://coverage.readthedocs.io/en/latest/subprocess.html>`_ way of dealing with subprocesses,
by either having this in a ``.pth`` file or ``sitecustomize.py`` will break everything:

.. code-block::

import coverage; coverage.process_startup() # this will break pytest-cov

Do not do that as that will restart coverage with the wrong options.

If you use ``multiprocessing``
==============================

Builtin support for multiprocessing was dropped in pytest-cov 4.0.
This support was mostly working but very broken in certain scenarios (see `issue 82408 <https://github.com/python/cpython/issues/82408>`_)
and made the test suite very flaky and slow.

However, there is `builtin multiprocessing support in coverage <https://coverage.readthedocs.io/en/latest/config.html#run-concurrency>`_
and you can migrate to that. All you need is this in your preferred configuration file (example: ``.coveragerc``):
Subprocess support was removed in pytest-cov 7.0 due to various complexities resulting from coverage's own subprocess support.
To migrate you should change your coverage config to have at least this:

.. code-block:: ini

[run]
concurrency = multiprocessing
parallel = true
sigterm = true

Now as a side-note, it's a good idea in general to properly close your Pool by using ``Pool.join()``:

.. code-block:: python

from multiprocessing import Pool

def f(x):
return x*x

if __name__ == '__main__':
p = Pool(5)
try:
print(p.map(f, [1, 2, 3]))
finally:
p.close() # Marks the pool as closed.
p.join() # Waits for workers to exit.


.. _cleanup_on_sigterm:

Signal handlers
===============

pytest-cov provides a signal handling routines, mostly for special situations where you'd have custom signal handling that doesn't
allow atexit to properly run and the now-gone multiprocessing support:

* ``pytest_cov.embed.cleanup_on_sigterm()``
* ``pytest_cov.embed.cleanup_on_signal(signum)`` (e.g.: ``cleanup_on_signal(signal.SIGHUP)``)

If you use multiprocessing
--------------------------

It is not recommanded to use these signal handlers with multiprocessing as registering signal handlers will cause deadlocks in the pool,
see: https://bugs.python.org/issue38227).

If you got custom signal handling
---------------------------------

**pytest-cov 2.6** has a rudimentary ``pytest_cov.embed.cleanup_on_sigterm`` you can use to register a SIGTERM handler
that flushes the coverage data.

**pytest-cov 2.7** adds a ``pytest_cov.embed.cleanup_on_signal`` function and changes the implementation to be more
robust: the handler will call the previous handler (if you had previously registered any), and is re-entrant (will
defer extra signals if delivered while the handler runs).

For example, if you reload on SIGHUP you should have something like this:

.. code-block:: python

import os
import signal

def restart_service(frame, signum):
os.exec( ... ) # or whatever your custom signal would do
signal.signal(signal.SIGHUP, restart_service)

try:
from pytest_cov.embed import cleanup_on_signal
except ImportError:
pass
else:
cleanup_on_signal(signal.SIGHUP)

Note that both ``cleanup_on_signal`` and ``cleanup_on_sigterm`` will run the previous signal handler.

Alternatively you can do this:

.. code-block:: python

import os
import signal

try:
from pytest_cov.embed import cleanup
except ImportError:
cleanup = None

def restart_service(frame, signum):
if cleanup is not None:
cleanup()

os.exec( ... ) # or whatever your custom signal would do
signal.signal(signal.SIGHUP, restart_service)

If you use Windows
------------------

On Windows you can register a handler for SIGTERM but it doesn't actually work. It will work if you
`os.kill(os.getpid(), signal.SIGTERM)` (send SIGTERM to the current process) but for most intents and purposes that's
completely useless.

Consequently this means that if you use multiprocessing you got no choice but to use the close/join pattern as described
above. Using the context manager API or `terminate` won't work as it relies on SIGTERM.

However you can have a working handler for SIGBREAK (with some caveats):

.. code-block:: python
patch = subprocess

import os
import signal
Or if you use pyproject.toml:

def shutdown(frame, signum):
# your app's shutdown or whatever
signal.signal(signal.SIGBREAK, shutdown)
.. code-block:: toml

try:
from pytest_cov.embed import cleanup_on_signal
except ImportError:
pass
else:
cleanup_on_signal(signal.SIGBREAK)
[tool.coverage.run]
patch = ["subprocess"]

The `caveats <https://stefan.sofa-rockers.org/2013/08/15/handling-sub-process-hierarchies-python-linux-os-x/>`_ being
roughly:
Note that if you enable the subprocess patch then ``parallel = true`` is automatically set.

* you need to deliver ``signal.CTRL_BREAK_EVENT``
* it gets delivered to the whole process group, and that can have unforeseen consequences
If it still doesn't produce the same coverage as before you may need to enable more patches, see the `coverage config <https://coverage.readthedocs.io/en/latest/config.html#run-patch>`_ and `subprocess <https://coverage.readthedocs.io/en/latest/subprocess.html>`_ documentation.
72 changes: 1 addition & 71 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,80 +1,17 @@
#!/usr/bin/env python

import re
from itertools import chain
from pathlib import Path

from setuptools import Command
from setuptools import find_packages
from setuptools import setup

try:
# https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
from setuptools.command.build import build
except ImportError:
from distutils.command.build import build

from setuptools.command.develop import develop
from setuptools.command.easy_install import easy_install
from setuptools.command.install_lib import install_lib


def read(*names, **kwargs):
with Path(__file__).parent.joinpath(*names).open(encoding=kwargs.get('encoding', 'utf8')) as fh:
return fh.read()


class BuildWithPTH(build):
def run(self, *args, **kwargs):
super().run(*args, **kwargs)
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
dest = str(Path(self.build_lib) / Path(path).name)
self.copy_file(path, dest)


class EasyInstallWithPTH(easy_install):
def run(self, *args, **kwargs):
super().run(*args, **kwargs)
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
dest = str(Path(self.install_dir) / Path(path).name)
self.copy_file(path, dest)


class InstallLibWithPTH(install_lib):
def run(self, *args, **kwargs):
super().run(*args, **kwargs)
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
dest = str(Path(self.install_dir) / Path(path).name)
self.copy_file(path, dest)
self.outputs = [dest]

def get_outputs(self):
return chain(super().get_outputs(), self.outputs)


class DevelopWithPTH(develop):
def run(self, *args, **kwargs):
super().run(*args, **kwargs)
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
dest = str(Path(self.install_dir) / Path(path).name)
self.copy_file(path, dest)


class GeneratePTH(Command):
user_options = ()

def initialize_options(self):
pass

def finalize_options(self):
pass

def run(self):
with Path(__file__).parent.joinpath('src', 'pytest-cov.pth').open('w') as fh:
with Path(__file__).parent.joinpath('src', 'pytest-cov.embed').open() as sh:
fh.write(f'import os, sys;exec({sh.read().replace(" ", " ")!r})')


setup(
name='pytest-cov',
version='6.3.0',
Expand Down Expand Up @@ -125,7 +62,7 @@ def run(self):
python_requires='>=3.9',
install_requires=[
'pytest>=6.2.5',
'coverage[toml]>=7.5',
'coverage[toml]>=7.10.6',
'pluggy>=1.2',
],
extras_require={
Expand All @@ -142,11 +79,4 @@ def run(self):
'pytest_cov = pytest_cov.plugin',
],
},
cmdclass={
'build': BuildWithPTH,
'easy_install': EasyInstallWithPTH,
'install_lib': InstallLibWithPTH,
'develop': DevelopWithPTH,
'genpth': GeneratePTH,
},
)
13 changes: 0 additions & 13 deletions src/pytest-cov.embed

This file was deleted.

1 change: 0 additions & 1 deletion src/pytest-cov.pth

This file was deleted.

15 changes: 0 additions & 15 deletions src/pytest_cov/compat.py

This file was deleted.

Loading
Loading