doctests: Detect "safe" external features even if "--optional=external" is not used #32174
Description
(from #30887 comment:28)
For some "safe" features such as imagemagick, rubiks, 4ti2 (#30887), pandoc, it really shouldn't require the use of external to check for these.
Likewise for the PythonModule features for Sage modules introduced in #32614. By delaying the test for these features, using the sage.doctest.external machinery, we fix the broken detection of some of these features. (This is why the ticket is marked "critical".)
This is a step toward #30746 (sage.doctest.control: Replace use of sage.misc.package.list_packages)
With this ticket, more doctests will be run by default. This may lead to new failures (discovered by patchbot for instance). But that does not mean it is introduced in this ticket, just that they were existing before this ticket, but simply not executed.
For example, let's compare the behavior of the following command:
sage -tp --show-skipped src/sage/tests/cmdline.py
returns:
BEFORE (9.5.beta7):
Git branch: develop
Using --optional=4ti2,build,ccache,cryptominisat,debian,dochtml,dot2tex,e_antic,fricas,glucose,latte_int,lidia,normaliz,notedown,pandoc_attributes,pip,pycosat,pynormaliz,rst2ipynb,sage,sage.geometry.polyhedron,sage.rings.real_double,sage_numerical_backends_coin,sage_spkg
Doctesting 1 file using 8 threads.
sage -t --random-seed=265206128165572261397002909756599539048 src/sage/tests/cmdline.py
3 gdb tests not run
12 internet tests not run
2 pandoc tests not run
4 r tests not run
0 tests not run because we ran out of time
[235 tests, 30.65 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
AFTER (9.5.beta7 + #32174):
$ sage -tp --show-skipped src/sage/tests/cmdline.py
Git branch: 32174
Using --optional=4ti2,build,ccache,cryptominisat,debian,dochtml,dot2tex,e_antic,fricas,glucose,latte_int,lidia,normaliz,notedown,pandoc_attributes,pip,pycosat,pynormaliz,rst2ipynb,sage,sage_numerical_backends_coin,sage_spkg
Features to be detected: 4ti2,benzene,bliss,buckygen,conway_polynomials,csdp,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_jones_numfield,database_knotinfo,dvipng,ffmpeg,graphviz,imagemagick,jupymake,kenzo,latte_int,lrslib,mcqd,meataxe,pandoc,pdf2svg,plantri,pynormaliz,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.plot,sage.rings.number_field,sage.rings.real_double,sage.symbolic,sage_numerical_backends_coin,tdlib
Doctesting 1 file using 8 threads.
sage -t --random-seed=100100306103974678574881156032164018710 src/sage/tests/cmdline.py
**********************************************************************
File "src/sage/tests/cmdline.py", line 732, in sage.tests.cmdline.test_executable
Failed example:
print(open(output, 'r').read() == t) # optional - pandoc
Expected:
True
Got:
False
**********************************************************************
1 item had failures:
1 of 238 in sage.tests.cmdline.test_executable
3 gdb tests not run
12 internet tests not run
4 r tests not run
0 tests not run because we ran out of time
[237 tests, 1 failure, 32.76 s]
----------------------------------------------------------------------
sage -t --random-seed=100100306103974678574881156032164018710 src/sage/tests/cmdline.py # 1 doctest failed
----------------------------------------------------------------------
Features detected for doctesting: pandoc
We observe that the number of tests run went from 235 to 237, because the feature pandoc was detected. Note that the doctest failure is already tracked at #32697.
Depends on #32866
Depends on #27744
CC: @dimpase @slel @antonio-rojas @kiwifb @kwankyu @seblabbe @videlec
Component: doctest framework
Author: Matthias Koeppe, Sébastien Labbé
Branch: 41c4cd4
Reviewer: Sébastien Labbé
Issue created by migration from https://trac.sagemath.org/ticket/32174