sage_wraps should only read the sources of wrapped functions when needed.

[simon-king-jena]

According to private conversation with Francois Bissey, #9976 has introduced a problem that makes it impossible start sage-4.7.1 on Gentoo. That's pretty serious, and so this ticket is a blocker.

The problem is that sage_wraps reads the sources of to-be-wrapped functions and methods. The sources are stored as an attribute to the wrapper, which, on the one hand, is a waste of memory. Worse: Reading the Cython sources is impossible for sage-on-gentoo. Therefore, sage can't start.

My patch introduces some lightweight classes, that simply store a reference to an object (e.g., the to-be-wrapped function) and return the sources or source lines or the argument list only when called.

---

Apply only attachment: trac11734_sage_wraps_no_sourceread_lambda.proper.patch to the Sage library.

CC: @kiwifb

Component: misc

Keywords: sage_wraps sources gentoo startuptime sd32

Author: Julian Rueth

Reviewer: Simon King

Merged: sage-4.7.2.alpha3

Issue created by migration from https://trac.sagemath.org/ticket/11734

[simon-king-jena]

sage_wraps should only read source files when needed.

[simon-king-jena]

comment:1

Attachment: trac11734_sage_wraps_no_sourceread.patch.gz

The patch should solve the problem with prematurely reading sources on gentoo. However, I can not test it, myself. Francois, can you verify it?

Benefits

Startuptime

The startuptime did slightly improve (but perhaps that is not significant):

Without patch

$ ./sage -startuptime
== Tree ==
sage.all: 1.132 (None)
...

With patch

$ ./sage -startuptime
== Tree ==
sage.all: 1.045 (None)

Memory

The memory consumption improved as well:

Without patch:

sage: get_memory_usage()
861.2890625

With patch:

sage: get_memory_usage()
861.234375

Bugfix

Not reading the sources when a function is wrapped has an additional benefit: If one interactively defines a lambda function and wraps it, then the source is not available. Hence, the following example (that is a new doctest) used to fail:

        sage: def square(f):
        ...     @sage_wraps(f)
        ...     def new_f(x):
        ...         return f(x)*f(x)
        ...     return new_f
        ... 
        sage: f = lambda x:x^2
        sage: g = square(f)
        sage: g(3)
        81

Without the patch, the last line fails with a mysterious attribute error (the user would certainly have not easily been able to track the problem down).

[simon-king-jena]

Changed keywords from sage_wraps sources gentoo to sage_wraps sources gentoo startuptime

[saraedum]

comment:3

I do not fully understand your patch. How does it differ from simply taking out the call to sage_getargspec and moving it into a lambda? (see my attached patch)

Btw. On my install of sage-4.7.2alpha2 the doctest described in Bugfix does not fail.

[simon-king-jena]

comment:4

Replying to @saraedum:

I do not fully understand your patch. How does it differ from simply taking out the call to sage_getargspec and moving it into a lambda? (see my attached patch)

If I remember correctly, I once tried that approach. The line argspec = sage_getargspec(wrapped) was introduced for a reason (something went wrong), but I can't remember what it was.

Did you run full tests with your patch?

Btw. On my install of sage-4.7.2alpha2 the doctest described in Bugfix does not fail.

Really? Strange. It failed for me both with sage-4.7.2.alpha1 and with sage-4.6.2.

[simon-king-jena]

comment:5

The tests in sage/misc pass with your patch, and the example from my post above (the "bugfix") works as well.

If all tests pass, then you patch should be preferred.

It turned out that I am not to blame for the "argspec = sage_getargspec(wrapped)". The first author of #9976 had introduced an alternative approach to provide the argument list when building the documentation -- and I think what we have here is an artefact of that approach.

Since it was about documentation: Could you please build the documentation with your patch (I will only be able to do so tomorrow) and look whether the list of arguments of functions and methods created with @sage_wraps are correct in the reference manual? See, for example, the interreduced_basis method of multivariate polynomial ideals.

[simon-king-jena]

comment:6

PS: Even though the example from my previous post does not fail on your machine (for reasons that I don't understand), it did fail for me. Hence, I think it would make sense to include that example into your patch (you can of course simply copy the example from my patch).

[saraedum]

comment:7

I ran the doctests without long and they passed. Let me try to build the reference manual.

[saraedum]

comment:8

The reference manual seems ok. For example I get complete_primary_decomposition(algorithm='sy') which is a method with the * @libsingular_standard_options* decorator.

[simon-king-jena]

comment:9

I can confirm that the doc tests pass. I can also confirm that your patch provides a similar (small) positive side effect (comparable to what I stated on my patch) on startuptime and memory consumption.

Please add the above "bugfix" example to your patch. In the author field, please replace my name by yours.