Skip to content

Few digits of precision in N(). #10164

@sagetrac-gerbicz

Description

@sagetrac-gerbicz

For N(1+10**-400000,digits=400001) the displayed number of digits is only 400000, it is printing 1. followed by 399999 zeroes. The reason is that in functional.py: prec = int((digits+1) * 3.32192) + 1. However log(10)/log(2)~3.3219280948874>3.32192, so if digits is large the used precision will be smaller by some digits than the requested number of digits.

The suggestion is to use 3.32193 instead of 3.32192, see the trivial patch.


Apply: attachment: trac_10164_folder_sans_whitespace.patch to Sage library

CC: @kcrisman

Component: misc

Keywords: N, digits, numerical approximation

Author: Robert Gerbicz, Douglas McNeil

Reviewer: Karl-Dieter Crisman, Benjamin Jones

Merged: sage-5.1.beta3

Issue created by migration from https://trac.sagemath.org/ticket/10164

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions