prin1 prints some double-floats with too few digits to read back the same float

[dg1sbg]

prin1 on certain double-floats produces a decimal string that does not read back to the same float — it emits one significant digit too few.

Minimal reproducer

(let* ((d (scale-float (float 8728896881958312 0d0) 9))
       (s (prin1-to-string d))
       (r (read-from-string s)))
  (format t "~&printed:       ~a~%" s)
  (format t "~&reads back as: ~a~%" (prin1-to-string r))
  (format t "~&(eql d r) =>   ~s~%" (eql d r)))

Observed:

printed:       4.469195203562656d18
reads back as: 4.4691952035626563d18
(eql d r) =>   NIL

16 significant digits are printed; 17 are needed for this value.

Expected: (eql d r) is T. CLHS 22.1.3.1.3 requires a float to be printed so that it reads back as the same float, and the printer is otherwise expected to produce the shortest representation that does so.

How common

About 1 in 13000. The sweep below is deterministic — it uses an LCG rather than cl:random, so the identical set of doubles is tested on every build and results are comparable across binaries.

(defun lcg-next (x)
  (mod (+ (* 6364136223846793005 x) 1442695040888963407) (expt 2 64)))

(defun make-double (x)
  (scale-float (float (ldb (byte 53 0) x) 0d0)
               (- (ldb (byte 8 53) x) 128)))

(let ((fails 0) (checked 0) (x 42))
  (dotimes (i 400000)
    (setf x (lcg-next x))
    (let ((d (make-double x)))
      (when (and (typep d 'double-float) (not (zerop d)))
        (incf checked)
        (let* ((s (with-standard-io-syntax (prin1-to-string d)))
               (y (with-standard-io-syntax
                    (let ((*read-default-float-format* 'double-float))
                      (read-from-string s)))))
          (unless (eql d y)
            (incf fails)
            (when (<= fails 5)
              (format t "~&iter ~6d  ~a  reads back as  ~a~%            integer-decode-float => ~s~%"
                      i s y (multiple-value-list (integer-decode-float d)))))))))
  (format t "~&~%checked ~d doubles, ~d failed to round-trip~%" checked fails))

Output — 30 failures out of 400000 checked:

iter   5188  4.469195203562656d18  reads back as  4.4691952035626563d18
            integer-decode-float => (8728896881958312 9 1)
iter  45858  4.44151070753808d18  reads back as  4.4415107075380803d18
            integer-decode-float => (8674825600660312 9 1)
iter  46629  5.16378726698624d19  reads back as  5.1637872669862404d19
            integer-decode-float => (6303451253645312 13 1)
iter  84937  3.407763170278048d18  reads back as  3.4077631702780483d18
            integer-decode-float => (6655787441949312 9 1)
iter  87221  4.296357292048032d18  reads back as  4.2963572920480323d18
            integer-decode-float => (8391322836031312 9 1)

checked 400000 doubles, 30 failed to round-trip

Every failure I captured sits around 1e18–1e19, with integer-decode-float exponents of 9 or 13.

Versions

Reproduces identically — the same 30 values, at the same iteration indices, with the same integer-decode-float output — on both:

• clasp-boehmprecise-2.7.0-873-g46626382f
• a build of current main (fc2beb4) carrying unrelated local commits

So this is long-standing rather than a recent regression. macOS arm64, :build-mode :native, :extensions ().

Side effect: ansi-test is flaky because of this

PRINT.DOUBLE-FLOAT.RANDOM (dependencies/ansi-test/printer/print-floats.lsp:276) draws random doubles across the whole magnitude range and checks readability, so at a ~1-in-13000 defect rate it fails some runs and passes others.

I saw exactly that: two runs of ninja -C build ansi-test on builds differing only by a semantics-neutral change gave 27 failures / 0 unexpected, then 28 failures / 1 unexpected (PRINT.DOUBLE-FLOAT.RANDOM), with every other entry identical. Since the target exits non-zero on any unexpected failure, its exit code alone is not a reliable signal — the failure lists have to be compared. PRINT.LONG-FLOAT.RANDOM is already in tools-for-build/ansi-test-expected-failures.sexp; the double-float one is not.

[yitzchak]

This is a known issue. There are some instances when the last digit is off by one because of the number of digits requested would have resulted in rounding in a different direction than Schubfach assumes. I've fixed this in Incless+Inravina+Invistra printer so I am very confident of the results.

For PRIN1 this shouldn't matter since it prints as many digits as needed. In fact:

$ sbcl 
This is SBCL 2.6.7, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (asdf:load-system :incless-extrinsic)
T
* (defvar q (scale-float (float 8728896881958312 0d0) 9))
Q
* (incless-extrinsic:prin1 q)
4.469195203562656d18
4.469195203562656d18
* (read-from-string (incless-extrinsic:prin1-to-string q))
4.469195203562656d18
20

So 4.469195203562656d18 is actually the correct printed representation. I spot checked the other ones you listed and they all appear correct. Which means its not an issue with PRIN1, but most likely with READ.

@scymtym had done some work a while back to use Quaviver in Eclector which would probably fix this. I think that he had some other issues he wanted to iron out first though. Currently, Eclector coerces floats in tokens.lisp via:

                   (let ((magnitude (* (decimal-mantissa)
                                       (expt 10 (- (if exponentp
                                                       (* exponent-sign (exponent))
                                                       0)
                                                   decimal-exponent)))))
                     (* sign (coerce magnitude type))))))

So the current issue is probably in COERCE.