SIGSEGV: :dx-call on REDUCE stack-allocates a closure a conforming SEQUENCE:REDUCE method retains

[dg1sbg]

Summary

src/lisp/kernel/cleavir/setup.lisp grants the :dx-call attribute to the sequence functions, REDUCE among them. Cleavir's closure extent analysis reads that attribute as a guarantee that the call does not retain its function arguments, and stack-allocates the closure on the strength of it alone — there is no escape analysis on that path.

CL:REDUCE dispatches into SEQUENCE:REDUCE, which is a documented extension point, and a conforming user method may keep the function it is handed. When it does, the retained closure points into a frame that has already returned.

The comment directly above the grants already names the hazard:

;; Can't do DYN-CALL for most sequence functions, as ext sequence
;; functions can do arbitrary things.

:dx-call is granted on the lines that follow it.

Reproduction

No package-lock violations and no internal APIs — a class inheriting SEQUENCE, two protocol methods, and a SEQUENCE:REDUCE specialisation.

(defclass my-seq (sequence standard-object) ())
(defmethod sequence:length ((s my-seq)) 3)
(defmethod sequence:elt ((s my-seq) i) (- 3 i))

(defparameter *retained* nil)

;;; Conforming: a SEQUENCE:REDUCE method may do as it likes, including
;;; keeping the function it was given.
(defmethod sequence:reduce (fn (s my-seq) &key &allow-other-keys)
  (setf *retained* fn)
  0)

(defun leak (n)
  (let ((obj (make-instance 'my-seq)))
    (reduce (lambda (a b) (+ a b n)) obj)
    :done))

(let ((cmp:*compile-native* t)) (compile 'leak))

(leak 100)

(defun burn (n)
  (declare (fixnum n) (optimize speed (safety 0)))
  (if (<= n 0) 0 (+ (logxor n #xA5A5A5A5) (burn (1- n)))))
(let ((cmp:*compile-native* t)) (compile 'burn))
(burn 20000)
(gctools:garbage-collect)

(funcall *retained* 1 2)   ; the correct answer is 103

Observed

build/boehm/iclasp:

Condition of type: SEGMENTATION-VIOLATION
Segmentation fault. Attempted to access restricted memory address #x30.

build/boehmprecise/iclasp:

(funcall *retained* 1 2) => (:ERROR EXT:BUS-ERROR)

Wrapping clasp-cleavir::enclose shows the closure being handed :DYNAMIC, i.e. cc_stack_enclose over an alloca in leak's frame.

Native path only — the bytecode compiler does not run this pass. This checkout has :default-native nil, hence the explicit cmp:*compile-native* binding above; a :default-native t build reaches it without that.

Where it comes from

contrib/Cleavir/BIR-transformations/process-captured-variables.lisp, determine-closure-extent, first typecase clause:

(bir:call (when (safe-call-p use)
            (setf (bir:extent enclose) :dynamic)))

with

(defun safe-call-p (call)
  (attributes:has-flag-p (bir:attributes call) :dx-call))

That attribute lookup is the entire test.

I think this is Clasp's bug rather than Cleavir's. :dx-call is a client-supplied contract — s-expressionists/Cleavir#12 phrases it as "reduce, which does not increase the extent of its function arguments (according to some client)" — so Cleavir is entitled to trust it. The claim made in setup.lisp is the part that is not true.

Dispatch reachability

Measured on this build, so a fix can be scoped rather than applied blindly. CL:REDUCE does reach a user SEQUENCE:REDUCE method. CL:SORT, FIND-IF, COUNT-IF and POSITION-IF do not — they run the default elt-based implementations, so they are not vectors for this on their own. REMOVE-IF signals SEQUENCE:PROTOCOL-UNIMPLEMENTED. Each function flagged at setup.lisp:83-112 would want checking individually.

Possible fixes

1. Drop :dx-call from the sequence functions that can dispatch into user code. Sound and simple, but gives up the optimisation for the overwhelmingly common list and vector cases.
2. Condition the flag on the sequence argument being statically known to be a list or a vector, excluding the extended-sequence path. Keeps the optimisation where it actually pays.

Version

clasp-boehm-3.0.1-94-g5b83b0307-non-cst, macOS arm64, LLVM 22.1.8.

The tree carries unrelated local commits, but the implicated code is unmodified: the (bir:call (when (safe-call-p use) ...)) clause is verbatim from Cleavir main at d2a97073, and the setup.lisp grants are untouched.