MP:CAS silently loses updates on the bytecode path: core:acas is not atomic

[dg1sbg]

Summary

MP:CAS silently loses updates whenever the compiler does not inline it — which, in Clasp 3.0 with
its bytecode default, is the ordinary case rather than a corner case. The runtime fallback
core:acas is a plain read-compare-write, and core:atomic-aref and its setf are plain
non-atomic accesses. All three ignore the memory order.

The code says so itself, which is why I want to be clear this is a severity report rather than a
discovery — src/core/array.cc:555-559:

// Big evil FIXME: These are functions are basically backup for when the
// compiler can't inline an atomic access. BUT: They aren't actually atomic.
// This is difficult to fix as the underlying GCArrays are actually not atomic.
// Perhaps C++20's atomic_ref could help in the future?

What seems to have changed since that comment was written is how often the fallback is reached.

Reproduction

Four threads, 50 000 CAS-retry increments each, on one simple-vector slot. An atomic CAS must
end at exactly 200 000.

(defparameter *v* (make-array 1 :initial-element 0))
(defun bump (n)
  (dotimes (i n)
    (loop for old = (svref *v* 0)
          until (eq old (mp:cas (svref *v* 0) old (1+ old))))))
(let ((threads (loop repeat 4 collect (mp:process-run-function
                                       (gensym) (lambda () (bump 50000))))))
  (mapc #'mp:process-join threads))
(format t "~&compile-native=~a  BUMP is a ~a~%RESULT: counter=~d want=200000~%"
        compiler:*compile-native* (type-of (symbol-function 'bump)) (svref *v* 0))

Result on clasp-boehmprecise-3.0.1-112-gc7faba5ec, macOS arm64:

compile-native=NIL  BUMP is a BYTECODE-SIMPLE-FUN
RESULT: counter=198885 want=200000

1115 increments lost, with no warning and no condition. The loop is a correct CAS retry loop;
it is the CAS that does not serialise.

Why the fallback is now the common path

compiler:*compile-native* and compiler:*compile-file-native* both default to NIL on this
installation, so compile and compile-file produce bytecode, and the bytecode path has no atomic
handling — it emits an ordinary call to core:acas. Verified in both shipped binaries with
--norc. So any code that is interpreted, bytecode-compiled, or compiled with a non-constant
order, or that CASes an array type the transforms do not cover, gets the non-atomic fallback.

src/core/array.cc:578-585:

CL_DEFUN T_sp core__acas(T_sp order, T_sp cmp, T_sp nvalue, Array_sp array, Vaslist_sp indices) {
  (void)order; // ignore
  T_sp old = cl__aref(array, indices);
  if (cmp == old)
    core__aset(nvalue, array, indices);
  return old;
}

The same shape appears at array.cc:565-576 (core:atomic-aref and its setf) and, for the order
argument only, at src/core/cons.cc:161,171,181,191 and src/core/instance.cc:91-98. By contrast
core__cas_rack at instance.cc:104 does validate its order and signal — so the validating
pattern already exists in the tree.

Why it matters

A CAS that silently fails to serialise is the failure mode that does not show up in testing and
does not show up in review. It looks like a working lock-free algorithm and corrupts a counter
under contention. Compared with the alternatives, a loud refusal would be strictly better: SBCL's
sb-ext:cas and AllegroCL's excl:atomic-conditional-setf both compile to a single LOCK CMPXCHG
on every path, so portable code written against MP:CAS acquires a Clasp-only silent data race.

Suggested direction, in increasing order of effort

1. Make the fallbacks signal rather than pretend. If core:acas cannot be atomic, having it
   signal — as core__cas_rack already does for a bad order — converts a silent data race into a
   loud, locatable error. This alone would remove the dangerous case.
2. Make them actually atomic where the underlying storage permits, e.g. std::atomic_ref over
   the element for the T-element case the transforms already special-case.
3. Document the boundary: MP:CAS's docstring at src/lisp/kernel/lsp/atomics.lisp:151-167
   describes the interface without saying that atomicity depends on inlining. A sentence naming the
   condition would let callers reason about it.

I am happy to prepare a PR for (1) if that direction is acceptable.

Environment

• clasp-boehmprecise-3.0.1-112-gc7faba5ec-non-cst, macOS arm64, Boehm precise GC, ASDF 3.3.7.2
• *features* includes :CLASP :ARM64 :DARWIN :THREADS :USE-PRECISE-GC :USE-BOEHM :RELEASE-BUILD;
  no :CCLASP or :BCLASP
• Verified against upstream/main @ 205f83feca442b694b9d1cb02ca4a573d37cc845 — same code