history: keep the HISTFILE write position across multiline resolution

[dfherr]

And one more. this is a race.

ble.sh's multiline resolution rebuilds the bash history list (history -c and re-add), which renumbers it and can change the entry count, but the "written up to here" pointer (wskip/prevmax) isn't adjusted. The rebuild runs asynchronously and takes seconds on macOS, so it regularly finishes after the first command has been recorded, and the next history -a re-appends already written entries (here: the previous session's last entry and the first command). Keep the number of unwritten entries constant across the rebuild instead.

[akinomyoga]

I'm not sure if I understand the problem correctly, but it seems you care about the situation that history changes happen during the rebuilding, but then this PR doesn't seem to fix the issue that the history changes are lost, though it fixes the superficial inconsistency in the wskip?

[dfherr]

you're right. my explanation was sloppy.

I think it's easier to show with a way to reproduce:

You need:

• HISTSIZE bounded and the in-memory list full at startup, so the rebuild renumbers it. The variables must be in the environment of the test shell, not set in the rc.
• An encoded multiline entry among the last HISTSIZE entries of HISTFILE. Step 1 seeds it as entry 98.
• history -a after every command, here via bleopt history_share=1.
• A slow rebuild. The rc below delays the worker by 5 seconds with sleep (it was easy to reproduce with my 4s loading time before the TZ fix)
• echo one typed and entered before the first history rebuild happens, then wait at least 10 seconds. Recording echo one restarts the rebuild, so the delay is paid again before the list is renumbered.

these were always true for me when debugging the slow macos history load time, so i hit the bug frequently. now it's really hard to reproduce, but it's there.

Steps to reproduce. Every block is one command:

1. setup session

tmp=$(mktemp -d)
# 100 entries; entry 98 is an encoded multiline command, inside the last 50
for i in $(seq 100); do
  if ((i == 98)); then printf "eval -- \$'echo multi\\\\necho line2'\n"; else printf 'echo cmd%d\n' "$i"; fi
done > "$tmp/histtest"
cat > "$tmp/rc" <<'EOF'
[[ $- == *i* ]] && source ~/project/ble.sh/out/ble.sh --attach=none
bleopt history_share=1
shopt -s histappend
PS1='A$ '
eval "$(declare -f ble/history:bash/resolve-multiline/.worker | sed '1s|.*|function ble/history:bash/resolve-multiline/.worker.orig|')"
function ble/history:bash/resolve-multiline/.worker { sleep 5; ble/history:bash/resolve-multiline/.worker.orig "$@"; }
[[ ! ${BLE_VERSION-} ]] || ble-attach
EOF
HISTFILE=$tmp/histtest HISTSIZE=50 HISTFILESIZE=100 bash --rcfile "$tmp/rc" -i

2. In the session

First within less than 5 seconds of opening the session:

echo one # wait 5 seconds

Wait at least 5 seconds. Type nothing. A space-prefixed command or a repeat of the previous command makes option:s return early, and history -a then runs its "uncontrolled change" repair, which hides the bug.

then one command each

echo two

echo three

tail -n 4 $HISTFILE; history | tail -n 5;

Outcome on master. echo two is missing from HISTFILE.

A$ tail -n 4 $HISTFILE; history | tail -n 5 ;                                                                                                                                                                                                                                                                          54/100%
echo cmd100
echo one # wait 5 seconds
echo three
tail -n 4 $HISTFILE; history | tail -n 5 ;
   49  echo cmd100
   50  echo one # wait 5 seconds
   51  echo two
   52  echo three
   53  tail -n 4 $HISTFILE; history | tail -n 5

Outcome with fix:

A$ tail -n 4 $HISTFILE; history | tail -n 5 ;                                                                                                                                                                                                                                                                          54/100%
echo one # wait 5 seconds
echo two
echo three
tail -n 4 $HISTFILE; history | tail -n 5 ;
   49  echo cmd100
   50  echo one # wait 5 seconds
   51  echo two
   52  echo three
   53  tail -n 4 $HISTFILE; history | tail -n 5 ;

Why this helps: the rebuild replays Bash's list with history -c and re-adds, which restarts the numbering at 1, but wskip is an absolute history number, so after the rebuild it points past the end and the next history -a computes a negative delta and writes nothing; this measures the unwritten tail as a count right before the rebuild (max - wskip), re-derives wskip from the new max afterwards and moves prevmax along with it, so the following history -a writes exactly the entries that had not been written yet.

the root cause is that ble.sh stores "written up to here" as an absolute number while the only operation that renumbers is this one rebuild, so re-deriving the number in .load is the minimal correct fix. structurally tracking unwritten entries as account everywhere in .write and .check-uncontrolled-change might be better, but is a much bigger change.