Skip to content

feat: add a minutes-per-run activity goal for Auto-run - #82

Merged
OjusWiZard merged 15 commits into
mainfrom
iasonrovis/ope-1801-agent-trading-after-having-earned-rewards
Oct 9, 2026
Merged

OjusWiZard merged 15 commits into
mainfrom
iasonrovis/ope-1801-agent-trading-after-having-earned-rewards

Conversation

@valory-coding-agent

@valory-coding-agent valory-coding-agent Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Implements: https://linear.app/valory/issue/OPE-1801

Summary
Connect gets a minutes-per-run activity goal (default 15, CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN). It is published as the shared activity_goal block (scope README D1, unit: "minutes") in the agent_performance.json that ActivityLog already writes, so Pearl's Auto-run rotates on Connect's is_met like any other agent's (README D5).

  • New connect/run_goal.py (RunGoal) owns the run window: one per process, opened at start and never rolled over. is_met is progress >= target, so once the minutes have elapsed it stays true for the rest of the process. A target change is measured against the same window: raising it above progress makes the goal unmet again, lowering it to or below progress meets it. A target of 0 is met at once. last_met_at is stamped once per process, on the first false → true. Connect never stops itself.
  • A job in the FastAPI lifespan rewrites the file every 15 seconds (via asyncio.to_thread). A failed write, of any kind, is logged and never ends the job.
  • The user's value lives in the file itself. The session changes activity_goal.target directly, and each server write reads it back under ActivityLog's lock. A valid new value is adopted and audited (minutes_per_run_changed). An invalid value, or a file that cannot be parsed at all, is logged, and the current target is written back. One check_target (whole number >= 0, not a bool) validates the env var, the read-back and RunGoal itself; a bad env var is a configuration error, like other bad config. The file is in STORE_PATH, so the value survives restarts.
  • The session brief (connect/assets/CLAUDE.md) gets a "Minutes per run" section, and the pearl-connect skill points to it. The brief explains what the number means under Auto-run, says to change it only when the user asks, read it back and confirm it, and to decline anything it cannot map. It mentions no guardrail mode and tells the session never to stop itself.
  • service.yaml declares minutes_per_run: ${MINUTES_PER_RUN:int:15}, and the packages are re-locked. README.md and CLAUDE.md are updated alongside the code.
  • Untouched, as the scope requires: signer.py, guard.py, settings.py, the MCP tools and the routes.

Points for reviewers

  • The boot restore is audited under its own kind. When the first read after a process start adopts a stored target that differs from the configured one, it writes a minutes_per_run_restored entry, not minutes_per_run_changed. That entry may be the value from before the restart, or an edit the session made while Connect was stopped. A stored value equal to the configured one writes nothing, so a restart with nothing changed adds no entry. Later edits by the session are audited as minutes_per_run_changed, as the scope specifies. Once stored, the file's value wins over CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN, so changing the env var later does not reach an existing install.
  • ActivityLog(store_path, run_goal=None) falls back to a 15-minute RunGoal, so existing call sites and fixtures stay unchanged. __main__ always passes the configured one.
  • A file with no activity_goal block logs one warning, as the scope's test list requires. Every existing install will therefore log it once, on its first boot after upgrading. A missing file is treated as a fresh store and is not warned about.
  • If the disk is unwritable, the 15-second job logs every tick. That is noisy, but the process never stops.
  • Self-review fix: a failed audit append no longer aborts the performance write or gets blamed on the caller's activity entry.
  • Design change in review (sticky is_met): the earlier rolling window, where is_met stayed false and Auto-run keyed on last_met_at, was replaced at the reviewer's request on olas-operate-app#2155. The scope (README D5, technical/connect.md) is updated to match.
  • Not run: the manual checks in scope Section 9 (a standalone run with the env var set to 1, and live Claude Code session tests), integration-tests (needs an RPC and Docker) and launch-tests (needs a TTY).

Verification

MUTATION (review round 3) tests/test_extras.py::TestActivityGoal::test_target_in_the_file_survives_a_restart: FAILS-ON-BASE (restore now audited as minutes_per_run_restored)
MUTATION (review round 3) tests/test_extras.py::TestActivityGoal::test_stored_default_is_not_audited_as_a_restore: PASSES-ON-BASE - guards the existing stored == configured early return so restarts with nothing changed stay out of the audit log
GATE flake8 / pylint / black-check / isort-check / bandit / safety / mypy / check-copyright (review round 3, tomte 0.7.0 on Python 3.12): PASS
GATE unit-tests-coverage (review round 3): PASS - 1242 passed, 100% coverage; the only failure is the same sandbox-only tests/test_connect_polymarket_skill.py::test_bootstrap_refuses_when_there_is_no_workspace noted below
MUTATION tests/test_run_goal.py, tests/test_extras.py (sticky is_met, this round): FAILS-ON-BASE (run_goal.py reverted to the rolling window: goal met and stays met, raising the target after met, last_met_at stamped once, lowering the target to progress)
MUTATION tests/test_endpoints.py: FAILS-ON-BASE (TestPerformanceRefresh - job writes the block while serving / failing disk does not end the job)
MUTATION tests/test_config.py, tests/test_extras.py, tests/test_run_goal.py: UNVERIFIED - collection ImportError on base (new MINUTES_PER_RUN_ENV / connect.run_goal). A manual per-behaviour revert during implementation made the covering tests fail (boot restore, target read-back, bool rejection, job write); the audit-failure test fails with the self-review fix reverted
MUTATION (review round 2) tests/test_endpoints.py::test_failing_write_does_not_end_the_job[RecursionError], tests/test_extras.py::TestActivityGoal (unusable file: too-deeply-nested; invalid configured default stops the boot), tests/test_config.py::test_invalid_minutes_per_run_is_a_configuration_error, tests/test_run_goal.py::test_target_that_is_not_whole_minutes_is_refused: FAILS-ON-BASE
MUTATION (review round 2) test_job_adopts_a_session_edit, test_job_writes_the_block_while_serving_and_stops_after (now counts writes across shutdown), test_block_is_written_with_the_default_target (no warning on a fresh store): PASSES-ON-BASE - they close coverage gaps on behaviour this round did not change; the fresh-store test was checked to fail with the FileNotFoundError branch removed
GATE flake8 / pylint / black-check / isort-check / bandit / mypy / check-copyright (review round 2, tomte 0.7.0 on Python 3.12): PASS
GATE unit-tests-coverage (review round 2): PASS - 100% coverage; the only failure is the same sandbox-only tests/test_connect_polymarket_skill.py::test_bootstrap_refuses_when_there_is_no_workspace noted below
GATE gitleaks (review round 2): PASS
GATE ascii-only (packages/): PASS
GATE flake8 / pylint / black-check / isort-check / bandit / safety / mypy / check-copyright: PASS
GATE autonomy packages lock --check: PASS
GATE uv lock --check: PASS
GATE gitleaks: PASS
GATE black-check / isort-check / flake8 / mypy / pylint / check-copyright (this round, Python 3.12): PASS
GATE unit-tests-coverage: FAIL locally on 1 test only - tests/test_connect_polymarket_skill.py::test_bootstrap_refuses_when_there_is_no_workspace. It fails identically on main in this sandbox, because a .mcp.json/.venv in the parent directory of the clone is picked up; it passes in a clean worktree. 1232 other tests pass, coverage 100%

Technical scope: OPE-1801/technical/connect.md on branch iasonrovis/ope-1801-agent-trading-after-having-earned-rewards of valory-xyz/scopes

🤖 Generated with Claude Code

valory-coding-agent and others added 7 commits October 7, 2026 17:39
The default length of Connect's run window, declared in the service
package so a template can override it. A bad value falls back to the
default with a warning rather than failing boot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Connect has no staking epoch, so to take part in Auto-run it needs a
stopping signal of its own: a run window that stamps last_met_at when
its minutes elapse and immediately reopens, since Connect never stops
itself. Rolling over at the current time rather than the theoretical
boundary turns a suspend into one completed run, not a burst.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent_performance.json now carries the activity_goal block Pearl reads
to decide Connect's Auto-run turn. The user's minutes per run live in
that block's target: every write reads it back, adopts a valid value and
puts the current one back over anything else, so the session can change
it by editing the file and the value survives restarts with no second
store. A boot restoring the stored value is not audited as a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without signer activity nothing wrote the file, so the run window would
never roll over for Pearl to see and a session's edit of the target
would never be read. The job runs in the app lifespan so TestClient
exercises it, and does its blocking write in a worker thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user changes Connect's run window by asking the session, which edits
activity_goal.target in agent_performance.json. The brief scopes that
edit to the one field, has the session confirm the value it reads back
and decline what it cannot map, and keeps it from touching the value
unasked. It deliberately says nothing about stopping at the end of a run:
Auto-run decides that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed minutes_per_run_changed audit append no longer aborts the
agent_performance.json write or gets reported as a failure of the
caller's activity entry; it is logged on its own and the change stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread connect/server/app.py

@valory-coding-agent valory-coding-agent Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — OPE-1801

Scope: Agent trading after having earned rewards — Connect minutes-per-run activity goal (OPE-1801/technical/connect.md)
Files reviewed: 15 of 15 changed
Comments: 0

Key findings

None. All seven dimensions returned no findings. The orchestrator also re-read connect/run_goal.py, connect/activity.py, connect/server/app.py, connect/config.py and connect/__main__.py, and found nothing that clears the evidence bar. Lock ordering holds (ActivityLog._lock, then RunGoal's lock), and _append does not re-take the non-reentrant lock. The read-back fails open, and the lifespan task is cancelled and awaited on shutdown.

Dimensions

Dimension Critical Important Advisory
Security & quality 0 0 0
Error handling 0 0 0
Test coverage 0 0 0
Type design 0 0 0
Comments 0 0 0
Simplification 0 0 0
Scope alignment 0 0 0
Orchestrator 0 0 0

Scope coverage

  • ✅ RunGoal module and D5 run-window rollover semantics (reopen at "now", a target of 0 is always met)
  • ✅ activity_goal block in agent_performance.json with the D1 shape
  • ✅ The session's write of activity_goal.target is read back, validated and audited (minutes_per_run_changed)
  • ✅ The target survives a restart through the boot read-back, which records no audit entry
  • ✅ CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN env var, with a fallback to the default of 15
  • ✅ 15-second lifespan refresh job: runs off the event loop, tolerates a failing disk, is cancelled on shutdown
  • ✅ service.yaml override and packages.json re-lock
  • ✅ README.md and CLAUDE.md updated with the code
  • ✅ "Minutes per run" section in the session brief and a pointer in the skill. Neither mentions a guardrail mode or tells the session to stop
  • ✅ Tests follow the scope's Section 9 strategy
  • ⏳ olas-operate-app Auto-run rotation, checklist and notification, plus the trader and optimus goals. These are separate repos and PRs per the scope README

Not reviewed

  • The tests were not re-run by the reviewer: the sandbox could not build the uv environment (the safe-pysha3 native build failed). CI and the PR's own verification block are the evidence for the gates.
  • The manual checks in scope Section 9 (a standalone run with the env var set to 1, and live session tests) remain open, as the PR body states.

🤖 Coding Agent [Beta] · Automated review against OPE-1801

Auto-run can now treat Connect like every other agent: it rotates on the
false -> true of is_met instead of keying on a Connect-only reading of
last_met_at. The window opens at process start and never rolls over;
is_met is progress >= target, so a target change is measured against the
same window. last_met_at is stamped once, on the first time the goal is
seen met.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@OjusWiZard OjusWiZard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sticky is_met switch looks right. Requesting changes for the items below.

1. The refresh job can die silently (connect/server/app.py refresh_performance, connect/activity.py _stored_target)

write_performance catches only OSError, _stored_target only OSError/ValueError, and the loop has no try. Reproduced: an agent_performance.json of 100 000 [ makes write_performance() raise RecursionError, and the session can write that file. The task then ends with nothing logged, is_met/progress freeze and target edits are never adopted (Auto-run hands over late or never), and the lifespan finally re-raises the error at shutdown.

Fix: try/except Exception: logger.exception(...) around the loop body, and treat any parse failure in _stored_target as unusable.

2. Test gaps (coverage is 100%, but these regressions would pass)

  • The job adopting a session edit is untested: test_job_writes_the_block_while_serving_and_stops_after only checks that the file appears. Edit target while serving and assert it is adopted and audited as minutes_per_run_changed.
  • Shutdown is checked by deleting the file and sleeping 0.1 s. Count write_performance calls across shutdown instead.
  • A fresh store must not warn: assert no WARNING on the first write with no file, so dropping the FileNotFoundError branch fails a test.
  • test_unusable_file_keeps_the_current_target asserts only "agent_performance.json" in caplog.text. Assert the specific warning per case, and that no minutes_per_run_changed entry was written.

3. A bad CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN is logged where nobody looks

load_config() runs before setup_logging() in connect/__main__.py, so the "using 15" warning goes only to stderr, not log.txt. Log it after setup_logging, or raise it as a configuration error like the other bad config.

4. Target validation lives in two places, and RunGoal does not enforce it

_parse_minutes_per_run (config.py) and _stored_target (activity.py) each check "int >= 0, not bool"; RunGoal.__init__/set_target accept anything, and a negative target is met at once. One shared check, used by both parsers and raised on in RunGoal, replaces the two.

5. Shorter session instructions, and say what the setting does

The brief's "Minutes per run" section tells the session how to edit a field, but not what the number means for the user. Make it shorter and explain Auto-run. Something like:

Minutes per run

With Pearl's Auto-run, the user's agents take turns. This agent's turn lasts at least N minutes: after that, Pearl hands over to the next agent that can run, or lets this one continue if none can. N is activity_goal.target in agent_performance.json; 0 hands over at Pearl's next check. Never stop yourself — Pearl ends the turn.

Change N only when the user asks: set activity_goal.target to a whole number of minutes, 0 or more (if you can't map the request to one, say so and change nothing), leave the rest of the file as is, then read it back after about 15 seconds and confirm the value.

The agent_performance.json bullet in the "don't hand-edit" list can then just point at this section, and the pearl-connect SKILL.md addition can shrink to one line pointing to it (or go, since the brief is always loaded).

6. Comments and docs

  • connect/run_goal.py module docstring: 7 lines that repeat README "Run window". One line.
  • connect/activity.py module docstring ("so the user's minutes per run live in the file and survive restarts"): repeats the README; point to it.
  • README "Run window", the last paragraph restates the brief's rules for the session (change only when asked, confirm, decline). Link to the brief instead.
  • PR description and scope D5 say last_met_at is stamped once "as for Trader and Optimus". Trader stamps it on every false → true; Connect once per process. Drop the comparison.
  • PR description: the second MUTATION line still lists "rollover at "now"".

valory-coding-agent and others added 4 commits October 8, 2026 11:00
A write that raised anything but OSError ended the task silently, freezing
the activity goal Pearl reads. Tests now also cover the job adopting a
session edit and stopping at shutdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
check_target is shared by the env var parser, the read-back of
agent_performance.json and RunGoal itself, which now refuses a negative
target. A bad CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN is a configuration
error, so it reaches log.txt instead of a pre-logging stderr warning. Any
parse failure of the stored file, RecursionError included, is unusable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@valory-coding-agent

Copy link
Copy Markdown
Author

Addressed the 6 items in the review from @OjusWiZard:

  1. The refresh job can no longer die silently. refresh_performance now catches any exception around the write and logs it with logger.exception. _stored_target treats any parse failure as an unusable file, including the RecursionError from 100 000 [. Both cases are tested.
  2. Test gaps:
    • New test: the job adopts a session edit while serving and audits it as minutes_per_run_changed (15 → 30).
    • Shutdown is now checked by counting write_performance calls across shutdown.
    • A fresh store must write with no WARNING. That test fails if the FileNotFoundError branch is dropped.
    • The unusable-file cases now each assert their own warning and that no minutes_per_run_changed entry was written. A too-deeply-nested case was added.
  3. A bad CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN is now a configuration error. It fails the boot with configuration error: ... in log.txt, like the other bad config.
  4. One shared check_target in connect/run_goal.py replaces the two checks. The env var parser and _stored_target both use it, and RunGoal.__init__/set_target raise on a non-target, so a negative target can no longer be set.
  5. Brief: the "Minutes per run" section is rewritten along the lines you suggested. The agent_performance.json bullet now points to that section, and the pearl-connect SKILL.md addition is down to one line.
  6. Comments and docs:
    • The run_goal.py docstring is one line, and the activity.py docstring points to the README.
    • The last paragraph of README "Run window" now links to the brief instead of restating its rules.
    • The comparison with Trader and Optimus is gone from the PR description and from scope D5 (valory-xyz/scopes, OPE-1801/technical/README.md).
    • The stale "rollover at now" is gone from the MUTATION line.

I also replied to the github-code-quality "Statement has no effect" alert on await refresher and resolved it. It is a false positive: the await lets the cancelled job finish.

Checks (pre-push verifier): flake8, pylint, black-check, isort-check, bandit, mypy, check-copyright and gitleaks pass. unit-tests-coverage reaches 100%. Its only failure is the sandbox-only test_bootstrap_refuses_when_there_is_no_workspace, which fails the same way on main. The new tests fail with their fixes reverted. Three tests that close gaps in behaviour this round did not change pass both with and without it. The PR description has the full GATE and MUTATION lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OjusWiZard
OjusWiZard previously approved these changes Oct 8, 2026
@OjusWiZard
OjusWiZard requested a review from bennyjo October 8, 2026 11:51

@bennyjo bennyjo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers from me; two small questions inline.

Comment thread README.md Outdated
Comment thread connect/activity.py Outdated
@OjusWiZard

Copy link
Copy Markdown
Member

@valory-coding-agent address the latest 2 review comments

valory-coding-agent and others added 2 commits October 8, 2026 12:46
The first write copies CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN into
agent_performance.json and the file wins after that, so a later template
default never reaches an existing install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A target the session changed while Connect was stopped was adopted on
the next boot as a silent restore, so it never reached the audit log.
Audit it as minutes_per_run_restored; a stored value equal to the
configured one is still not logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bennyjo
bennyjo previously approved these changes Oct 8, 2026
@valory-coding-agent

Copy link
Copy Markdown
Author

Addressed the latest 2 review comments, both from @bennyjo. Neither had a category tag. Each one has its own commit:

  • README.md:129 / connect/config.py (17d5f37): the docs now say that the first write stores CONNECTION_CONFIGS_CONFIG_MINUTES_PER_RUN in the file, and that the file's value wins from then on.
  • connect/activity.py:140 (d49b4aa): when a boot adopts a stored target that differs from the configured one, it is now audited as minutes_per_run_restored. That covers an edit the session made while Connect was stopped. README, CLAUDE.md and the PR body are updated.
    Skipped 0 comments. Left 0 [human-input] comments for human action. Both threads are resolved. Gates and the 100%-coverage unit tests pass; details are in the PR body under Verification.

@bennyjo
bennyjo self-requested a review October 8, 2026 12:59
@OjusWiZard
OjusWiZard merged commit 73aa840 into main Oct 9, 2026
25 checks passed
@OjusWiZard
OjusWiZard deleted the iasonrovis/ope-1801-agent-trading-after-having-earned-rewards branch October 9, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants