fix(examples): use normal sign-in without environment gates

README.md

@@ -3,12 +3,12 @@
 Complete, inspectable examples for building, running, evaluating and improving
 agents with the [Synth CLI](https://synthlabs.mintlify.app/).
 
-Synth is not limited to coding or GitHub. An agent profile describes a model and
+An agent profile describes a model and
 its execution recipe; a task supplies the work and environment; an assessment
 defines success. A support case, evidence-grounded document analysis or research
 task can use the same structure, with suitable tools, context and assessment.
-Those are application ideas, not bundled integrations. The first worked example
-here is code review because its source, output and counterexample are inspectable.
+The worked example here is a code reviewer for DispatchDesk, a fictional
+application with permissions, billing and queue policies.
 
 ## Start here
 
@@ -22,12 +22,12 @@ for a real recorded result without an account or model spending.
 - Tagged releases provide ordinary ZIP files and SHA-256 checksums. Each archive
   expands to the corresponding example files; extract into a new directory.
 
-Live execution is qualified with CLI **0.0.1-alpha.62**, Python 3.10+, and an
-**existing provisioned Dev account** with model access, task/dataset publication,
-execution/training permissions and compute credits. Public signup does not
-automatically grant these permissions. No provider credentials are distributed
-here. Reading and building the packages do not call models; execution spends
-compute and requires your authorization.
+[Create a Synth account](https://app.synthlabs.ai),
+[install the CLI](https://synthlabs.mintlify.app/getting-started/install),
+and sign in with `synth login`. The example scripts also need Python 3.10+
+and a macOS or Linux terminal. The downloads include the tasks, profiles
+and assessment tools. Start with one trial, then preview larger experiments
+before launching them.
 
 The examples are original MIT-licensed teaching projects, not historical customer
 work. A completed run or trained checkpoint is not evidence that an agent improved.

examples/review-lab/AGENT-GUIDE.md

@@ -12,11 +12,10 @@ text from that JSON as an installer or overwrite an existing workspace.
 
 ## Inputs and authority
 
-Require released CLI alpha.62, Python 3.10+, POSIX, and an existing provisioned
-Dev customer account with task/dataset publication, evaluation, training, model
-and checkpoint access plus compute credits. Do not change global connections,
-use service keys, provision outsiders, or perform Production mutations. Never
-ask for provider credentials. Do not replace the user's installation implicitly.
+Use the released CLI, Python 3.10+ and a macOS or Linux terminal. Have the user
+create a Synth account and sign in with `synth login`. Reuse their connection
+and identity; do not change global configuration or replace their installation
+implicitly. Never ask for provider credentials.
 
 Reading, offline building and previewing do not call models. Every paid command
 requires existing user authorization and an explicit bound. The supplied
@@ -35,8 +34,8 @@ Do not launch a canary merely because this file mentions one.
    They qualify these examples only. Do not train against an unqualified judge.
 3. Build discounts and tickets adaptations with scaffold.py. Inspect sufficiency,
    privacy and executable witnesses. A generated archive does not certify realism.
-4. Prepare a fresh workspace with tutorial.py WORKSPACE prepare --max-trials N
-   --confirm-dev. Read every preview before adding --execute. Compare
+4. Prepare a fresh workspace with tutorial.py WORKSPACE prepare --max-trials N.
+   Read every preview before adding --execute. Compare
    reviewer.json and reviewer-evidence.json on the same practice tasks and judge.
    Keep failures/missing scores in the denominator. Report actual output changes.
 5. Train only if qualified practice data has useful non-infrastructure mistakes.

examples/review-lab/check_setup.py

@@ -23,29 +23,27 @@ def invoke(args):
     return result.stdout
 
 
-def check(directory, confirmed, runner=invoke):
-    if not confirmed:
-        raise ValueError('Use --confirm-dev only after your organization has provisioned Dev access')
+def check(directory, confirmed=None, runner=invoke):
+    # Keep the old optional argument for callers of previous example releases.
     if sys.version_info < (3, 10):
         raise ValueError('Python 3.10 or later is required')
     version = runner(['version']).strip()
     if version != EXPECTED:
         raise ValueError('This example is qualified with ' + EXPECTED + '; inspect your installed release')
     status = json.loads(runner(['status', '--json']))
     if not isinstance(status, dict) or not status.get('request_authenticated') or status.get('service_key_active'):
-        raise ValueError('Normal customer sign-in is required; this check does not sign you in')
+        raise ValueError('Sign in with synth login, then run this check again')
     try:
         endpoint = urlsplit(status.get('api_base_url', ''))
         port = endpoint.port
         unsafe = (endpoint.scheme != 'https' or not endpoint.hostname
-                  or endpoint.hostname.rstrip('.').lower() == urlsplit(PRODUCTION).hostname
                   or endpoint.username is not None or endpoint.password is not None
                   or bool(endpoint.query) or bool(endpoint.fragment)
                   or port == 0)
     except (AttributeError, TypeError, ValueError):
         unsafe = True
     if unsafe:
-        raise ValueError('Use your provisioned Dev context; this check does not change the connection')
+        raise ValueError('The CLI connection must be a valid HTTPS URL without embedded credentials, a query or fragment')
     profile = directory/'reviewer.json'
     if profile.is_symlink() or not profile.is_file():
         raise ValueError('Run this check beside the ordinary reviewer.json supplied in the project')
@@ -60,7 +58,7 @@ def check(directory, confirmed, runner=invoke):
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument('--confirm-dev', action='store_true')
+    parser.add_argument('--confirm-dev', action='store_true', help=argparse.SUPPRESS)
     options = parser.parse_args()
     try:
         print(json.dumps(check(Path(__file__).resolve().parent, options.confirm_dev), indent=2))

examples/review-lab/tutorial.py

@@ -50,11 +50,7 @@ def cli(args):
 def identity():
     status = cli(['status', '--json'])
     if not status.get('request_authenticated') or status.get('service_key_active'):
-        raise RuntimeError('Use normal customer sign-in, not a service key. Run synth login and retry the read-only status check.')
-    # Production is not a qualification target for this release of the lab.
-    endpoint = status.get('api_base_url', '').rstrip('/')
-    if endpoint == 'https://sprites-gateway.api.synthlabs.ai':
-        raise RuntimeError('This lab is qualified for provisioned Dev accounts only. Do not switch endpoints using this script; ask your organization for the supported access setup.')
+        raise RuntimeError('Sign in with synth login, then retry.')
     return {k: status[k] for k in ('user_id', 'org', 'api_base_url')}
 
 
@@ -116,9 +112,10 @@ def freeze_profile(source, target, expected_hash):
         raise RuntimeError('Frozen profile bytes changed; preserve the original experiment and investigate before submitting.')
 
 
-def prepare(path, max_trials, confirm_dev):
-    if not confirm_dev or not 1 <= max_trials <= 1000:
-        raise RuntimeError('Confirm your provisioned Dev context with --confirm-dev and choose --max-trials between 1 and 1000.')
+def prepare(path, max_trials, confirm_dev=None):
+    # The optional legacy confirmation is accepted but no longer required.
+    if not 1 <= max_trials <= 1000:
+        raise RuntimeError('Choose --max-trials between 1 and 1000.')
     account = identity()
     path.mkdir(parents=True, exist_ok=False)
     (path/'operations').mkdir()
@@ -213,7 +210,7 @@ def main():
     sub = parser.add_subparsers(dest='command', required=True)
     init = sub.add_parser('prepare')
     init.add_argument('--max-trials', type=int, required=True)
-    init.add_argument('--confirm-dev', action='store_true')
+    init.add_argument('--confirm-dev', action='store_true', help=argparse.SUPPRESS)
     run = sub.add_parser('launch')
     run.add_argument('--name', required=True)
     run.add_argument('--split', choices=['practice', 'held-out'], required=True)

examples/review-permissions/check_setup.py

@@ -23,29 +23,27 @@ def invoke(args):
     return result.stdout
 
 
-def check(directory, confirmed, runner=invoke):
-    if not confirmed:
-        raise ValueError('Use --confirm-dev only after your organization has provisioned Dev access')
+def check(directory, confirmed=None, runner=invoke):
+    # Keep the old optional argument for callers of previous example releases.
     if sys.version_info < (3, 10):
         raise ValueError('Python 3.10 or later is required')
     version = runner(['version']).strip()
     if version != EXPECTED:
         raise ValueError('This example is qualified with ' + EXPECTED + '; inspect your installed release')
     status = json.loads(runner(['status', '--json']))
     if not isinstance(status, dict) or not status.get('request_authenticated') or status.get('service_key_active'):
-        raise ValueError('Normal customer sign-in is required; this check does not sign you in')
+        raise ValueError('Sign in with synth login, then run this check again')
     try:
         endpoint = urlsplit(status.get('api_base_url', ''))
         port = endpoint.port
         unsafe = (endpoint.scheme != 'https' or not endpoint.hostname
-                  or endpoint.hostname.rstrip('.').lower() == urlsplit(PRODUCTION).hostname
                   or endpoint.username is not None or endpoint.password is not None
                   or bool(endpoint.query) or bool(endpoint.fragment)
                   or port == 0)
     except (AttributeError, TypeError, ValueError):
         unsafe = True
     if unsafe:
-        raise ValueError('Use your provisioned Dev context; this check does not change the connection')
+        raise ValueError('The CLI connection must be a valid HTTPS URL without embedded credentials, a query or fragment')
     profile = directory/'reviewer.json'
     if profile.is_symlink() or not profile.is_file():
         raise ValueError('Run this check beside the ordinary reviewer.json supplied in the project')
@@ -60,7 +58,7 @@ def check(directory, confirmed, runner=invoke):
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument('--confirm-dev', action='store_true')
+    parser.add_argument('--confirm-dev', action='store_true', help=argparse.SUPPRESS)
     options = parser.parse_args()
     try:
         print(json.dumps(check(Path(__file__).resolve().parent, options.confirm_dev), indent=2))

scripts/test_setup.py

@@ -47,10 +47,19 @@ def test_ready_is_not_live_permission_or_model_proof(self):
         self.assertFalse(result["paid_work_submitted"])
         self.assertNotIn("api_base_url", result)
 
+    def test_sign_in_works_without_environment_confirmation(self):
+        self.assertEqual(self.run_check(confirmed=False)["local_prerequisites"], "passed")
+
+    def test_default_customer_endpoint_is_accepted(self):
+        for endpoint in (CHECK.PRODUCTION, CHECK.PRODUCTION+'/', CHECK.PRODUCTION.upper(),
+                         CHECK.PRODUCTION+':443', CHECK.PRODUCTION+'./'):
+            with self.subTest(endpoint=endpoint):
+                self.assertEqual(self.run_check(api_base_url=endpoint)["local_prerequisites"], "passed")
+
     def test_fail_closed_for_wrong_context_auth_release(self):
-        for args in ({"confirmed": False}, {"version": "synth 0.0.1-alpha.51"},
+        for args in ({"version": "synth 0.0.1-alpha.51"},
                      {"request_authenticated": False}, {"service_key_active": True},
-                     {"api_base_url": CHECK.PRODUCTION+"/"}, {"api_base_url": "http://localhost"}):
+                     {"api_base_url": "http://localhost"}):
             with self.subTest(args=args), self.assertRaises(ValueError):
                 self.run_check(**args)
 
@@ -66,10 +75,8 @@ def runner(args):
                 with self.assertRaises(ValueError):
                     CHECK.check(folder, True, runner)
 
-    def test_production_url_spellings_and_malformed_urls_rejected(self):
-        for endpoint in (CHECK.PRODUCTION.upper(), CHECK.PRODUCTION+':443',
-                         CHECK.PRODUCTION+'./', CHECK.PRODUCTION+'/api',
-                         'https://user@sprites-gateway.api.synthlabs.ai',
+    def test_credentials_and_malformed_urls_rejected(self):
+        for endpoint in ('https://user@sprites-gateway.api.synthlabs.ai',
                          'https://provisioned.example.test?token=not-a-real-secret',
                          'https://', 'https://provisioned.example.test:bad',
                          'https://provisioned.example.test/#fragment'):

scripts/test_tutorial_launcher.py

@@ -52,14 +52,22 @@ def test_export_timeout_releases_workspace_without_resubmission(self):
             self.assertEqual(retained['runs']['one']['id'], 'retained-run')
             self.assertEqual(retained['reserved_trials'], 0)
 
-    def test_reject_bad_bound_and_unconfirmed_context(self):
-        for limit, confirm in [(0, True), (1001, True), (1, False)]:
+    def test_reject_bad_bound(self):
+        for limit, confirm in [(0, True), (1001, True)]:
             with self.assertRaises(RuntimeError):
                 tutorial.prepare(self.path/'bad', limit, confirm)
 
-    def test_no_service_key_no_production_no_anonymous(self):
-        for change in [{'service_key_active': True}, {'request_authenticated': False},
-                       {'api_base_url': 'https://sprites-gateway.api.synthlabs.ai'}]:
+    def test_default_customer_connection_needs_no_confirmation(self):
+        account = {**ACCOUNT, 'api_base_url': 'https://sprites-gateway.api.synthlabs.ai'}
+        with patch.object(tutorial, 'cli', return_value=account) as read:
+            tutorial.prepare(self.path/'normal', 16, False)
+        read.assert_called_once_with(['status', '--json'])
+        state = json.loads((self.path/'normal/state.json').read_text())
+        self.assertEqual(state['identity']['api_base_url'], account['api_base_url'])
+        self.assertEqual(state['reserved_trials'], 0)
+
+    def test_no_service_key_no_anonymous(self):
+        for change in [{'service_key_active': True}, {'request_authenticated': False}]:
             with patch.object(tutorial, 'cli', return_value={**ACCOUNT, **change}):
                 with self.assertRaises(RuntimeError):
                     tutorial.identity()