diff --git a/CHANGELOG.md b/CHANGELOG.md index 03bba260..d24a71c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,10 @@ Work on this branch is **3.0** (M1–M3 plus the follow-on issues below). Linux - **ITSI search groups results**: `spa apps search itsi` in human mode groups hits into "Core ITSI" (1841), "Content packs (from Library 5391)", "Single content packs", and "Other apps". The library section shows the `content-packs list 5391` and `--content-pack NAME` commands, and each single pack shows its add command and that Core ITSI 1841 is added first when missing. Library 5391 is pinned for ITSI queries, so its row appears even when the page omits it. Agent output and non-ITSI searches are unchanged. +- **ITSI content pack help**: `spa apps search --help` now explains the ITSI path (`spa apps search itsi`, then `spa apps content-packs list 5391`, then `spa apps add 5391 --content-pack NAME`) and what each ITSI `--kind` means. The agent schema for `apps search` names the pack list and uses an ITSI example, and `apps content-packs list` says it lists ITSI content packs in Content Library 5391. Search rows still do not list packs inside 5391. + +- **Apps ask for roles**: `/spa-apps` and the `/spa-create-config` apps phase ask with AskQuestion which roles a normal app goes to, even with one host and one role. ITSI (1841) and content packs do not ask. The create-config app table now shows the real `--roles` flag. + - **terraform.aws rejects unknown keys**: a misspelled key such as `ami` next to `ami_id` now fails validation instead of being ignored. - **AWS discovery uses the environment region**: `spa aws` without `--region` searches `terraform.aws.region` from the environment config before `AWS_REGION`, `AWS_DEFAULT_REGION`, or `~/.aws/config`. The skills pass `--region` on every AMI and survey lookup. @@ -26,6 +30,10 @@ Work on this branch is **3.0** (M1–M3 plus the follow-on issues below). Linux - **Init needs a provider**: `spa init` and `spa environment init` require `--provider aws|virtualbox|external` unless a default is saved with `spa environment set --provider NAME`. With neither, init stops before it writes anything and names the three choices. Saving `aws` as the default now writes `providers.yml` (it used to delete the file), so an explicit `aws` counts as a choice. In agent mode, a failed init returns `ok` false and `error` only, with no `data`. +- **Hidden per-environment fact cache**: the Ansible fact cache for each environment moved from `$SPA_ENV_DIR/ansible_facts` to the hidden `$SPA_ENV_DIR/.ansible_facts`. An existing visible `ansible_facts` directory is left in place and is no longer read; the cache rebuilds on the next run. + +- **Baseconfig check looks for `org_s2_indexer_indexes`**: the Software check (`spa validate`, deploy preflight, and the Ansible inventory check) now requires `*/org_s2_indexer_indexes` and `*/org_cluster_manager_base`. A Software directory that has only `org_ds_secure_server` now fails the check. + ### Added - **Skills pack** ([#67](https://github.com/splunk/splunk-platform-automator/issues/67)): three operator skills (`spa`, `spa-create-config`, `spa-apps`). Agents ask in chat and wait before provision, deploy, destroy, suspend, resume, and environment remove. `spa validate --splunk-config aws` checks that provider from the environment config. App-scope scenario steps live in `tests/README.md`. @@ -79,7 +87,7 @@ Work on this branch is **3.0** (M1–M3 plus the follow-on issues below). Linux ### Changed -- **Per-environment Ansible fact cache**: `spa` sets `ANSIBLE_CACHE_PLUGIN_CONNECTION` to `$SPA_ENV_DIR/ansible_facts`, so two environments that reuse a hostname do not share facts. `/tmp/ansible_facts` remains the fallback when that variable is unset. `spa hosts check` and a successful `spa hosts register` drop that host's cached interpreter in the current environment before Ansible runs, so a previous machine's Python path is not reused. A missing remote interpreter is reported as a missing interpreter. Passwordless sudo is reported only when sudo itself failed. +- **Per-environment Ansible fact cache**: `spa` sets `ANSIBLE_CACHE_PLUGIN_CONNECTION` to `$SPA_ENV_DIR/.ansible_facts`, so two environments that reuse a hostname do not share facts. `/tmp/ansible_facts` remains the fallback when that variable is unset. `spa hosts check` and a successful `spa hosts register` drop that host's cached interpreter in the current environment before Ansible runs, so a previous machine's Python path is not reused. A missing remote interpreter is reported as a missing interpreter. Passwordless sudo is reported only when sudo itself failed. - **Test venv installs the config editor YAML library**: `tests/requirements.txt` includes `ruamel.yaml`, so a fresh CI environment can import the config editor. The operator venv already installed it from `requirements.txt`. @@ -129,7 +137,7 @@ Work on this branch is **3.0** (M1–M3 plus the follow-on issues below). Linux - **`spa venv`**: `spa venv path` prints the target; `create`, `reinstall`, `upgrade`, and `rebuild` manage the shared `SPA_HOME/.venv` (default) or `--environment` venv and require `--yes`. `--upgrade` runs `pip install --upgrade -r requirements.txt` and `ansible-galaxy collection install --upgrade`. `--shared` / `--environment` ignore a stale exported `SPA_VENV_DIR`. Progress streams (`Creating virtual environment...`, `Installing packages...` / `Upgrading packages...`, `Installing Ansible collections...` / `Updating Ansible collections...`); pip/galaxy logs stay hidden unless the command fails or `spa -v venv` is used. Confirmation is local (`will change the Python environment for this framework`; `spa venv rebuild` deletes and recreates it). `bin/spa` re-execs into a venv that can import `yaml` / `pydantic`. Commands that need those packages fail with the state of each candidate and the matching `spa venv` repair (`create` / `reinstall` / `rebuild`); `spa venv`, `doctor`, `init`, and `environment` stay available without a complete venv. `PyYAML` and `boto3` are in `requirements.txt`. -- **Main `/spa` skill and environment registry** ([#54](https://github.com/splunk/splunk-platform-automator/issues/54)): `skills/spa/spa/` (`name: spa`). Software checklist in the [user guide](docs/user-guide.md#prepare-software) (employee baseconfig download: [https://go2.cisco.com/baseconfigs](https://go2.cisco.com/baseconfigs)). `spa environment` (alias `env`) `list` / `init` / `set` / `remove`; `spa init` aliases `environment init`. Default env parent `~/Splunk-Platform-Automator`; `spa environment set --env-dir` writes `paths.yml` `env_dir` only when non-default; `--software-dir` / `--baseconfig-dir` / `--apps-dir` write the same file for the shared installer, PS baseconfig, and local-apps trees. Init fills any still-unset shared path from its explicit options or discovered directories without replacing established globals. `spa environment set --default NAME` selects the registered fallback. One-off `init --env-dir PARENT --name NAME` (same as a dest path) does not persist the parent. `--name` is the registry key. `spa env init NAME --force` adopts an existing lab into the list. Default provider is aws (`providers.yml` only when set to virtualbox). `spa` resolves `--env`, cwd, or the registered default and applies its environment internally. `spa env --export` remains for external tools. `spa environment remove` deletes the local env dir (`--yes`; `--force` if it looks deployed) and does not call `spa destroy`. A registered path that no longer exists is shown as `(missing)` and is ignored as the default. Test runs isolate `XDG_CONFIG_HOME` so they never write the operator's `~/.config/spa/environments.yml`. +- **Main `/spa` skill and environment registry** ([#54](https://github.com/splunk/splunk-platform-automator/issues/54)): `skills/spa/spa/` (`name: spa`). Software checklist in the [user guide](docs/user-guide.md#prepare-software) (employee baseconfig download). `spa environment` (alias `env`) `list` / `init` / `set` / `remove`; `spa init` aliases `environment init`. Default env parent `~/Splunk-Platform-Automator`; `spa environment set --env-dir` writes `paths.yml` `env_dir` only when non-default; `--software-dir` / `--baseconfig-dir` / `--apps-dir` write the same file for the shared installer, PS baseconfig, and local-apps trees. Init fills any still-unset shared path from its explicit options or discovered directories without replacing established globals. `spa environment set --default NAME` selects the registered fallback. One-off `init --env-dir PARENT --name NAME` (same as a dest path) does not persist the parent. `--name` is the registry key. `spa env init NAME --force` adopts an existing lab into the list. Default provider is aws (`providers.yml` only when set to virtualbox). `spa` resolves `--env`, cwd, or the registered default and applies its environment internally. `spa env --export` remains for external tools. `spa environment remove` deletes the local env dir (`--yes`; `--force` if it looks deployed) and does not call `spa destroy`. A registered path that no longer exists is shown as `(missing)` and is ignored as the default. Test runs isolate `XDG_CONFIG_HOME` so they never write the operator's `~/.config/spa/environments.yml`. - **Generated `docs/commands.md`**: `spa --no-agent agent schema --markdown` renders `COMMAND_SCHEMA` as a checked-in CLI catalog (not a second user guide). `--json` / agent mode still print the JSON envelope. Drift-tested against the committed file. diff --git a/README.md b/README.md index c4d247fb..343e10bc 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,34 @@ ![Python](https://img.shields.io/badge/Python-3.9%2B-blue.svg?logo=python&logoColor=white) ![Terraform](https://img.shields.io/badge/Terraform-1.3.0%2B-purple.svg?logo=terraform&logoColor=white) -SPA provisions and configures Splunk Enterprise—clustering, apps, AWS or VirtualBox—from **testing and training through staging and production**. One config file, one CLI: **`spa`**. The install follows Splunk best-practice config apps. There are many valid conf layouts; this is one automated path. +> **From architecture diagram to running Splunk cluster in less than an hour.** +> Splunk Platform Automator turns a single YAML file into a fully built, +> best-practice Splunk Enterprise environment: indexer clusters, multisite, +> search head clusters, forwarders, and the works. No hand-built hosts, no +> copy-pasting `.conf` files, no guessing at the bootstrap order. + +SPA provisions and configures Splunk Enterprise—clustering, apps, AWS, VirtualBox, or existing hosts (external provider)—from **testing and training through staging and production**. One config file, one CLI: `spa`. The install follows Splunk best-practice config apps. There are many valid conf layouts; this is one automated path. + +## Why SPA + +Splunk Platform Automator is the deployment engine for the Splunk Enterprise +topologies you designed. Describe the environment once, then let SPA build it +the same way every time. + +- **One file, the whole environment.** Describe your topology in +`splunk_config.yml` and let SPA handle the rest: hosts, cluster bootstrap, +and apps. +- **Ready-made topologies.** Examples cover single-node through multisite search +head clusters with Enterprise Security, so you start from a tested layout +instead of a blank page. +- **Your choice of infrastructure.** Provision on AWS with Terraform, spin up +local VirtualBox VMs, or deploy onto hosts you already have. +- **Best practices built in.** Base configuration apps, secrets from environment +lookups or Ansible Vault, and validation before anything is deployed. +- **Agent-ready.** `spa skills install cursor` links the `/spa`, +`/spa-create-config`, and `/spa-apps` skills into Cursor, so an agent can +design, validate, and deploy an environment with you, waiting for your yes at +each step. ## Support @@ -15,12 +42,19 @@ SPA provisions and configures Splunk Enterprise—clustering, apps, AWS or Virtu ## Features -- Reproducible Splunk Enterprise: cluster manager, indexer clusters, deployer, search head clusters, deployment server, forwarders, license manager, monitoring console -- One YAML definition (`splunk_config.yml`); topology examples composed at `spa init` -- **Apps:** Splunkbase, local, or URL via `splunk_app_deployment` — [Apps](docs/apps.md) -- **Secrets:** environment lookups or Ansible Vault — [Secrets](docs/secrets.md) -- **AWS:** Terraform from the same config (`spa provision` / `destroy`) -- **VirtualBox:** local VMs through `spa` (Vagrant only as an implementation detail) +- **Multiple environments:** `spa environment list`, `init`, `set`, and `remove` keep each lab or deployment in its own directory, with a default you can change. +- **Guided config editing:** `spa config set|add|merge|topology|provider` and `spa apps add|remove` make schema-checked edits with dry-run support, so you don't hand-edit `splunk_config.yml`. `spa config search` and `spa config explain` look up any setting. +- **Preflight checks:** `spa validate` checks the config, inventory, and license pairings. `spa doctor` checks host tools. A missing environment variable stops the run before anything changes. +- **Lifecycle control:** `spa suspend` and `spa resume` stop and start instances to save cost, and `spa destroy` removes the environment. Changing commands require an explicit `--yes`. +- **Run logs:** provision, deploy, and run write a log under the environment's `logs/` folder. `spa logs` replays any run, and live progress shows each step as it happens. +- **Playbook catalog:** `spa run list` shows the shipped playbooks, including rolling upgrades for indexer and search head clusters, plus your own. `spa run NAME --help` describes each one. +- **Verification:** verification playbooks confirm that deployed apps match the config. +- **Splunkbase and ITSI discovery:** `spa apps search` finds apps, `spa apps snippet` generates the config entry, and `spa apps content-packs list 5391` lists ITSI content packs and the command to add each one. +- **Existing hosts:** `spa init --provider external` with `spa hosts register` manages machines you already run. SPA does not create, stop, or terminate them. +- **Host access:** `spa hosts list --status`, `spa hosts ssh`, and `spa hosts copy` work with the hosts in your environment. +- **Shell completion:** `spa completion bash` and `spa completion zsh` add tab completion for commands, flags, and config paths. +- **Sizing guidance:** the `/spa-create-config` skill maps Splunk Validated Architectures to topologies and sizes replication and search factors. +- **Platform coverage:** Splunk 9 and 10 on Amazon Linux 2023, RHEL 10, and Ubuntu 24.04, on amd64 or arm64, with optional FIPS mode. ## Install @@ -28,6 +62,7 @@ Requires **Python 3.9+** with the `venv` module (`python3` on PATH). The install ```sh curl -fsSL https://github.com/splunk/splunk-platform-automator/releases/latest/download/install.sh | sh +spa skills install cursor ``` That installs the latest GitHub Release into `~/.local/share/spa` and puts `spa` on `PATH` via `~/.local/bin/spa`. Add `~/.local/bin` to `PATH` if `spa` is not found. Flags, uninstall, and clone/develop: [docs/install.md](docs/install.md). Put Splunk installers and PS baseconfig apps in a shared `Software/` directory ([Prepare Software](docs/user-guide.md#prepare-software)). @@ -44,6 +79,8 @@ spa provision --yes && spa deploy --yes VirtualBox: `--provider virtualbox` (for example `single_node`). Full narrative: [user guide](docs/user-guide.md). +Existing servers: `spa init --example TOPOLOGY --provider external ENV`, then `spa hosts register NAME --ssh-host HOST --user USER --key PATH --yes`. + ## Docs | Audience | Link | diff --git a/ansible.cfg b/ansible.cfg index 5c3e0e12..5ae9ca9e 100644 --- a/ansible.cfg +++ b/ansible.cfg @@ -18,7 +18,7 @@ forks = 20 gathering = smart fact_caching = jsonfile # Fallback when spa has not set ANSIBLE_CACHE_PLUGIN_CONNECTION. -# spa points that variable at $SPA_ENV_DIR/ansible_facts. +# spa points that variable at $SPA_ENV_DIR/.ansible_facts. fact_caching_connection = /tmp/ansible_facts fact_caching_timeout = 86400 diff --git a/ansible/group_vars/all/ansible.yml b/ansible/group_vars/all/ansible.yml index d6f56078..0892c798 100644 --- a/ansible/group_vars/all/ansible.yml +++ b/ansible/group_vars/all/ansible.yml @@ -2,7 +2,7 @@ # Ansible runtime settings used by SPA playbooks (see also ansible.cfg). # jsonfile fact cache. spa sets ANSIBLE_CACHE_PLUGIN_CONNECTION to -# $SPA_ENV_DIR/ansible_facts so two environments can reuse a hostname. +# $SPA_ENV_DIR/.ansible_facts so two environments can reuse a hostname. # The /tmp path is only the ansible.cfg fallback when that variable is unset. spa_ansible_fact_cache_dir: "{{ lookup('env', 'ANSIBLE_CACHE_PLUGIN_CONNECTION') | default('/tmp/ansible_facts', true) }}" diff --git a/ansible/plugins/inventory/splunk-platform-automator.py b/ansible/plugins/inventory/splunk-platform-automator.py index fb74cab6..f1b3aee1 100644 --- a/ansible/plugins/inventory/splunk-platform-automator.py +++ b/ansible/plugins/inventory/splunk-platform-automator.py @@ -342,7 +342,7 @@ def _populate(self): splunk_baseconfig_dir = self._resolve_override_dir( lab_dir, self.groups['all']['splunk_baseconfig_dir'], 'SPA_BASECONFIG_DIR' ) - check_base = glob.glob(os.path.join(splunk_baseconfig_dir, "*/org_ds_secure_server")) + check_base = glob.glob(os.path.join(splunk_baseconfig_dir, "*/org_s2_indexer_indexes")) check_cluster = glob.glob(os.path.join(splunk_baseconfig_dir, "*/org_cluster_manager_base")) if len(check_base) < 1 or len(check_cluster) < 1: lab_sw = os.path.normpath(os.path.join(lab_dir, '..', 'Software')) diff --git a/docs/commands.md b/docs/commands.md index cbee9998..843172f3 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -926,7 +926,7 @@ spa --agent apps search unix ## `apps search` -Search Splunkbase (compact rows: app_id, name, title, type, kind, version, summary) +Search Splunkbase (compact rows: app_id, name, title, type, kind, version, summary). An itsi query lists Core ITSI 1841 and Content Library 5391 first. ITSI content packs are not in search rows: list the packs in 5391 with spa apps content-packs list 5391. | Flag | Description | | --- | --- | @@ -937,12 +937,12 @@ Search Splunkbase (compact rows: app_id, name, title, type, kind, version, summa Example: ```bash -spa --agent apps search unix +spa --agent apps search itsi ``` ## `apps content-packs list` -Inspect Content Library 5391 from --archive, apps_dir, an existing install archive, or cached pack app.conf files. An install archive is left in place. --version selects the release (default latest). +List ITSI content packs in Content Library 5391 from --archive, apps_dir, an existing install archive, or cached pack app.conf files. An install archive is left in place. --version selects the release (default latest). | Flag | Description | | --- | --- | diff --git a/docs/user-guide.md b/docs/user-guide.md index daec5d51..31b34ea7 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -1,6 +1,6 @@ # Splunk Platform Automator user guide -SPA installs and configures Splunk Enterprise for testing, training, staging, and production. The operator interface is **`spa`**. Ansible, Terraform, and Vagrant are implementation details, not alternate operator CLIs. +SPA installs and configures Splunk Enterprise on AWS, VirtualBox, or existing hosts for testing, training, staging, and production. The operator interface is **`spa`**. Ansible, Terraform, and Vagrant are implementation details, not alternate operator CLIs. ## Start here @@ -50,7 +50,7 @@ spa init --software-dir ~/Software --example cm_2idxc_sh --provider aws my-lab - [Splunk Enterprise](https://www.splunk.com/en_us/download/splunk-enterprise.html) - [Splunk Universal Forwarder](https://www.splunk.com/en_us/download/universal-forwarder.html) - Filenames look like `splunk---linux-amd64.tgz` and `splunkforwarder---linux-amd64.tgz`. The version must match `splunk_version` in config. -3. Copy Splunk **Professional Services Best Practices baseconfig apps** into the same tree and extract them. They are not a public Splunk.com download. Splunk / Cisco employees can download them from [https://go2.cisco.com/baseconfigs](https://go2.cisco.com/baseconfigs). Everyone else: contact Splunk Professional Services. Typical layout: `Configurations - Base` and `Configurations - Index Replication`. +3. Copy Splunk **Professional Services Best Practices baseconfig apps** into the same tree and extract them. They are not a public Splunk.com download. Splunk / Cisco employees can download them from [http://go/baseconfigs](http://go/baseconfigs). Everyone else: contact Splunk Professional Services. Typical layout: `Configurations - Base` and `Configurations - Index Replication`. 4. If you have a Splunk license, place or symlink it as **`Splunk_Enterprise.lic`** in that directory (`spa licenses` lists basenames for config). Trial-only labs can omit licenses. ```text diff --git a/lib/spa/agent.py b/lib/spa/agent.py index 7b42c2bc..dcd1d001 100644 --- a/lib/spa/agent.py +++ b/lib/spa/agent.py @@ -812,7 +812,7 @@ def clear_envelope_identity() -> None: }, { "name": "apps search", - "summary": "Search Splunkbase (compact rows: app_id, name, title, type, kind, version, summary)", + "summary": "Search Splunkbase (compact rows: app_id, name, title, type, kind, version, summary). An itsi query lists Core ITSI 1841 and Content Library 5391 first. ITSI content packs are not in search rows: list the packs in 5391 with spa apps content-packs list 5391.", "flags": [ {"long": "--limit", "help": "Max hits (default 10)"}, { @@ -834,11 +834,11 @@ def clear_envelope_identity() -> None: ], }, ], - "example": "spa --agent apps search unix", + "example": "spa --agent apps search itsi", }, { "name": "apps content-packs list", - "summary": "Inspect Content Library 5391 from --archive, apps_dir, an existing install archive, or cached pack app.conf files. An install archive is left in place. --version selects the release (default latest).", + "summary": "List ITSI content packs in Content Library 5391 from --archive, apps_dir, an existing install archive, or cached pack app.conf files. An install archive is left in place. --version selects the release (default latest).", "flags": [ { "long": "--version", diff --git a/lib/spa/cli.py b/lib/spa/cli.py index 16f9e71e..1ebb9274 100644 --- a/lib/spa/cli.py +++ b/lib/spa/cli.py @@ -613,6 +613,16 @@ def _print_completion_candidates(session, extra: Sequence[str]) -> int: "spa apps content-packs needs an action.\n" "List: spa apps content-packs list 5391" ) +APPS_SEARCH_EPILOG = """\ +ITSI content packs: + spa apps search itsi Core ITSI 1841 and Content Library 5391 come first + spa apps content-packs list 5391 List the packs inside Content Library 5391 + spa apps add 5391 --content-pack NAME Add one pack + +Kinds: + itsi_content_library Content Library 5391; its packs come from content-packs list + itsi_content_pack_single One pack with its own Splunkbase app ID; no --roles +""" APPS_ADD_DESCRIPTION = """\ Idempotently add or update one apps[] entry. This command edits splunk_config.yml but does not deploy software. @@ -1573,7 +1583,14 @@ def _config_mutation_flags(config_parser, *, explicit_default=False): ) _add_env_selector(p_apps) apps_sub = p_apps.add_subparsers(dest="apps_cmd", metavar="ACTION") - p_apps_search = _add_action(apps_sub, "search", aliases=["sea"], help="Search Splunkbase (compact rows)") + p_apps_search = _add_action( + apps_sub, + "search", + aliases=["sea"], + help="Search Splunkbase (compact rows)", + epilog=APPS_SEARCH_EPILOG, + formatter_class=argparse.RawDescriptionHelpFormatter, + ) _add_mode_flags(p_apps_search) p_apps_search.add_argument("query", nargs="+", help="Technology or app name keywords") p_apps_search.add_argument( @@ -1598,7 +1615,7 @@ def _config_mutation_flags(config_parser, *, explicit_default=False): apps_sub, "content-packs", aliases=["cp"], - help="Discover packs bundled in an ITSI Content Library archive", + help="Discover ITSI content packs in Content Library 5391", epilog=CONTENT_PACKS_NEEDS_ACTION, formatter_class=argparse.RawDescriptionHelpFormatter, ) diff --git a/lib/spa/paths.py b/lib/spa/paths.py index 7cf1c557..f41c1860 100644 --- a/lib/spa/paths.py +++ b/lib/spa/paths.py @@ -356,7 +356,7 @@ def ansible_inventory(self) -> str: def fact_cache_dir(self) -> Path: """Jsonfile fact cache for this environment. Hostnames are not unique across envs.""" - return self.spa_env_dir / "ansible_facts" + return self.spa_env_dir / ".ansible_facts" def export_env(self) -> Dict[str, str]: """Environment overlay for validate, spa shell, and ansible-playbook.""" diff --git a/lib/spa/preflight.py b/lib/spa/preflight.py index ca4e716d..a25a7bc5 100644 --- a/lib/spa/preflight.py +++ b/lib/spa/preflight.py @@ -11,7 +11,7 @@ from spa.paths import DEFAULT_APPS_REL, DEFAULT_SOFTWARE_REL, SpaPaths, parse_flat_yaml, resolve_shared_data_dir -BASECONFIG_MARKERS = ("org_ds_secure_server", "org_cluster_manager_base") +BASECONFIG_MARKERS = ("org_s2_indexer_indexes", "org_cluster_manager_base") _HINT_SOFTWARE = "Point once with: spa init --software-dir DIR ENV" _HINT_APPS = "Point once with: spa init --apps-dir DIR ENV" _DEFAULTS = { @@ -186,7 +186,7 @@ def check_controller_data( ok=False, reason=( "Cannot find Splunk baseconfig apps in %s " - "(need */org_ds_secure_server and */org_cluster_manager_base)." + "(need */org_s2_indexer_indexes and */org_cluster_manager_base)." % baseconfig_dir ), hint=_HINT_SOFTWARE, diff --git a/skills/spa/spa-apps/SKILL.md b/skills/spa/spa-apps/SKILL.md index 63a086ff..5d008c48 100644 --- a/skills/spa/spa-apps/SKILL.md +++ b/skills/spa/spa-apps/SKILL.md @@ -50,9 +50,8 @@ Re-run the set / not-set loop after the operator says it is done. 1. `spa --agent apps search QUERY` with technology keywords. Optional `--type app|addon` and `--kind ta|premium_itsi|itsi_content_library|itsi_content_pack_single|es_not_premium`. An `itsi` query pins **1841** first; `es` pins **263**. Do **not** open splunkbase.com first. 2. Pick one `app_id` from the compact rows (`app_id`, `name`, `title`, `type`, `kind`, `version`, `summary`). No full description on search. Default 10 hits; tighten QUERY if the page is wrong. 3. For a normal app, use one transaction: `spa --agent apps add APP_ID --roles ROLE[,ROLE]`. Add schema-backed fields with repeatable `--set FIELD VALUE`; lists and maps use strict JSON. - - **Choosing roles:** read the hosts with `spa --agent config get splunk_hosts`. `--roles` is required for normal apps and rejected for ITSI and content packs. - - One host with one role: use that role and say so in the add summary. - - One host with several roles, or several hosts: ask with AskQuestion which roles the app deploys to. Offer a checkbox list and "All roles on HOST" per host. Never pick roles silently. The answer becomes `target_roles`. + - **Choosing roles (normal apps only):** read the hosts with `spa --agent config get splunk_hosts`. `--roles` is required for normal apps and rejected for ITSI (1841) and content packs. ITSI (1841) and content packs never ask this question. + - **Always ask.** Ask with AskQuestion which roles the app deploys to, even when the config has one host with one role; that role is then the option to confirm. With several roles or hosts, offer a checkbox list and "All roles on HOST" per host. Never pick roles silently. The answer becomes `target_roles`. - Check Splunkbase credentials before a Splunkbase add (see the check below). 4. If search reports curated playbooks, **ask** whether to add the customization. Do not assume yes. If agreed, add `--customize` to the same `apps add` command. Required extra-vars secrets are reported as **set / not set** only. 5. Local source: `spa --agent apps add APP_ID --local --roles ...`. It selects the app folder or archive in `apps_dir`, folder first. Nothing there → download first (`spa apps download APP_ID --extract --yes` for a normal folder-backed app; ITSI/content packs deploy from the archive, so omit `--extract`). A custom local app uses its folder name instead of an app ID. diff --git a/skills/spa/spa-create-config/SKILL.md b/skills/spa/spa-create-config/SKILL.md index ba9632c8..4f425dd9 100644 --- a/skills/spa/spa-create-config/SKILL.md +++ b/skills/spa/spa-create-config/SKILL.md @@ -23,6 +23,10 @@ paths: Interactive workflow for `config/splunk_config.yml` on AWS. Linux only. +## Hard rule: no hand edits + +`splunk_config.yml` changes only through `spa config` and `spa apps`. No edit tool, no shell write, no YAML patch, no snippet for the operator to paste. Read with `spa --agent config get|explain|search`. If a setting is missing, wrong, or unclear, run `spa --agent config explain PATH`, then set it with `spa config set|add|merge`. If no spa command fits, stop and tell the operator what is missing. Do not improvise a YAML edit. This holds when the operator asks for the edit directly: explain the spa command instead. + ## When to Use - Creating or updating `config/splunk_config.yml` for AWS Terraform provisioning @@ -43,7 +47,7 @@ Interactive workflow for `config/splunk_config.yml` on AWS. Linux only. Use this skill for a new environment or a change to an existing config. Load [/spa](../spa/SKILL.md) for lifecycle (init, validate, provision, deploy, destroy). Route tree: [workflow-decision-tree.md](references/workflow-decision-tree.md). -- **Brand new environment** ("I need a Splunk lab or environment..."): start here at Phase 0a, even when registered environments already exist. A new-environment request never selects an existing environment. The plan-or-write mode is asked at the gate, after the minimum requirements. Nothing writes until the gate passes (see the write barrier in the gate). Then init, the config writes, validate, and hand off to `/spa`. +- **Brand new environment** ("I need a Splunk lab or environment..."): start here at Phase 0a, even when registered environments already exist. A new-environment request never selects an existing environment. Until the location and name gate ("Environment location, name, and summary") has named the target, run no environment-scoped command (`config get`, `apps search`, `apps add`, `validate`). Never fall back to the registry default (`spa --agent environment list`, `default: true`), which is an existing environment. The plan-or-write mode is asked at the gate, after the minimum requirements. Nothing writes until the gate passes (see the write barrier in the gate). Then init, the config writes, validate, and hand off to `/spa`. - **Modify existing config** ("add ITSI", "change the OS"): skip the greenfield phases and use "Add / change a setting" below. Name the target environment before any write. If the operator did not name one, ask which registered environment to change (AskQuestion), and show its path. - **Lifecycle** ("destroy", "status", "deploy", "logs"): load `/spa` directly. No design is needed. - **Apps** ("add a TA", "install a content pack"): load `/spa-apps`. @@ -53,14 +57,33 @@ The design phases exist so the first `--example` template is not accepted blindl **Provider:** `spa init` requires `--provider aws|virtualbox|external` unless the operator has saved a default with `spa environment set --provider NAME`. Ask the operator which provider to use before any init, then record it with `--aws` in the header when it is AWS. See [config-header-best-practices.md](references/config-header-best-practices.md). ## Never edit by hand (mandatory) - -Never edit `config/splunk_config.yml` yourself: no editor, no shell write, no YAML patch, no pasted block. Every change goes through `spa config`: + +Never edit `config/splunk_config.yml` yourself: no editor, no shell write, no YAML patch, no pasted block, and no edit tool on any `splunk_config.yml` path (including `$SPA_ENV_DIR` and any environment outside this repo, such as `~/labs/NAME`). Shell writes include `sed -i`, `perl -i`, `yq -i`, `tee`, `>`/`>>` redirects, `cp`/`mv` onto the file, `python`/`node` file writes, and **any tool that writes a file directly** (including cursor tools like `Write`, editing the file in an IDE, etc.). This holds even when a change looks pending, unfinished, or missing from the config. spa has no pending-operation state that the agent may finish by hand. If a spa write fails, report the error and stop. If a hand edit already happened, stop, tell the operator which keys changed, and fix them only with `spa config` or `spa apps`. Every change goes through `spa config`: - Read: `spa --agent config search QUERY`, `spa --agent config explain PATH`, `spa --agent config get [PATH]` - Write: `spa --agent config set PATH VALUE`, `add`, `unset`, `merge --file FILE`, `topology`, `provider`, and `header set` App entries go to [spa-apps](../spa-apps/SKILL.md) (`spa --agent apps add`). Validation, provision, deploy, and destroy stay in [/spa](../spa/SKILL.md), which waits for your yes. +## Phase Sequencing & Guardrails (mandatory) + +Phases must be completed in strict order: **0a → 0b → 1 → 2 → 3 → gate → init → 4 → 5 → 5b → 6 → 6b → 6c (plan) or write (7) → 8 → 8b → 9**. + +- **Before the gate:** Phases 0a, 0b, 1, 2, 3 must be completed and user must explicitly approve before proceeding to the environment location gate. +- **No environment-scoped commands before init:** Do not run `spa init`, `spa --env NAME`, `spa config set`, `spa config add`, `spa apps add`, or `spa validate` until the gate has been passed and init is confirmed. +- **Architecture file ingestion (Phase 5b):** When an architecture-input JSON file is provided (e.g., from the SVA skill), always read it first, validate it against the architecture-input.md contract, and use it to populate answers in phases 1, 5, and 5b. Merge its suggested values into the session state before asking follow-up questions. Do not skip to init without first acknowledging the file. +- **Phase 6 (apps) is mandatory:** Always run Phase 6, even when the user did not mention apps. Always ask "Do you want to add apps?" with two clear options: "Yes, add apps" or "No, trial-only". Record the choice and proceed accordingly. +- **Content packs follow questionnaire:** When ITSI (app 1841) is selected, always run `spa apps content-packs list 1841` and ask with AskQuestion which packs to add. Do not guess pack names or IDs; always search first. +- **No spa config writes until Phase 7:** In plan mode, record all decisions but run no `spa config set`, `add`, `merge`, or `apps add` until the plan is approved and Phase 7 begins. + +**Guardrail enforcement for the agent:** +1. If you complete Phase 0a/0b but user asks to proceed to init before phases 1-3 are done, respond: "Phases 1 (topology), 2 (sizing), and 3 (OS) must be completed first. Let me ask..." Then ask them. +2. If a user tries to modify the config with a file editor, IDE, or manual YAML, stop immediately and say: "I detected a hand edit to `splunk_config.yml`. This violates the SPA hard rule. Let me redo this with the correct `spa config` command instead." Then recover using `spa config get` to see the current state and `spa config set/add/unset` to make the change. +3. If you are about to write the config file directly (using Write, shell redirects, etc.), stop and ask: "Should I use `spa config set/add/merge` instead?" +4. If a phase is skipped (e.g., jumping from Phase 0 to init), backtrack explicitly: "I skipped phases X, Y, Z. Let me go back and collect those decisions properly before proceeding." + +Command table for every config change: [config-commands.md](references/config-commands.md). Add an app: `spa --agent apps search QUERY` then `spa --env NAME apps add APP_ID --roles ROLE1,ROLE2 --yes`. **Never use:** Direct Write tool, shell redirects (`>`, `>>`), `sed -i`, `yq -i`, `python -c '...' > file`, etc. on `splunk_config.yml`. + ## Secrets — never display credential values (mandatory) Follow [secrets-handling.md](references/secrets-handling.md) for the full list. @@ -293,12 +316,12 @@ Summarize hosts before YAML write. Always ask, even when the user did not mention apps. Use AskQuestion if available, otherwise ask in chat: -- **Yes, add apps:** follow [apps-questionnaire.md](references/apps-questionnaire.md) and load [/spa-apps](../spa-apps/SKILL.md) to search and add each app. +- **Yes, add apps:** ask the [apps-questionnaire.md](references/apps-questionnaire.md) sections in order, one AskQuestion each, using [/spa-apps](../spa-apps/SKILL.md): (a) Premium ITSI (section 5); (b) ITSI content packs (section 6), only if ITSI is yes; (c) technical add-ons (section 7): ask which technologies, such as Windows, Linux, Cisco, or AWS, then run `spa --agent apps search TECHNOLOGY` for each and offer only the apps those searches return; (d) local org apps (section 8). Never offer an app from memory or with a generic query such as "common". Each `spa --agent apps add` uses the `app_id` from a search result. - **No, trial-only:** reconfirm once: "Confirmed: no apps for this environment. Continue?" Leave out `splunk_app_deployment` only after that yes. **Loop (when the answer is yes):** -1. Search and add one app with [spa-apps](../spa-apps/SKILL.md) (search, pick the ID and roles, `spa --agent apps add`). +1. Search and add one app with [spa-apps](../spa-apps/SKILL.md). For a normal app, ask with AskQuestion which roles it goes to before `spa --agent apps add`. ITSI and content packs have no roles question. 1b. **ITSI (1841) first.** If the app just added is ITSI (1841), ask about content packs before the next question. Run `spa --agent apps content-packs list 5391`, ask with AskQuestion which packs to add (include "None"), and add each chosen pack with `spa --agent apps add 5391 --content-pack NAME`. Then go to step 2. 2. After each add, ask with AskQuestion: **"Add another app?"** Options: Yes, add another; No, done with apps. 3. On yes, repeat from step 1 until the operator answers no. Do not skip the question after any add. @@ -438,9 +461,9 @@ Do not hand off until validation passes. See [validation.md](references/validati ## Phase 9 — Handoff -Validation passed. Ask which way to continue (AskQuestion if available): +Validation passed. **Always** ask which way to continue with an AskQuestion call, even when AskQuestion is not the usual choice. Never end with only a message such as "Ready for deployment, use /spa": the operator must be able to pick the handoff in the question. Put A first and mark it "(Recommended)": -- **A) Hand off to `/spa`:** load [/spa](../spa/SKILL.md) and run provision, then deploy, under its gates ("Handoff from spa-create-config" there). This skill runs no lifecycle command. +- **A) Hand off to `/spa` now (Recommended):** load [/spa](../spa/SKILL.md) and run provision, then deploy, under its gates ("Handoff from spa-create-config" there). This skill runs no lifecycle command. Asking here does not approve provision or deploy; `/spa` asks for those yeses itself. - **B) Show me the command:** print the commands below; you run them. - **C) Wait:** stop here; you run them later. @@ -454,18 +477,4 @@ Each run waits for its own explicit yes in chat. A request earlier in the chat i App-scope scenarios: [tests/README.md](../../../tests/README.md). -## Terminology - -Use consistently: `cluster_manager`, `terraform.aws`, `splunk_hosts`, `plugin: splunk-platform-automator` (not "master"). - -## Skill quality checklist - -- [ ] `name` matches folder `spa-create-config` -- [ ] Description third person with trigger terms -- [ ] SKILL.md under 500 lines; details in `references/` -- [ ] When to Use / When NOT to Use present -- [ ] Mode (`plan` vs `write`) asked at the gate after the minimum requirements; plan mode skips YAML until approval -- [ ] Step 0 and phase exit criteria followed -- [ ] `bin/*` invoked from project root -- [ ] Secrets: Splunkbase and AWS credential env values never shown in chat or terminal output -- [ ] Incremental add uses `spa --agent config search` then `spa --agent config explain`, then `spa --agent config set`, `spa --agent config add`, or [spa-apps](../spa-apps/SKILL.md), followed by `spa --agent validate` +Skill quality checklist: [skill-quality-checklist.md](references/skill-quality-checklist.md) diff --git a/skills/spa/spa-create-config/references/apps-questionnaire.md b/skills/spa/spa-create-config/references/apps-questionnaire.md index 2a579653..f3dbd683 100644 --- a/skills/spa/spa-create-config/references/apps-questionnaire.md +++ b/skills/spa/spa-create-config/references/apps-questionnaire.md @@ -31,9 +31,9 @@ See [docs/apps.md](../../../../docs/apps.md). Optional vault in config. Do not v ## 4. Target roles -For **normal** apps, `target_roles`: `search_head`, `indexer`, `universal_forwarder`, `heavy_forwarder` (maps to deployer/CM/DS/direct per SPA). Pass `--roles` to `apps add`. +For **normal** apps, `target_roles`: `search_head`, `indexer`, `universal_forwarder`, `heavy_forwarder` (maps to deployer/CM/DS/direct per SPA). Before `apps add` for a normal app, ask with AskQuestion which roles it goes to, even with one host and one role. Pass `--roles` to `apps add`. -Do **not** set `target_roles` on `premium_app: itsi` or `itsi_content_pack: true` entries. +Do **not** set `target_roles` on `premium_app: itsi` or `itsi_content_pack: true` entries. Do not ask for roles on those entries. ## 5. Premium ITSI? diff --git a/skills/spa/spa-create-config/references/config-commands.md b/skills/spa/spa-create-config/references/config-commands.md new file mode 100644 index 00000000..39d447f2 --- /dev/null +++ b/skills/spa/spa-create-config/references/config-commands.md @@ -0,0 +1,21 @@ +# Config change commands + +Reference for `spa-create-config`. Use these commands for every config change. Never hand-edit `splunk_config.yml`. + +Always use these commands, never hand-edit: +| Change | Correct Command | Why | +|--------|-----------------|-----| +| Set a scalar (region, instance type, OS) | `spa --env NAME config set PATH VALUE --yes` | Schema validation, audit trail | +| Add a host | `spa --env NAME config add 'splunk_hosts[name=HOSTNAME]' '{"roles":[...], ...}' --yes` | SPA validates role constraints, cluster membership, site mapping | +| Add an indexer cluster | `spa --env NAME config set splunk_idxclusters[idxc_name=idxc1].idxc_rf 3 --yes` | Constraints checked against peer count | +| Add an app | `spa --agent apps search QUERY` then `spa --env NAME apps add APP_ID --roles ROLE1,ROLE2 --yes` | Validates app exists, role exists, app not duplicate | +| Add ITSI content pack | `spa --agent apps content-packs list 1841` then `spa --env NAME apps add 1841 --content-pack CP_NAME --yes` | Pack name validated against ITSI manifest | +| Change OS | `spa --env NAME config merge --file -` with YAML input | Ensures os.ssh_username, disable_apparmor are set together | +| Merge multiple settings | `spa --env NAME config merge --file FILE.yml --yes` | Atomic, all-or-nothing; validation happens once | +| Read current state | `spa --env NAME config get [PATH]` | Always check before assuming a value | +| Search for a path | `spa --env NAME config search KEYWORD` | Finds related config paths | +| Explain a path | `spa --env NAME config explain PATH` | Shows type, constraints, examples, related commands | + +## Terminology + +Use consistently: `cluster_manager`, `terraform.aws`, `splunk_hosts`, `plugin: splunk-platform-automator` (not "master"). diff --git a/skills/spa/spa-create-config/references/skill-quality-checklist.md b/skills/spa/spa-create-config/references/skill-quality-checklist.md new file mode 100644 index 00000000..1be469c2 --- /dev/null +++ b/skills/spa/spa-create-config/references/skill-quality-checklist.md @@ -0,0 +1,11 @@ +# Skill quality checklist + +- [ ] `name` matches folder `spa-create-config` +- [ ] Description third person with trigger terms +- [ ] SKILL.md under 500 lines; details in `references/` +- [ ] When to Use / When NOT to Use present +- [ ] Mode (`plan` vs `write`) asked at the gate after the minimum requirements; plan mode skips YAML until approval +- [ ] Step 0 and phase exit criteria followed +- [ ] `bin/*` invoked from project root +- [ ] Secrets: Splunkbase and AWS credential env values never shown in chat or terminal output +- [ ] Incremental add uses `spa --agent config search` then `spa --agent config explain`, then `spa --agent config set`, `spa --agent config add`, or [spa-apps](../../spa-apps/SKILL.md), followed by `spa --agent validate` diff --git a/skills/spa/spa/SKILL.md b/skills/spa/spa/SKILL.md index 019760f3..0ccb0934 100644 --- a/skills/spa/spa/SKILL.md +++ b/skills/spa/spa/SKILL.md @@ -74,7 +74,7 @@ spa environment cleanup # lists registry entries ## Software -One shared directory per machine, not inside `SPA_HOME` and not copied per env. Linux tgz installers, extracted PS baseconfig apps, optional `Splunk_Enterprise.lic`. Splunk / Cisco employees: [https://go2.cisco.com/baseconfigs](https://go2.cisco.com/baseconfigs). Others: Splunk PS. Checklist: [Prepare Software](../../../docs/user-guide.md#prepare-software). +One shared directory per machine, not inside `SPA_HOME` and not copied per env. Linux tgz installers, extracted PS baseconfig apps, optional `Splunk_Enterprise.lic`. Splunk / Cisco employees: [http://go/baseconfigs](http://go/baseconfigs). Others: Splunk PS. Checklist: [Prepare Software](../../../docs/user-guide.md#prepare-software). ```bash spa environment set --software-dir ~/Software --apps-dir ~/labs/apps diff --git a/tests/spa_testutil.py b/tests/spa_testutil.py index 7ec0adc4..e3e80193 100644 --- a/tests/spa_testutil.py +++ b/tests/spa_testutil.py @@ -116,7 +116,7 @@ def min_env_vars(root: Path) -> dict: def seed_software_dir(root: Path) -> Path: """Minimal PS baseconfig layout so controller-data checks pass in tests.""" software = Path(root) / "Software" - for name in ("org_ds_secure_server", "org_cluster_manager_base"): + for name in ("org_s2_indexer_indexes", "org_cluster_manager_base"): (software / "ps" / name).mkdir(parents=True, exist_ok=True) return software diff --git a/tests/test_app_scope_scenarios.py b/tests/test_app_scope_scenarios.py index 6795d5b0..f1d6a6c8 100644 --- a/tests/test_app_scope_scenarios.py +++ b/tests/test_app_scope_scenarios.py @@ -84,7 +84,7 @@ def _ensure_software_stubs(stub_dir: Path) -> None: Real Splunk tarballs are gitignored (*.tgz) and are not in CI. Empty placeholders are enough for existence checks during debug_app_scope.yml. """ - (stub_dir / "spa_ci_stub" / "org_ds_secure_server").mkdir(parents=True, exist_ok=True) + (stub_dir / "spa_ci_stub" / "org_s2_indexer_indexes").mkdir(parents=True, exist_ok=True) (stub_dir / "spa_ci_stub" / "org_cluster_manager_base").mkdir(parents=True, exist_ok=True) for name in ( "Splunk_Enterprise.lic", diff --git a/tests/test_docs.py b/tests/test_docs.py index a3198787..a72a830f 100644 --- a/tests/test_docs.py +++ b/tests/test_docs.py @@ -156,7 +156,7 @@ def test_user_guide_does_not_repeat_install_steps(): assert "do not copy those commands" in intro install = (_docs_dir() / "install.md").read_text(encoding="utf-8") - assert "go2.cisco.com/baseconfigs" not in install + assert "go/baseconfigs" not in install assert "user-guide.md#prepare-software" in install assert "spa environment set --software-dir" in install @@ -168,6 +168,46 @@ def test_user_guide_does_not_repeat_install_steps(): assert "Homebrew" in setup +def test_baseconfig_link_is_the_short_go_link_and_the_changelog_names_no_link(): + """The employee baseconfig link is http://go/baseconfigs. The CHANGELOG carries no link to it.""" + for rel in ("skills/spa/spa/SKILL.md", "docs/user-guide.md"): + text = (PROJECT_ROOT / rel).read_text(encoding="utf-8") + assert "go2.cisco.com/baseconfigs" not in text, rel + assert "http://go/baseconfigs" in text, rel + changelog = (PROJECT_ROOT / "CHANGELOG.md").read_text(encoding="utf-8") + assert "go2.cisco.com/baseconfigs" not in changelog + assert "go/baseconfigs" not in changelog + + +def test_readme_and_user_guide_name_the_external_provider_for_existing_hosts(): + """The README intro, Features, and Start here, and the user guide intro, all name existing hosts.""" + readme = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8") + intro = readme.split("\n## ", 1)[0] + assert "existing hosts" in intro + assert "external provider" in intro + features = readme.split("## Features", 1)[1].split("\n## ", 1)[0] + assert "Existing hosts" in features + assert "--provider external" in features + assert "spa hosts register" in features + start = readme.split("## Start here", 1)[1].split("\n## ", 1)[0] + assert "--provider external" in start + assert "spa hosts register" in start + + guide_intro = (_docs_dir() / "user-guide.md").read_text(encoding="utf-8").split("## Start here", 1)[0] + assert "existing hosts" in guide_intro + + +def test_readme_opens_with_the_spa_pitch_before_support(): + """The SPA pitch sits above the support notice, so a new reader sees the case before the caveats.""" + readme = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8") + assert "## Why SPA" in readme + assert readme.index("## Why SPA") < readme.index("## Support") + pitch = readme.split("## Why SPA", 1)[1].split("\n## ", 1)[0] + assert "**One file, the whole environment.**" in pitch + assert "**Agent-ready.**" in pitch + assert "spa skills install cursor" in pitch + + def test_architecture_documents_playbook_resolution_contract(): text = (_docs_dir() / "architecture.md").read_text(encoding="utf-8") assert "`resolve()` and `resolve_named()` enforce" in text @@ -582,6 +622,35 @@ def test_create_config_asks_content_packs_after_itsi_before_more_apps(self): assert "content-packs list 5391" in step8 assert step8.index("content-packs list 5391") < step8.index("Add another app?") + def test_apps_skill_asks_roles_for_every_normal_app(self): + apps = (PROJECT_ROOT / "skills" / "spa" / "spa-apps" / "SKILL.md").read_text(encoding="utf-8") + workflow = apps[apps.index("## Workflow"):apps.index("ITSI / ITE Work")] + assert "One host with one role" not in workflow, "a single role must still be asked, not picked" + assert "even when the config has one host with one role" in workflow + assert "ITSI (1841) and content packs never ask this question" in workflow + + def test_create_config_phase6_asks_roles_before_a_normal_app_add(self): + create = (PROJECT_ROOT / "skills" / "spa" / "spa-create-config" / "SKILL.md").read_text(encoding="utf-8") + phase6 = create[create.index("## Phase 6 — Apps"):create.index("## Phase 6b")] + loop = phase6[phase6.index("**Loop"):] + step1 = loop.split("\n1. ", 1)[1].split("\n1b.", 1)[0] + assert "pick the ID and roles" not in loop + assert "AskQuestion" in step1 and "roles" in step1 + assert "no roles question" in step1 + + def test_create_config_app_table_uses_the_roles_flag(self): + create = (PROJECT_ROOT / "skills" / "spa" / "spa-create-config" / "SKILL.md").read_text(encoding="utf-8") + assert "apps add APP_ID --target-roles" not in create + assert "apps add APP_ID --roles ROLE1,ROLE2 --yes" in create + + def test_apps_questionnaire_asks_roles_for_normal_apps_only(self): + questionnaire = ( + PROJECT_ROOT / "skills" / "spa" / "spa-create-config" / "references" / "apps-questionnaire.md" + ).read_text(encoding="utf-8") + section4 = questionnaire[questionnaire.index("## 4. Target roles"):questionnaire.index("## 5. Premium ITSI?")] + assert "AskQuestion" in section4 + assert "normal" in section4 and "not ask" in section4.lower() + def test_region_default_never_comes_from_agent_knowledge(self): skills = PROJECT_ROOT / "skills" / "spa" / "spa-create-config" create = (skills / "SKILL.md").read_text(encoding="utf-8") diff --git a/tests/test_env_scaffold.py b/tests/test_env_scaffold.py index 129f4bb2..8d0db13b 100644 --- a/tests/test_env_scaffold.py +++ b/tests/test_env_scaffold.py @@ -12,7 +12,7 @@ def _ensure_software_stubs(stub_dir: Path) -> None: - (stub_dir / "spa_ci_stub" / "org_ds_secure_server").mkdir(parents=True, exist_ok=True) + (stub_dir / "spa_ci_stub" / "org_s2_indexer_indexes").mkdir(parents=True, exist_ok=True) (stub_dir / "spa_ci_stub" / "org_cluster_manager_base").mkdir(parents=True, exist_ok=True) for name in ( "Splunk_Enterprise.lic", diff --git a/tests/test_spa_apps.py b/tests/test_spa_apps.py index 8df11f9d..51b864b7 100644 --- a/tests/test_spa_apps.py +++ b/tests/test_spa_apps.py @@ -1205,6 +1205,37 @@ def test_cli_help_and_schema(): assert "Splunkbase app ID or local app folder name" in add_help.stdout +def test_agent_schema_apps_search_names_the_itsi_pack_list(): + schema = json.loads(run_spa(["agent", "schema"]).stdout) + search = next(row for row in schema["data"]["commands"] if row["name"] == "apps search") + assert "spa apps content-packs list 5391" in search["summary"] + assert search["example"] == "spa --agent apps search itsi" + + +def test_agent_schema_content_packs_list_says_itsi_content_packs(): + schema = json.loads(run_spa(["agent", "schema"]).stdout) + packs = next(row for row in schema["data"]["commands"] if row["name"] == "apps content-packs list") + assert "ITSI content packs" in packs["summary"] + assert "Content Library 5391" in packs["summary"] + + +def test_apps_search_help_explains_itsi_packs_and_kinds(): + help_result = run_spa(["--no-agent", "apps", "search", "--help"]) + assert help_result.returncode == 0, help_result.stderr + text = " ".join(help_result.stdout.split()) + assert "spa apps content-packs list 5391" in text + assert "spa apps add 5391 --content-pack NAME" in text + assert "itsi_content_library Content Library 5391" in text + assert "itsi_content_pack_single One pack with its own Splunkbase app ID" in text + + +def test_apps_help_names_content_packs_as_itsi_content_packs(): + help_result = run_spa(["--no-agent", "apps", "--help"]) + assert help_result.returncode == 0, help_result.stderr + text = " ".join(help_result.stdout.split()) + assert "content-packs Discover ITSI content packs in Content Library 5391" in text + + def test_cli_content_pack_catalog_from_explicit_archive(tmp_path, capsys): archive = tmp_path / "library.tgz" archive.write_bytes(_content_library_tgz()) diff --git a/tests/test_spa_paths.py b/tests/test_spa_paths.py index 67aafcec..c95dea6f 100644 --- a/tests/test_spa_paths.py +++ b/tests/test_spa_paths.py @@ -190,8 +190,8 @@ def exported(dest): cache_a = exported(env_a)["ANSIBLE_CACHE_PLUGIN_CONNECTION"] cache_b = exported(env_b)["ANSIBLE_CACHE_PLUGIN_CONNECTION"] - assert cache_a == str(env_a.resolve() / "ansible_facts") - assert cache_b == str(env_b.resolve() / "ansible_facts") + assert cache_a == str(env_a.resolve() / ".ansible_facts") + assert cache_b == str(env_b.resolve() / ".ansible_facts") assert cache_a != cache_b assert cache_a != "/tmp/ansible_facts" @@ -209,7 +209,7 @@ def test_apply_paths_env_replaces_a_stale_shared_fact_cache(tmp_path, monkeypatc for key in paths.export_env(): monkeypatch.setenv(key, "/tmp/ansible_facts" if key == "ANSIBLE_CACHE_PLUGIN_CONNECTION" else "stale") apply_paths_env(paths) - assert os.environ["ANSIBLE_CACHE_PLUGIN_CONNECTION"] == str(dest.resolve() / "ansible_facts") + assert os.environ["ANSIBLE_CACHE_PLUGIN_CONNECTION"] == str(dest.resolve() / ".ansible_facts") def test_software_prefers_env_sibling(tmp_path): diff --git a/tests/test_spa_preflight.py b/tests/test_spa_preflight.py index 41f6fcab..87e86c7a 100644 --- a/tests/test_spa_preflight.py +++ b/tests/test_spa_preflight.py @@ -49,6 +49,29 @@ def test_software_without_baseconfig_apps_fails(tmp_path): assert "baseconfig apps" in state.reason +def _baseconfig_software(tmp_path, names): + software = tmp_path / "Software" + for name in names: + (software / "ps" / name).mkdir(parents=True) + return software + + +def test_baseconfig_check_accepts_the_s2_indexer_indexes_app(tmp_path): + software = _baseconfig_software(tmp_path, ("org_s2_indexer_indexes", "org_cluster_manager_base")) + paths = _paths(tmp_path, "splunk_hosts:\n - name: idx1\n", software=software) + state = check_controller_data(paths) + assert state.ok is True + + +def test_baseconfig_check_no_longer_accepts_only_the_ds_secure_server_app(tmp_path): + software = _baseconfig_software(tmp_path, ("org_ds_secure_server", "org_cluster_manager_base")) + paths = _paths(tmp_path, "splunk_hosts:\n - name: idx1\n", software=software) + state = check_controller_data(paths) + assert state.ok is False + assert "org_s2_indexer_indexes" in state.reason + assert "org_ds_secure_server" not in state.reason + + def test_seeded_software_without_local_apps_ok(tmp_path): software = seed_software_dir(tmp_path) paths = _paths(tmp_path, "splunk_hosts:\n - name: idx1\n", software=software)