Skip to content

Expose structured app validation issues in JSON output#7070

Closed
dmerand wants to merge 1 commit intodlm-app-validate-errorsfrom
dlm-app-validate-error-shape
Closed

Expose structured app validation issues in JSON output#7070
dmerand wants to merge 1 commit intodlm-app-validate-errorsfrom
dlm-app-validate-error-shape

Conversation

@dmerand
Copy link
Contributor

@dmerand dmerand commented Mar 20, 2026

What

Expose structured validation issues in shopify app config validate --json.

This changes the public JSON response from a flat list of rendered error strings to a file-oriented {valid, issues} shape, and keeps that shape consistent when validation fails before the normal service path runs.

Why

Agentic validation work needs a stable machine-readable validation contract.

The human-readable output is fine for interactive use, but it is not a reliable interface for automation or repair loops. The command also needs to return JSON even when app loading aborts before validateApp() can render the result.

How

Use the structured issues preserved in the lower stack PR to emit a public issues array.

When only rendered text is available, synthesize a root issue so file-level context is not lost. For early aborts, parse the abort message into the same JSON shape and strip ANSI so colored terminal output does not leak into JSON strings.

Testing

Run shopify app config validate --json against invalid app configs and confirm failures return {valid: false, issues: [...]} for both normal validation errors and early loader-style aborts.

Copy link
Contributor Author

dmerand commented Mar 20, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 20, 2026

Coverage report

St.
Category Percentage Covered / Total
🟢 Statements 82.34% 15132/18377
🟡 Branches 74.96% 7489/9991
🟢 Functions 81.41% 3818/4690
🟢 Lines 82.73% 14299/17283

Test suite run success

4005 tests passing in 1529 suites.

Report generated by 🧪jest coverage report action from fceb5d6

@dmerand dmerand changed the base branch from dlm-app-validate-errors to graphite-base/7070 March 23, 2026 23:17
@dmerand dmerand force-pushed the dlm-app-validate-error-shape branch from 9b0ba08 to b34f147 Compare March 24, 2026 15:54
@dmerand dmerand force-pushed the graphite-base/7070 branch from 3c313f8 to 9d58583 Compare March 24, 2026 15:54
@dmerand dmerand changed the base branch from graphite-base/7070 to dlm-app-validate-errors March 24, 2026 15:54
@github-actions
Copy link
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/constants.d.ts
@@ -8,7 +8,6 @@ export declare const environmentVariables: {
     env: string;
     firstPartyDev: string;
     noAnalytics: string;
-    cliToken: string;
     partnersToken: string;
     runAsUser: string;
     serviceEnv: string;
packages/cli-kit/dist/public/node/environment.d.ts
@@ -10,12 +10,17 @@
  */
 export declare function getEnvironmentVariables(): NodeJS.ProcessEnv;
 /**
- * Returns the value of the SHOPIFY_CLI_TOKEN environment variable,
- * falling back to the deprecated SHOPIFY_CLI_PARTNERS_TOKEN.
+ * Returns the value of the SHOPIFY_CLI_PARTNERS_TOKEN environment variable.
  *
- * @returns The CLI token value, or undefined if neither env var is set.
+ * @returns Current process environment variables.
+ */
+export declare function getPartnersToken(): string | undefined;
+/**
+ * Check if the current proccess is running using the partners token.
+ *
+ * @returns True if the current proccess is running using the partners token.
  */
-export declare function getCliToken(): string | undefined;
+export declare function usePartnersToken(): boolean;
 /**
  * Returns the value of the organization id from the environment variables.
  *
packages/cli-kit/dist/private/node/session/exchange.d.ts
@@ -25,7 +25,7 @@ export declare function exchangeAccessForApplicationTokens(identityToken: Identi
  */
 export declare function refreshAccessToken(currentToken: IdentityToken): Promise<IdentityToken>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid Partners API token.
+ * Given a custom CLI token passed as ENV variable, request a valid Partners API token
  * This token does not accept extra scopes, just the cli one.
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
@@ -35,7 +35,7 @@ export declare function exchangeCustomPartnerToken(token: string): Promise<{
     userId: string;
 }>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid App Management API token.
+ * Given a custom CLI token passed as ENV variable, request a valid App Management API token
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
  */
@@ -44,7 +44,7 @@ export declare function exchangeCliTokenForAppManagementAccessToken(token: strin
     userId: string;
 }>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid Business Platform API token.
+ * Given a custom CLI token passed as ENV variable, request a valid Business Platform API token
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
  */

@dmerand dmerand changed the base branch from dlm-app-validate-errors to graphite-base/7070 March 26, 2026 14:01
@dmerand dmerand force-pushed the dlm-app-validate-error-shape branch from 0456af0 to fceb5d6 Compare March 26, 2026 14:33
@dmerand dmerand force-pushed the graphite-base/7070 branch from 15b52cf to 964455a Compare March 26, 2026 14:33
@dmerand dmerand changed the base branch from graphite-base/7070 to dlm-app-validate-errors March 26, 2026 14:33
@dmerand
Copy link
Contributor Author

dmerand commented Mar 26, 2026

Superseded by #7090 after collapsing the stack. #7090 now carries both the public {valid, issues} JSON contract and the typed local-abort path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant