refactor: move step templates to source-first authoring under src#1665
Draft
bcullman wants to merge 21 commits intoOctopusDeploy:masterfrom
Draft
refactor: move step templates to source-first authoring under src#1665bcullman wants to merge 21 commits intoOctopusDeploy:masterfrom
bcullman wants to merge 21 commits intoOctopusDeploy:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
This PR addresses the long-standing reviewability problem described in #575.
Previously, step templates lived under
/step-templatesas packed JSON files with embedded script content. That packed format is still required for compatibility with Octopus and the Library site, but it made review difficult because script changes appeared as escaped JSON string blobs instead of normal source files. Over time that also made Git history and blame much less useful for understanding how templates had changed.To compensate for that, the repo accumulated helper workflows and scripts for packing, unpacking, and diffing. Those helped, but the authoring experience still centered around generated artifacts instead of source files.
This PR moves the repository to a source-first model:
/src/step-templates/<template>/metadata.jsonfile plus sibling source script files such asscriptbody.ps1,scriptbody.sh,scriptbody.py,predeploy.ps1,deploy.ps1, andpostdeploy.ps1/step-templates/*.jsonremain generated compatibility output onlyThe migration used
git mvfor template metadata files so history remains attached to the authored template sources as much as possible.Results
This PR delivers four main outcomes:
Step templates are now reviewed as source files.
Script changes are visible as normal file diffs under
/src/step-templates/..., which makes PR review, blame, and history inspection much clearer./src/step-templatesis now the source of truth./step-templates/*.jsonis generated-only compatibility output and is ignored by Git.The website and Library data flow remain compatible.
The site still builds from generated packed JSON, and the aggregate template data is still produced for the app.
The developer workflow is improved.
npm run devand watch mode now support source-first authoring directly, with better generation performance and better live-reload behavior.Before
/step-templatesnpm run devopened the BrowserSync proxy tab on3000, even if the useful app URL was9000Historylinks on the site could point at broken or legacy-generated pathsAfter
/src/step-templates/<template>/metadata.json/step-templates/*.jsonis generated-onlynpm run devstartup generation is significantly faster9000while keeping BrowserSync available in the backgroundNotable Implementation Details
step-templates/logosandstep-templates/testsintosrc/step-templates_pack.ps1and_unpack.ps1behavior as the canonical packing and unpacking implementation.json.origfile before conversion.json.origfiles were used as the validation baseline after repacking from the newsrc/step-templates/...source layout.json.origversion to prove that the new source-first representation could round-trip back to the original packed artifactExportedAtformatting behavior of the unchanged packer and a trailing final newline difference where present.gitignoreso generatedstep-templates/*output is not committed.github/workflows/Hyponome.ymlin place for now, but it may also no longer serve a meaningful purpose after this changeAdditional Fixes Included In This PR
Along the way, this PR also fixes several related issues that surfaced during the migration:
npm run devopens the primary app URL on9000instead of the less-useful proxy tab on3000HistoryUrlgeneration so template history links resolve to the source-authored template location instead of broken generated pathsHistorylink UX so it opens in a new tab instead of navigating away from the current pageFollow-Up Candidates
The following items were intentionally retained in this branch, but may no longer be necessary after this change:
.github/workflows/Hyponome.ymltools/_diff.ps1tools/Converter.ps1(possible follow-up cleanup if we confirm it is fully redundant with the current pack/unpack and source-first tooling)Assumptions
https://library.octopus.com/. The assumption for this PR is that the existing publication path either already runs the normal build and generation flow, or only needs a small workflow adjustment so generatedstep-templates/*.jsonare produced before publication.https://library.octopus.com/, not directly from this Git repository. Based on the Octopus documentation, this PR should therefore remain compatible with Octopus as long as the Library site continues to publish valid packed step template JSON.Pre-requisites
Idvalues were preserved for existing templatesLastModifiedByfields remain presentsrc/step-templates/logoshumanizecategory mapping remains in place ingulpfile.babel.jsVersionshould be incrementedNote: template
Versionvalues were not broadly incremented in this PR because the intent was repository authoring-model migration and compatibility preservation rather than semantic template behavior changes. If reviewers want version increments purely for republished template sync behavior, that should be called out explicitly because it would intentionally change the update semantics for every migrated template.Addresses #575