Skip to content

[WIP] CI Build Reworked#446

Draft
jbachorik wants to merge 19 commits intomainfrom
jb/merge_build
Draft

[WIP] CI Build Reworked#446
jbachorik wants to merge 19 commits intomainfrom
jb/merge_build

Conversation

@jbachorik
Copy link
Copy Markdown
Collaborator

What does this PR do?:

Motivation:

Additional Notes:

How to test the change?:

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
  • This PR doesn't touch any of that.
  • JIRA: [JIRA-XXXX]

Unsure? Have a question? Request a review!

jbachorik and others added 19 commits March 27, 2026 11:07
Replaces the 25-line trigger stub (.gitlab-ci.yml) with a full
self-contained pipeline. CI now runs from within the repo — GitLab's
standard checkout eliminates the need to clone java-profiler separately.

Key changes:
- Root .gitlab-ci.yml: full pipeline (build, test, deploy, benchmarks,
  reliability, jdk-integration, dd-trace-integration, image management)
- .gitlab/scripts/: adapted build/deploy/prepare/stresstests scripts;
  removed git clone, cd java-profiler, capture/restore_git_ref
- .gitlab/common.yml: cache key no longer has java-profiler/ prefix
- .gitlab/build-deploy/: artifact paths drop java-profiler/ prefix;
  upload-s3 uses CI_COMMIT_BRANCH instead of DDPROF_COMMIT_BRANCH
- .gitlab/{benchmarks,reliability,jdk-integration,dd-trace-integration}/:
  sub-pipeline configs adapted for in-repo execution
- gitlab/: Dockerfiles, config.env, test-apps (formerly ci/)
- doc/: octo-sts policy documentation

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
**/build-*/ pattern was silently excluding .gitlab/build-deploy/.
Add gitignore exception !.gitlab/build-deploy/ to prevent this.

Also: dd_crash_uploader.sh is auto-generated at JVM startup (not a
static file to copy).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…line)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- create-image-update-pr.sh: GITHUB_REPO + OCTO_STS_SCOPE -> DataDog/java-profiler
- check-image-updates.sh: GITLAB_PROJECT_PATH -> DataDog/java-profiler
- rebuild-images.sh: Dockerfile paths ci/ -> gitlab/

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- gh-pages.sts.yaml: allows CI to push gh-pages reports
- update-images.sts.yaml: allows CI to create image update PRs
- async-profiler-build.ci.sts.yaml: update subject_pattern from
  apm-reliability/async-profiler-build to DataDog/java-profiler

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Required for git-cache-s3 / gitretriever infrastructure to work.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- config.env: SSM_PREFIX ci.async-profiler-build -> ci.java-profiler
- create_key, deploy-artifact, upload-s3: KUBERNETES_SERVICE_ACCOUNT_OVERWRITE=java-profiler

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Collapse build-deploy/benchmarks/reliability child pipeline triggers
  into direct includes; jdk-integration-test kept as child pipeline
  (externally triggered with mutually exclusive variables)
- Add unified stages list and default tags/interruptible in root
- Extract .bootstrap-gh-tools, .deploy-sa, fix .cache-config-pull
  to extend .cache-config (policy: pull only)
- Fix KUBERNETES_SERVICE_ACCOUNT_OVERWRITE: async-profiler-build -> java-profiler
- Replace deprecated except: with rules: in rebuild-images
- Remove redundant dependencies: blocks superseded by needs: artifacts: true
- Extract .reliability_job template; amd64/aarch64 extend it with ARCH var
- Gate prepare:start on DOWNSTREAM != null to prevent spurious runs
- Single get-versions job shared by benchmarks and reliability

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add workflow:rules to prevent pipelines starting on non-default
  branch pushes (feature branches require manual/trigger/schedule)
- Extend .deploy-sa on notify jobs so the pod gets IRSA credentials
  needed by postmessage to reach SSM for the Slack token

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- prepare.sh exits 1 (no API token available for clean cancel)
- get-versions now needs prepare:start so it is skipped too

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
prepare:start writes CANCELLED=true to build.env and exits 0.
default.before_script checks it and exits 0 immediately on all
downstream jobs — pipeline stays green, no real work done.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@dd-octo-sts
Copy link
Copy Markdown

dd-octo-sts bot commented Mar 27, 2026

CI Test Results

Run: #23641308283 | Commit: 7a340f6 | Duration: 10m 56s (longest job)

All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - - -
8-ibm - - -
8-j9 - -
8-librca - -
8-orcl - - -
11 - - -
11-j9 - -
11-librca - -
17 - -
17-graal - -
17-j9 - -
17-librca - -
21 - -
21-graal - -
21-librca - -
25 - -
25-graal - -
25-librca - -

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-03-27 10:25:58 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant