Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .azure-pipelines/1ES.Release.Crates.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
#
# MXC Crates.io Release Pipeline (ESRP)
#
# Packages mxc-sdk and submits it to crates.io through ESRP.
#
# Crate versions come from src/Cargo.toml at that ref.
#
# Setup and prerequisites: .azure-pipelines/README.md

trigger: none
pr: none

name: $(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)

resources:
repositories:
- repository: 1ESPipelineTemplates
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release

parameters:
- name: dryRun
displayName: 'Dry run β€” run Cargo publication checks without uploading'
type: boolean
default: true

variables:
- group: MXC-ESRP-Signing

extends:
template: v1/1ES.Official.PipelineTemplate.yml@1ESPipelineTemplates
parameters:
pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: windows-2022
os: windows
customBuildTags:
- ES365AIMigrationTooling

sdl:
sourceAnalysisPool:
name: Azure-Pipelines-1ESPT-ExDShared
image: windows-latest
os: windows

stages:
# Create the crate artifact consumed by the publish stage.
- stage: Package_Crates_For_Release
displayName: 'Package Crates'
dependsOn: []
jobs:
- template: templates/Package.Crates.Job.yml@self

- stage: Publish_to_CratesIo
displayName: 'Publish Crates.io Packages'
dependsOn: Package_Crates_For_Release
condition: succeeded()
jobs:
- template: templates/Publish.CratesIo.Job.yml@self
parameters:
ESRPInfo:
serviceName: $(serviceName)
tenantId: $(tenantId)
azureKeyVaultName: $(azureKeyVaultName)
authCertName: $(authCertName)
signCertName: $(signCertName)
clientId: $(clientId)
OwnersEmail: $(Build.RequestedForEmail)
ApproversEmail: $(Build.RequestedForEmail)
dryRun: ${{ parameters.dryRun }}
17 changes: 17 additions & 0 deletions .azure-pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,23 @@ from crates.io and npmjs, helping ensure secure and vetted consumption of third
the fixed-version package publicly to NuGet.org through the `MXC Nuget`
service connection.

### Rust crates.io release

`1ES.Release.Crates.yml` is a manual pipeline. `dryRun` defaults to `true` and
runs `cargo publish --dry-run` from any ref after creating the release archives.
Publishing through ESRP requires an immutable `refs/tags/v*` tag and explicitly
setting `dryRun` to `false`.

- `mxc-sdk` uses the workspace version and complete crates.io metadata.
- `Invoke-CratePackage.ps1` verifies that `mxc-sdk` is the only Rust release
crate and packages it into the release artifact.
- Azure Pipelines packages `mxc-sdk` once using the private Rust toolchain and
dependency feed.
- The publish template owns the crate list; release operators cannot override
it at queue time. GitHub CI rejects any difference from the validated list.
- Every release runs Cargo's crates.io dry run before the publish job submits
the package through ESRP.

### PR Pipelines
- GitHub Actions runs the PR validation build automatically on every pull
request β€” it mirrors the ADO build stages on native hardware for faster
Expand Down
6 changes: 6 additions & 0 deletions .azure-pipelines/templates/1ES.Build.Stages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,12 @@ stages:

- template: Mxc.Binary.Packaging.Job.yml

- stage: Package_Crates
displayName: 'Package Crates'
dependsOn: []
jobs:
- template: Package.Crates.Job.yml
Comment thread
bbonaby marked this conversation as resolved.

- stage: Build_DotNet_SDK
displayName: 'Build .NET SDK'
dependsOn:
Expand Down
58 changes: 58 additions & 0 deletions .azure-pipelines/templates/Package.Crates.Job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
#
# Packages mxc-sdk into a .crate file using `cargo package --no-verify`. The
# publish template always performs Cargo's crates.io dry run before a
# production ESRP release.
#
# The steps hold no logic: Invoke-CratePackage.ps1 does the packaging and
# validates the single published crate, and the same command reproduces a leg
# locally.

parameters:
- name: cratesArtifactName
type: string
default: mxc-crates-package

jobs:
- job: package_crates
displayName: Package Crates
pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: ubuntu-latest
os: linux

variables:
outDir: $(Build.SourcesDirectory)/out/crates

templateContext:
rust:
rustToolchain:
toolchainFeed: https://microsoft.pkgs.visualstudio.com/Dart/_packaging/Mxc-Azure-Feed/nuget/v3/index.json
version: 'ms-prod-1.93'

steps:
- checkout: self

# Configure and authenticate the private registry explicitly before the
# 1ES task installs the internal Rust toolchain.
- template: Cargo.Setup.Private.yml@self
parameters:
authenticate: true

- task: 1ES.Build.Rust.Setup@1

- task: PowerShell@2
displayName: Package mxc-sdk
inputs:
targetType: filePath
filePath: $(Build.SourcesDirectory)/scripts/ci/Invoke-CratePackage.ps1
arguments: -OutDir '$(outDir)' -PackageRegistry 'Mxc-Azure-Feed'
pwsh: true
workingDirectory: $(Build.SourcesDirectory)

- task: 1ES.PublishPipelineArtifact@1
displayName: Publish crate package artifact
inputs:
path: $(outDir)
artifactName: ${{ parameters.cratesArtifactName }}
125 changes: 125 additions & 0 deletions .azure-pipelines/templates/Publish.CratesIo.Job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
#
# Publishes mxc-sdk to crates.io through ESRP Release under the
# microsoft-oss-releases account. Consumes the mxc-crates-package artifact.
#
# Dry runs invoke Cargo against the checked-out workspace. Real releases submit
# one packaged crate per ESRP task; each task waits for completion before the
# crate is submitted. CI keeps the crate parameter synchronized.
#
# The pool enforces 1ES network isolation (CFSClean), so this job performs no
# crates.io reads.

parameters:
- name: ESRPInfo
type: object
- name: publishCrates
type: object
# Keep this list internal to the template so release operators cannot
# override the Cargo-validated publication order at queue time.
default:
# BEGIN MXC-SDK PUBLISH CRATES
- mxc-sdk
# END MXC-SDK PUBLISH CRATES
- name: dryRun
type: boolean
default: true
- name: cratesArtifactName
type: string
default: mxc-crates-package

jobs:
- job: validate_crates_publication
displayName: Validate crates.io publication
pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: ubuntu-latest
os: linux
templateContext:
rust:
rustToolchain:
toolchainFeed: https://microsoft.pkgs.visualstudio.com/Dart/_packaging/Mxc-Azure-Feed/nuget/v3/index.json
version: 'ms-prod-1.93'
steps:
- checkout: self

- template: Cargo.Setup.Private.yml@self
parameters:
authenticate: true

- task: 1ES.Build.Rust.Setup@1

- pwsh: |
$ErrorActionPreference = 'Stop'
$crates = [string[]] ($env:PUBLISH_CRATES_JSON | ConvertFrom-Json)
$arguments = [System.Collections.Generic.List[string]]::new()
$arguments.Add('publish')
$arguments.Add('--dry-run')
$arguments.Add('--locked')
Comment on lines +57 to +59
$arguments.Add('--manifest-path')
$arguments.Add('src/Cargo.toml')
foreach ($crate in $crates)
{
$arguments.Add('-p')
$arguments.Add($crate)
}
Write-Host "cargo $($arguments -join ' ')"
cargo @arguments
if ($LASTEXITCODE -ne 0) { throw "cargo publish --dry-run failed with exit $LASTEXITCODE" }
displayName: Cargo publish dry run
env:
PUBLISH_CRATES_JSON: ${{ convertToJson(parameters.publishCrates) }}

- ${{ if eq(parameters.dryRun, false) }}:
- job: publish_crates
displayName: Publish crates to crates.io (ESRP)
dependsOn: validate_crates_publication
condition: succeeded()
pool:
name: Azure-Pipelines-1ESPT-ExDShared
image: windows-2022
os: windows
templateContext:
type: releaseJob
isProduction: true
inputs:
- input: pipelineArtifact
artifactName: ${{ parameters.cratesArtifactName }}
targetPath: $(Pipeline.Workspace)/crates
variables:
cratesDir: $(Pipeline.Workspace)/crates
steps:
- pwsh: |
$ErrorActionPreference = 'Stop'
$ref = $env:SOURCE_REF
Write-Host "Run was queued against: $ref"
if ($ref -cnotlike 'refs/tags/v?*')
{
throw "Crates.io publication requires refs/tags/v*; queued ref was '$ref'."
}
Write-Host "Releasing from tag $($ref.Substring('refs/tags/'.Length))"
displayName: Require a version tag
env:
SOURCE_REF: $(Build.SourceBranch)

- ${{ each crate in parameters.publishCrates }}:
- task: EsrpRelease@12
Comment thread
bbonaby marked this conversation as resolved.
displayName: "Publish ${{ crate }} to crates.io"
inputs:
connectedservicename: ${{ parameters.ESRPInfo.serviceName }}
usemanagedidentity: false
keyvaultname: ${{ parameters.ESRPInfo.azureKeyVaultName }}
authcertname: ${{ parameters.ESRPInfo.authCertName }}
signcertname: ${{ parameters.ESRPInfo.signCertName }}
clientid: ${{ parameters.ESRPInfo.clientId }}
intent: 'PackageDistribution'
contenttype: 'Rust'
contentsource: 'Folder'
folderlocation: $(cratesDir)\${{ crate }}
owners: ${{ parameters.ESRPInfo.OwnersEmail }}
approvers: ${{ parameters.ESRPInfo.ApproversEmail }}
waitforreleasecompletion: true
serviceendpointurl: 'https://api.esrp.microsoft.com'
mainpublisher: 'ESRPRELPACMAN'
domaintenantid: ${{ parameters.ESRPInfo.tenantId }}
4 changes: 4 additions & 0 deletions .github/workflows/Versioning.Checks.Job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,10 @@ jobs:
- name: Check Rust toolchain sync (rust-toolchain.toml == ADO template)
run: node scripts/versioning/check-rust-toolchain-sync.js

- name: Check crate publishing metadata
shell: pwsh
run: ./scripts/ci/Invoke-CratePackage.ps1 -ValidateOnly

- name: Check dotnet ErrorCode parity (C# enum == native MXC_STATUS_*)
run: node scripts/check-dotnet-errorcode-parity.js

Expand Down
4 changes: 3 additions & 1 deletion docs/pull-requests.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ Every PR is validated automatically by the GitHub Actions workflows under
it fans out to the reusable `Build.Windows.Job.yml`, `Build.Linux.Job.yml`, and
`Build.MacOS.Job.yml` workflows, which build and test on native Windows
x64/arm64, Linux x64/arm64, and macOS arm64 hosts, then runs the lint,
versioning, and SDK jobs.
versioning, repository-consistency, and SDK jobs. Repository consistency also
checks that `mxc-sdk` is the only Rust release crate and has a valid package
name, version, description, repository, and license.

### Local Node SDK package validation

Expand Down
Loading
Loading