* Add warning-only skill Eval coverage checks Add deterministic changed-skill coverage checks and trusted PR comments with dashboard links. Preserve blocking Vally lint and cover discovery, fork safety, stale updates, and unknown-scope behavior. Related to #16345. * Isolate coverage test output and report build details * Fix coverage lint selection and multi-turn validation * Use Node 24 throughout skill coverage workflows * Use workflow dependency locks for readable skill check action versions Onboard only the two skill coverage workflows to the official generated Actions lockfile while preserving the current checkout/setup-node commits and repository identities. Add scoped read-only frozen verification, regression contracts and maintenance documentation. Keep Node 24, independent blocking lint, trusted reporting, centralized links and all repository policies unchanged. * Announce November 9 skill Eval coverage enforcement checkpoint Add a prominent grace-period notice to missing-coverage comments and summaries, allowing authors to proceed only when other required checks pass. Keep coverage warning-only; do not add automatic date-based enforcement. Extend existing renderer, reporter and CLI regressions and document follow-up issue #17402. * Discover skill-local Eval presence and address workflow review feedback Remove pipeline discovery requirements from the presence checker and configuration. Preserve warning-only behavior, current lint scope, immutable GitHub reads, and the November 9 enforcement notice. Restore direct action SHA pins, use frozen standalone pnpm installs, separate script tests, and sparse-checkout the trusted reporter. Remove obsolete shared glob and lockfile tooling. Document semantic coverage as a separate advisory follow-up. * Use Eval dashboard short link * Keep skill Eval coverage tests portable across synced repositories Move Tools workflow assertions to eng/common-tests, run the full shared suite in an isolated consumer, and remove README links to unsynced files. Preserve read-only test workflow permissions and frozen dependency installs. --------- Co-authored-by: helen229 <gaoh@microsoft.com>
Build Scripts
eng/scripts/New-BuildInfo.ps1
To simplify the work of collection server and platform metadata in build scripts, we use the common metadata file build_info.json. New-BuildInfo.ps1 creates the build_info.json file containing server details, along with a list of projects that should be tested in a CI/PR build and matrices for CI build and test stages.
./eng/scripts/New-BuildInfo.ps1 produces .work/build_info.json with contents like:
{
"buildId": 99999,
"publishTarget": "none",
"dynamicPrereleaseVersion": true,
"repositoryUrl": "https://github.com/microsoft/mcp",
"branch": "main",
"commitSha": "8ae9e6f971b0eb97c1e64534773cc9e6045952a8",
"servers": [
{
"name": "Template.Mcp.Server",
"path": "servers/Template.Mcp.Server/src/Template.Mcp.Server.csproj",
"artifactPath": "Template.Mcp.Server",
"version": "0.0.12-alpha.99999",
"releaseTag": "Template.Mcp.Server-0.0.12-alpha.99999",
"cliName": "mcptmp",
"assemblyTitle": "Template MCP Server",
"description": "Template MCP Server - Basic Model Context Protocol implementation",
"readmeUrl": "https://github.com/Microsoft/mcp/blob/main/servers/Template.Mcp.Server#readme",
"readmePath": "servers/Template.Mcp.Server/README.md",
"packageIcon": "eng/images/microsofticon.png",
"npmPackageName": "@azure/mcp-template",
"npmDescription": "",
"npmPackageKeywords": [
"azure",
"mcp",
"model-context-protocol"
],
"dockerImageName": "azure-sdk/template-mcp",
"dockerDescription": "",
"dnxPackageId": "Microsoft.Template.Mcp",
"dnxDescription": "",
"dnxToolCommandName": "mcptmp",
"dnxPackageTags": [
"template",
"mcp"
],
"vsixPackagePrefix": "vscode-template-mcp",
"vsixDescription": "Template project for validating the microsoft/mcp engineering system",
"vsixPublisher": "ms-azuretools",
"platforms": [
{
"name": "linux-x64",
"artifactPath": "Template.Mcp.Server/linux-x64",
"operatingSystem": "linux",
"nodeOs": "linux",
"dotnetOs": "linux",
"architecture": "x64",
"extension": "",
"native": false
},
...
]
}
],
"pathsToTest": [
{
"path": "core/Microsoft.Mcp.Core",
"testResourcesPath": null,
"hasLiveTests": false,
"hasTestResources": false,
"hasUnitTests": false
},
{
"path": "core/Template.Mcp.Core",
"testResourcesPath": null,
"hasLiveTests": false,
"hasTestResources": false,
"hasUnitTests": false
},
{
"path": "servers/Template.Mcp.Server",
"testResourcesPath": null,
"hasLiveTests": false,
"hasTestResources": false,
"hasUnitTests": false
},
{
"path": "tools/Fabric.Mcp.Tools.PublicApi",
"testResourcesPath": null,
"hasLiveTests": false,
"hasTestResources": false,
"hasUnitTests": false
}
],
"matrices": {
"linuxBuildMatrices": {
"x64": {
"linux_x64": {
"Pool": "Missing-LINUXPOOL",
"OSVmImage": "Missing-LINUXVMIMAGE",
"Architecture": "x64",
"Native": false,
"RunUnitTests": true
},
"linux_musl_arm64_docker": { ... }
},
"arm64": {
"linux_arm64": {
...
}
}
},
"linuxSmokeTestMatrix": {
"linux_x64": {
"Pool": "Missing-LINUXPOOL",
"OSVmImage": "Missing-LINUXVMIMAGE",
"Architecture": "x64"
}
},
"macosBuildMatrices": {
"x64": {
"macos_x64": {},
"macos_arm64": {}
},
"arm64": {}
},
...
"liveTestMatrix": {
"tools/Azure.Mcp.Tools.Workbooks": {
"pathToTest": "tools/Azure.Mcp.Tools.Workbooks",
"testResourcesPath": "tools/Azure.Mcp.Tools.Workbooks/tests",
"hasTestResources": true
},
...
}
}
}
eng/scripts/Build-Code.ps1
Build-Code.ps1 is a common build script that compiles server projects in the repository, using the metadata collected in build_info.json. It supports building for multiple platforms, including native builds, and can be used in both local and CI environments.
Build-Code.ps1 defaults to:
-BuildInfoPath:.work/build_info.json, the default output path ofNew-BuildInfo.ps1-OutputPath:.work/build, the default -ArtifactsPath for the Pack scripts-ServerName: empty, which builds all servers listed inbuild_info.json.-OperatingSystem,-Architecture, and-AllPlatforms: all empty or false, which builds for the current, local platform only.
For more production-like builds, you can enable the switches: -SelfContained -SingleFile -Trimmed
So, a parameterless ./eng/scripts/Build-Code.ps1 will build all servers listed in build_info.json for the local platform as framework-dependent, non-trimmed, non-single-file applications, outputting them to the default path .work/build.
eng/scripts/Compress-ForSigning.ps1
Compress-ForSigning.ps1 collects the build output from Build-Code.ps1, organizes it into a standardized folder structure, and compresses Mac binaries into ZIP archives suitable for signing. This script uses the metadata in build_info.json to determine the correct paths and naming conventions for the ZIP files.
Compress-ForSigning.ps1 doesn't have any use in local development other than to test that its output is correct for later signing in a CI pipeline.
Published artifact types
npm/npx
- packed with
Pack-Npm.ps1 - released via
release-npm.yml - required project file properties:
NpmPackageName: the package name shown on npmjs.com (for example,@azure-tools/azure-mcp).NpmDescription: (optional, defaults toDescriptionif not set) the package description shown on npmjs.comNpmPackageKeywords: (comma separated) an array of keywords to help users find the package on npmjs.com
Servers are packed into two npm package types:
- A platform-specific package for each OS/architecture combination (for example,
@azure-tools/azure-mcp-linux-x64), containing the native binaries for that platform. - A wrapper package (for example,
@azure-tools/azure-mcp) that depends on the platform-specific packages and provides a unified CLI experience vianpx(for example,npx @azure-tools/azure-mcp@latest server start).
NuGet/dnx
- packed with
Pack-Nuget.ps1 - released via
release-nuget.yml - required project file properties:
DnxPackageId: the package name shown on nuget.org (for example,Azure.Mcp).DnxDescription: (optional, defaults toDescriptionif not set) the package description shown on nuget.orgDnxToolCommandName: (optional, defaults toCliName) the command users run after installing the package as a global tool (for example,azmcp).DnxPackageTags: (comma separated) an array of keywords to help users find the package on nuget.org
Docker
- packed with
Prepare-Docker.ps1 - released via
release-docker.yml - required project file properties:
DockerImageName: the name of the Docker image (for example,azure-sdk/azure-mcp).
VSIX
- packed with
Pack-Vsix.ps1 - released via
release-vsix.yml - requires customized package manifest and assets under
servers/<ServerName>/vscode
Zip
- packed with
Pack-Zip.ps1 - added to GitHub releases via
release.yml - no special project file properties required
Zip files for each server in a release are added to the GitHub release assets.
MCPB (MCP Bundle)
- packed with
Pack-Mcpb.ps1 - signed via
pack-and-sign-mcpb.yml - released via
release-mcpb.yml - requires a
manifest.jsonandicon.pngunderservers/<ServerName>/mcpb/
MCPB files are signed MCP bundles that can be installed directly into Claude Desktop with a single click. Each platform (win-x64, linux-x64, osx-x64, osx-arm64) produces a separate .mcpb file containing trimmed server binaries. The bundles are signed using ESRP's Pkcs7DetachedSign operation and verified with the mcpb verify CLI.
Signing pipeline scripts:
Stage-McpbForSigning.ps1- Prepares files for ESRP signingApply-McpbSignatures.ps1- Applies PKCS#7 signatures to MCPB filesVerify-McpbSignatures.ps1- Verifies signatures using mcpb CLI
For design details, see docs/design/mcpb-packaging-and-signing-via-esrp.md.
Server release process
We only ship the repo's server projects, with each server stored under servers/<ServerName>. The release pipelines are driven by the server-specific build.yml files (for example, servers/Azure.Mcp.Server/build.yml). Those files opt a server into the artifact types we publish and pass the settings that flow through /eng/pipelines/templates/common.yml.
Scheduled Azure MCP release preparation
release-prep.yml opens an Azure MCP Server release-preparation PR every Tuesday and Thursday at 12 PM Pacific, except on Microsoft US holidays. It does not merge PRs, publish packages, or queue the release pipeline.
- Azure DevOps uses UTC schedules, so the pipeline has 19:00 and 20:00 UTC triggers. A Pacific time-zone check selects exactly one trigger for the current daylight-saving offset, including when agent allocation delays the job within the scheduled Pacific day. The inactive trigger exits without requesting GitHub credentials.
- The calendar excludes these 10 Microsoft US holidays: New Year's Day, Martin Luther King Jr. Day, Washington's Birthday, Memorial Day, Independence Day, Labor Day, Thanksgiving Day, the day after Thanksgiving, Christmas Eve, and Christmas Day. Fixed-date holidays are excluded on both their actual and observed dates. Saturday holidays are observed on Friday and Sunday holidays on Monday, including New Year's Day observed in the previous December. Holiday runs are skipped, not moved to another day.
- The
ReleaseOwnersparameter is the list of GitHub usernames, without@, to mention in the PR description. The description asks them to review, approve and merge the PR after checks pass, then start Azure.Mcp release. - Preparation uses the existing version in the Azure MCP Server project and requires a matching
Unreleasedchangelog heading. Already-finalized versions and runs without changelog entries are skipped; merge the normal post-release version-increment PR before the next release. - The job compiles both changelogs, removes the compiled entry files, and opens a PR from a version-specific branch in
azure-sdk/mcp. An existing open preparation PR for that version is left unchanged, preserving reviewer edits. - The repository PR template is preserved, with a pipeline link and validation results added. Required PR checks still run normally; preparation does not claim that builds or tests have passed.
Queuing a release run
- Manually queue the server pipeline (e.g.
mcp - Azure.Mcp.Server) withPublishTargetset tonone,internal, orpublic.nonebuilds and validates only. Outputs are saved as unsigned pipeline artifacts. Servers will be given a prerelease version based on the version in the servers' project file, with a suffix like-alpha.<buildId>.internalruns signing, packaging and the internal integration stage. Servers will also be given a prerelease version for internal release based on the project file version. Outputs are saved as pipeline artifacts and pushed to internal registries.publicruns signing, packaging, and the release jobs in eng/pipelines/templates/jobs/release.yml. This creates the GitHub release tag and pushes artifacts to their public registries. Public releases use the version in the server project file as-is, not dynamic prerelease versions.
- Expect post-release automation: After a successful public release,
jobs/release.ymlrunsUpdate-Version.ps1to increment the server version and open a PR. This PR must be merged before the next release can start.
Server release configuration
The parameters in a server's build.yml decide what we produce. Set PackageDocker: true to include Docker images, PackageVSIX: true to include VS Code extensions, PackageMCPB: true to include signed MCP bundles. Every pipeline always creates the npm/npx and NuGet/dnx artifacts.
Project file properties
Packaging and release scripts depend on metadata found in a build's build_info.json file, which is generated by eng/scripts/New-BuildInfo.ps1 using properties from each server's project file. See eng/scripts/Get-ProjectProperties.ps1 for a list of all supported project file properties.
Project references
Use $(RepoRoot) for ProjectReference paths between components and for wildcard references. Directory.Build.props defines this property with a trailing directory separator, so do not add another separator after it or use .. segments in rooted references.
Projects under a component's tests directory should reference that component's own src project using a relative path: ..\..\src\... for projects nested under tests\<project>, or ..\src\... for projects directly under tests. This includes test helpers and fixtures. References from tests to other components still use $(RepoRoot).
<ProjectReference Include="$(RepoRoot)core\Azure.Mcp.Core\src\Azure.Mcp.Core.csproj" />
<ProjectReference Include="$(RepoRoot)tools\Azure.*\src\*.csproj" />
The project reference repair script uses the same convention.