* Add warning-only skill Eval coverage checks
Add deterministic changed-skill coverage checks and trusted PR comments with dashboard links. Preserve blocking Vally lint and cover discovery, fork safety, stale updates, and unknown-scope behavior. Related to #16345.
* Isolate coverage test output and report build details
* Fix coverage lint selection and multi-turn validation
* Use Node 24 throughout skill coverage workflows
* Use workflow dependency locks for readable skill check action versions
Onboard only the two skill coverage workflows to the official generated Actions lockfile while preserving the current checkout/setup-node commits and repository identities. Add scoped read-only frozen verification, regression contracts and maintenance documentation. Keep Node 24, independent blocking lint, trusted reporting, centralized links and all repository policies unchanged.
* Announce November 9 skill Eval coverage enforcement checkpoint
Add a prominent grace-period notice to missing-coverage comments and summaries, allowing authors to proceed only when other required checks pass. Keep coverage warning-only; do not add automatic date-based enforcement. Extend existing renderer, reporter and CLI regressions and document follow-up issue #17402.
* Discover skill-local Eval presence and address workflow review feedback
Remove pipeline discovery requirements from the presence checker and configuration. Preserve warning-only behavior, current lint scope, immutable GitHub reads, and the November 9 enforcement notice. Restore direct action SHA pins, use frozen standalone pnpm installs, separate script tests, and sparse-checkout the trusted reporter. Remove obsolete shared glob and lockfile tooling. Document semantic coverage as a separate advisory follow-up.
* Use Eval dashboard short link
* Keep skill Eval coverage tests portable across synced repositories
Move Tools workflow assertions to eng/common-tests, run the full shared suite in an isolated consumer, and remove README links to unsynced files. Preserve read-only test workflow permissions and frozen dependency installs.
---------
Co-authored-by: helen229 <gaoh@microsoft.com>
Fix auto-release eligibility for shared CI triggers
Require original PR file changes under each package directory before marking artifacts releasable or updating release plans. Preserve validation detection and cover shared triggers, deleted and renamed paths, and path boundaries.
Co-authored-by: Ray Chen <raychen@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Add package name and package version to logs.
* Pass dummy hash when no hash is provided.
* Revert "Pass dummy hash when no hash is provided."
This reverts commit 3449133ed429c799db553ea5e2a96a69aa7cc5ed.
---------
Co-authored-by: Travis Prescott <trpresco@microsoft.com>
* Publish complete eval build results directly to Blob
* Report sanitized Blob publication failure diagnostics
* Opt eval publishing into the AzureStorage network policy
* Publish real skill and live eval results to Blob
* Enable scoped automatic publication for tools main evals
* Remove dedicated synthetic storage smoke mode
* Restrict refresh token destinations and make publisher lock portable
* Reject noncanonical archive identities and clarify publisher boundaries
* Document the production-only dashboard boundary
* Convert Blob publisher scripts to TypeScript
* Notify the dashboard after publishing evaluation results
* Centralize eval publication configuration and simplify pipeline IDs
* Simplify eval publication and close validation gaps
* Simplify eval result publication and strengthen failure recovery
* Use one eval publication opt-in for upload and storage egress
* Keep one main-only eval publication switch
* Use pipeline configuration and Azure CLI for eval publication
* Fix eval publication edge cases and run regressions in PR CI
* Configure eval dashboard endpoints through pipeline variables
* Remove optional eval publication receipts
* Simplify eval bundling and support Node 24 with pnpm
---------
Co-authored-by: helen229 <gaoh@microsoft.com>
New-TestResources: Resolve deployment state instead of blindly retrying on failure
New-AzResourceGroupDeployment retries reused the same -Name as the
original deployment. If the original was still running server-side
(e.g. a long-running template), the retry collided with it
(Code=DeploymentActive) and every retry failed the same way.
Resolve-DeploymentAfterFailure now asks ARM for the existing
deployment's actual state before deciding what to do:
- still running -> poll it to completion (bounded, 60 min timeout)
instead of starting a redundant concurrent deployment, which
could otherwise cause resource-level conflicts and leave an
orphaned duplicate deployment record behind
- already succeeded/failed/canceled -> return that result as-is so
normal failure handling reports the real error instead of masking
it with a retry
- not found at all -> safe to submit a new deployment under a
unique name, truncating the base name so the result stays within
ARM's 64-character deployment-name limit
Also resets $deployment to $null at the start of each
template-file iteration so a stale value from a previous template
can't leak through if the try block throws without assigning a new
one.
Co-authored-by: Connie Yau <conniey@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ce6c37d3-fb5b-4100-8c45-fc8ee31b4170
Accept list of files to check from stdin
Invoke-Cspell.ps1 works fine when piping a list of file names to it from
within PowerShell, but not another shell like bash. This is the fix.
Co-authored-by: Heath Stewart <heaths@microsoft.com>
