[release/v7.6] Add GitHub Actions annotations for Pester test failures#26969
Conversation
…6789) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: TravisEz13 <10873629+TravisEz13@users.noreply.github.com> Co-authored-by: Travis Plunk <travis.plunk@microsoft.com>
There was a problem hiding this comment.
Pull request overview
Backports GitHub Actions annotations for Pester test failures to release/v7.6, improving CI failure visibility by emitting ::error workflow commands with file/line locations parsed from Pester stack traces.
Changes:
- Add
Get-PesterFailureFileInfohelper inbuild.psm1to extract file path and line number from Pester 4/5 stack traces. - Update the
process-pester-resultsGitHub Action script to emit::error file=...,line=...annotations for failed test cases.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| build.psm1 | Adds stack-trace parsing helper to locate failing test file/line for annotations. |
| .github/actions/test/process-pester-results/process-pester-results.ps1 | Generates GitHub Actions error annotations for each failed Pester test case. |
| # Create annotation title | ||
| $annotationTitle = "Test Failure: $description / $testName" | ||
|
|
||
| # Build the annotation message | ||
| $annotationMessage = $message -replace "`n", "%0A" -replace "`r" | ||
|
|
||
| # Build and output the workflow command | ||
| $workflowCommand = "::error file=$filePath" | ||
| if ($fileInfo.Line) { | ||
| $workflowCommand += ",line=$($fileInfo.Line)" | ||
| } | ||
| $workflowCommand += ",title=$annotationTitle::$annotationMessage" | ||
|
|
There was a problem hiding this comment.
GitHub Actions workflow commands require escaping in both properties (file/title) and the message (at minimum %, \r, \n; and for properties also : and ,). As written, $annotationTitle contains : and $filePath can contain C:\ (also :) when the workspace-relative conversion doesn't happen, which can break parsing and prevent annotations from showing up. Please add a small escaping helper that follows the Actions command escaping rules and apply it to $filePath, $annotationTitle, and $annotationMessage before emitting ::error ....
| # GitHub Actions expects paths relative to the workspace root | ||
| if ($env:GITHUB_WORKSPACE) { | ||
| $workspacePath = $env:GITHUB_WORKSPACE | ||
| if ($filePath.StartsWith($workspacePath)) { | ||
| $filePath = $filePath.Substring($workspacePath.Length).TrimStart('/', '\') | ||
| # Normalize to forward slashes for consistency | ||
| $filePath = $filePath -replace '\\', '/' | ||
| } |
There was a problem hiding this comment.
The workspace-relative path conversion uses $filePath.StartsWith($workspacePath) with the default string comparison and without ensuring a trailing path separator. On Windows this can fail due to case differences (leaving an absolute C:\... path), and it can also incorrectly match sibling paths that share the same prefix. Consider normalizing both paths (e.g., GetFullPath), ensuring the workspace path ends with a separator, and using an ordinal/ignore-case comparison on Windows before doing the substring.
Backport of #26789 to release/v7.6
Triggered by @daxian-dbw on behalf of @app/copilot-swe-agent
Original CL Label: CL-Test
/cc @PowerShell/powershell-maintainers
Impact
REQUIRED: Choose either Tooling Impact or Customer Impact (or both). At least one checkbox must be selected.
Tooling Impact
Improves CI/CD test failure reporting by adding GitHub Actions annotations. This makes test failures more visible and easier to debug in PR workflows.
Customer Impact
Regression
REQUIRED: Check exactly one box.
This is not a regression.
Testing
Successfully backported to v7.4 and v7.5 previously. Verified by running CI workflows with Pester test failures to confirm annotations are generated correctly.
Risk
REQUIRED: Check exactly one box.
Medium risk as it modifies test infrastructure and CI processing scripts. However, changes are isolated to test result processing and have been validated in v7.4 and v7.5 releases. No impact on runtime or production code.