From e4fb7bd67c720c31c2f6365da0ae778467ec4254 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Tue, 11 Aug 2026 10:17:35 +0000 Subject: [PATCH] docs: add PR review and maintenance skills --- .agents/skills/check-pr/SKILL.md | 114 ---------- .agents/skills/maintain-pr/SKILL.md | 89 ++++++++ .agents/skills/maintain-pr/agents/openai.yaml | 4 + .agents/skills/review-pr/SKILL.md | 205 ++++++++++++++++++ .agents/skills/review-pr/agents/openai.yaml | 4 + 5 files changed, 302 insertions(+), 114 deletions(-) delete mode 100644 .agents/skills/check-pr/SKILL.md create mode 100644 .agents/skills/maintain-pr/SKILL.md create mode 100644 .agents/skills/maintain-pr/agents/openai.yaml create mode 100644 .agents/skills/review-pr/SKILL.md create mode 100644 .agents/skills/review-pr/agents/openai.yaml diff --git a/.agents/skills/check-pr/SKILL.md b/.agents/skills/check-pr/SKILL.md deleted file mode 100644 index 15c76cde1c..0000000000 --- a/.agents/skills/check-pr/SKILL.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -name: check-pr -description: > - Check a single PR's CI status, review comments, and requested changes. - Fix actionable failures and address feedback. "check PR 1234", "what's - the status of my PR", "address review comments on #500". -argument-hint: "" -context: fork ---- - -# Check PR - -Do one pass over PR **$ARGUMENTS**: check CI, read reviews, fix what's -actionable, report status. - -## 1. Gather PR state - -```bash -# Overall state -gh pr view $ARGUMENTS --repo docker/docs --json state,title,url,headRefName - -# CI checks -gh pr checks $ARGUMENTS --repo docker/docs --json name,state,detailsUrl - -# Top-level reviews -gh pr view $ARGUMENTS --repo docker/docs --json reviews,reviewDecision - -# Inline (line-level) comments — NOT included in the above -gh api repos/docker/docs/pulls/$ARGUMENTS/comments \ - --jq '[.[] | {id: .id, author: .user.login, body: .body, path: .path, line: .line}]' -``` - -Always check both the reviews endpoint and the inline comments endpoint. -A review with an empty body may still have line-level comments requiring -action. - -## 2. If merged - -Report the final state. Then check for any unanswered review comments (both -top-level and inline) and reply to each one explaining what was done or that -the issue was addressed in a follow-up. Skip to step 6 after. - -## 3. If closed without merge - -Read the closing context to understand why: - -```bash -gh pr view $ARGUMENTS --repo docker/docs --json closedAt,comments \ - --jq '{closedAt, lastComment: .comments[-1].body}' -``` - -Report the reason. Common causes: rejected by maintainers, superseded by -another PR, closed by automation. - -## 4. If CI is failing - -- Read the failure details (follow `detailsUrl` if needed) -- Determine if the failure is in the PR's changed files or pre-existing -- **Actionable:** check out the branch, fix, commit, push - ```bash - git checkout - # fix the issue - git add - git commit -m "fix: " - git push - ``` -- **Pre-existing / upstream:** note it, do not block - -## 5. If review comments or changes requested - -- Read each unresolved comment -- Address feedback in a follow-up commit -- Push, then reply to each comment explaining what was done: - ```bash - gh api repos/docker/docs/pulls/$ARGUMENTS/comments \ - --method POST \ - --field in_reply_to= \ - --field body="" - ``` -- End every comment reply with an accurate agent-disclosure footer that names - the active coding agent, for example `Generated by Codex` or `Generated by - Claude Code`. -- Resolve each thread via GraphQL after replying: - ```bash - # Get thread IDs - gh api graphql -f query=' - query($owner:String!, $repo:String!, $pr:Int!) { - repository(owner:$owner, name:$repo) { - pullRequest(number:$pr) { - reviewThreads(first:50) { - nodes { id isResolved comments(first:1) { nodes { path } } } - } - } - } - }' -f owner=docker -f repo=docs -F pr=$ARGUMENTS \ - --jq '.data.repository.pullRequest.reviewThreads.nodes[] | select(.isResolved == false) | {id, path: .comments.nodes[0].path}' - - # Resolve a thread - gh api graphql -f query=' - mutation($id:ID!) { resolveReviewThread(input:{threadId:$id}) { thread { isResolved } } } - ' -f id= - ``` -- Re-request review if changes were requested - -## 6. Report - -``` -## PR #$ARGUMENTS: - -**State:** <open|merged|closed> -**CI:** <passing|failing|pending> -**Review:** <approved|changes requested|pending> -**Action taken:** <what was done, or "none needed"> -``` diff --git a/.agents/skills/maintain-pr/SKILL.md b/.agents/skills/maintain-pr/SKILL.md new file mode 100644 index 0000000000..ceb563616a --- /dev/null +++ b/.agents/skills/maintain-pr/SKILL.md @@ -0,0 +1,89 @@ +--- +name: maintain-pr +description: > + Maintain and follow up on a single Docker documentation pull request that + you own or are responsible for updating. Check CI and review feedback, fix + actionable failures, push changes, reply to comments, and report status. + Use for requests such as "babysit this PR", "check the status of my PR", + "fix CI on my PR", or "address review comments on #500". Do not use for + maintainer review of an incoming contribution; use review-pr for that. +--- + +# Maintain PR + +Do one maintenance pass over the specified author-owned PR: inspect its +state, fix actionable failures or feedback, reply to reviewers, and report +the result. This workflow may modify the branch and GitHub because the user +is asking to maintain the PR. Do not apply it to an incoming PR merely +because the user asks to review or assess it. + +## 1. Gather PR state + +```bash +gh pr view <PR> --repo docker/docs --json state,title,url,headRefName,headRepositoryOwner,comments,reviews,reviewDecision +gh pr checks <PR> --repo docker/docs --json name,state,detailsUrl +gh api repos/docker/docs/pulls/<PR>/comments \ + --jq '[.[] | {id, author: .user.login, body, path, line, in_reply_to_id}]' +``` + +Always check both top-level reviews and inline comments. A review with an +empty body may still contain line-level feedback. Confirm that the PR is one +the user owns or is authorized to update before checking out or pushing its +branch. If not, stop and use `review-pr`. + +## 2. Handle terminal states + +If merged, report the final state and identify unanswered review comments. +Reply only when the user remains responsible for follow-up. + +If closed without merge, read the closing context and report the reason. +Common causes include maintainer rejection, supersession, or automation. + +## 3. Diagnose CI failures + +- Read the failure details. +- Determine whether the failure comes from the PR or predates it. +- Fix actionable failures in the PR's changed files. +- Report pre-existing or upstream failures without changing unrelated files. + +Follow repository instructions for formatting, targeted linting, explicit +staging, commits, and pushes. Preserve unrelated working-tree changes. + +## 4. Address review feedback + +Treat every review comment as a claim to verify. Implement it only when the +evidence supports it; explain any evidence-based disagreement. + +After each fix: + +1. Format and validate the changed files. +2. Commit and push the focused change. +3. Reply to every addressed thread with what changed or why no change was + made. +4. End replies with an accurate agent-disclosure footer, such as + `Generated by Codex`. +5. Resolve threads only after replying. +6. Re-request review when appropriate. + +Use the inline comment endpoint to reply: + +```bash +gh api repos/docker/docs/pulls/<PR>/comments \ + --method POST \ + --field in_reply_to=<COMMENT_ID> \ + --field body='<RESPONSE>' +``` + +Use GraphQL to retrieve unresolved review-thread IDs and resolve only the +threads that were addressed. Do not silently fix feedback without replying. + +## 5. Report + +```markdown +## PR #<number>: <title> + +**State:** <open, merged, or closed> +**CI:** <passing, failing, or pending> +**Review:** <approved, changes requested, or pending> +**Action taken:** <changes, replies, and thread resolution, or none needed> +``` diff --git a/.agents/skills/maintain-pr/agents/openai.yaml b/.agents/skills/maintain-pr/agents/openai.yaml new file mode 100644 index 0000000000..54c4fb6d1d --- /dev/null +++ b/.agents/skills/maintain-pr/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Maintain PR" + short_description: "Maintain an authored PR through review and CI" + default_prompt: "Use $maintain-pr to maintain this pull request through CI and review follow-up." diff --git a/.agents/skills/review-pr/SKILL.md b/.agents/skills/review-pr/SKILL.md new file mode 100644 index 0000000000..7f5bfdb59b --- /dev/null +++ b/.agents/skills/review-pr/SKILL.md @@ -0,0 +1,205 @@ +--- +name: review-pr +description: > + Review one or more incoming Docker documentation pull requests as a + maintainer. Independently validate technical claims, assess editorial fit + and information architecture, choose a verdict, and draft exact inline or + PR-wide feedback behind a confirmation gate. Use for requests such as + "review PR 123", "is this PR correct?", "does this information belong + here?", "validate this PR", or "help review backlog PRs". Do not use to + maintain or fix a PR you own; use maintain-pr for that. +--- + +# Review PR + +Review incoming contributions for factual correctness and whether they make +the documentation better as a whole. Treat a technically true addition as +insufficient when it is misplaced, overemphasized, redundant, or unhelpful +to the page's intended reader. + +## Preserve the write boundary + +Perform the review in two phases: + +1. Research the PR, decide a verdict, and present the exact proposed + comment or review text. +2. Wait for explicit user confirmation, then post only the confirmed text. + +Before confirmation, do not post comments, submit a GitHub review, approve or +request changes, resolve threads, push commits, edit labels, or otherwise +mutate GitHub. A request to review or draft feedback is not confirmation to +post it. Ask `Post these comments?` and stop. Treat revisions to a draft as +unconfirmed until the user explicitly asks to post them. + +## 1. Gather the full context + +For each PR, inspect its metadata, body, commits, changed files, checks, +conversation, reviews, and linked issues. Always fetch inline comments +separately because `gh pr view --json reviews` omits them. + +```bash +gh pr view <PR> --repo docker/docs \ + --json number,title,url,state,author,body,baseRefName,headRefName,headRefOid,commits,files,comments,reviews,reviewDecision,statusCheckRollup +gh api repos/docker/docs/pulls/<PR>/comments \ + --jq '[.[] | {id, author: .user.login, body, path, line, side, commit_id}]' +gh pr diff <PR> --repo docker/docs +``` + +Read linked issues and relevant discussion. An issue is evidence that a +reader was confused, but it does not establish the reporter's diagnosis or +justify a new highlighted note by itself. Green CI establishes only that automated +checks passed, not that the content is correct. + +Fetch the PR head when local inspection is useful. Compare it with the +canonical upstream base rather than assuming the local branch is fresh. +Read each changed file in full, not only its diff. + +## 2. Research independently + +Verify every material claim against authoritative sources such as product +source code, upstream documentation, specifications, release notes, or safe +local reproduction. Do not accept the PR description, issue diagnosis, or +existing review feedback as fact. + +Search the documentation for related explanations and canonical pages. Read +`STYLE.md`, `COMPONENTS.md`, and applicable repository instructions. Check +whether a changed file is generated or maintained upstream and identify the correct +upstream repository instead of proposing a local edit. + +Distinguish among: + +- a wrong fact +- a correct fact expressed inaccurately +- a correct fact placed on the wrong page +- content already explained elsewhere +- a real discovery problem better addressed with a short signpost and link +- a request that needs no documentation change. + +If an external claim or replacement URL cannot be verified, report that +limitation instead of guessing. + +## 3. Assess editorial fit + +Apply these questions to each addition: + +- Does it change a reader's decision or next action on this page? +- Is this the canonical page for the concept? +- Is the fact general, or specific to this page, feature, or component? +- Is the information already documented elsewhere? +- Would a concise local signpost to canonical coverage solve the discovery + problem better than duplicating the explanation? +- Is the visual and textual weight proportional to the information's value? +- Does it preserve the page's scope, flow, and character? + +Prefer one coherent explanation in the canonical location. Add local context +only when it helps the reader complete the task at hand. Avoid stray notes, +callouts, and exhaustive edge cases whose prominence exceeds their value. + +## 4. Choose a decisive verdict + +Lead with one of these outcomes: + +- **Approve**: correct, useful, well placed, and ready to merge. +- **Approve with optional polish**: ready to merge; suggestions are genuinely + non-blocking. +- **Focused rewrite**: the underlying need is valid, but wording, scope, + placement, or structure should change before merge. +- **Close / no docs change**: incorrect, redundant, out of scope, or not a + documentation problem. + +Explain the verdict with evidence. When wording is the issue, provide exact +replacement text rather than a vague request to improve it. + +## 5. Place feedback deliberately + +Use an inline comment when the finding is anchored to a narrow changed line +or range and acting on it is local. Examples include an inaccurate sentence, +an ambiguous option description, a broken link, or a precise wording +replacement. + +Use a PR-wide comment for scope, information architecture, overall approach, +multiple intertwined edits, or a proposed replacement section. Do not attach +holistic feedback to an arbitrary line. + +Use both when appropriate: put the overall direction in the PR-wide comment +and line-specific corrections inline. Do not repeat the same point in both. +Consolidate related feedback so the author receives the fewest comments that +remain clear and actionable. + +For every proposed inline comment, resolve and display the current changed +file path and right-side diff line. If the target line is not part of the +current diff or cannot be identified reliably, use a PR-wide comment that +quotes the target text instead. Never guess a line number. + +End comments posted on the user's behalf with an accurate agent-disclosure +footer, such as `Generated by Codex`. + +## 6. Present drafts and stop + +Before any GitHub write, show the review in this form, omitting empty +sections: + +```markdown +## Verdict + +Focused rewrite + +## Findings + +- <finding and evidence> + +## Proposed inline comments + +1. `path/to/file.md:42` + > Exact comment text + +## Proposed PR-wide comment + +> Exact comment text + +Post these comments? +``` + +For multiple PRs, give each PR its own verdict and comment set. Make the +confirmation scope unambiguous. Do not interpret approval of one PR's drafts +as approval to post comments on the others. + +## 7. Post only confirmed feedback + +Immediately before posting, re-fetch the PR head SHA and diff. If either the +head or an inline target changed, stop and show the updated draft or +placement for confirmation. + +Post confirmed inline comments as a single comment-only review when +practical. Use the current head SHA and right-side diff lines: + +```bash +gh api repos/docker/docs/pulls/<PR>/reviews --method POST --input <payload> +``` + +The JSON payload contains `commit_id`, `event: "COMMENT"`, and a `comments` +array whose entries contain `path`, `line`, `side: "RIGHT"`, and `body`. +Submitting a review with `APPROVE` or `REQUEST_CHANGES` requires separate, +explicit user authorization; a verdict alone does not grant it. + +Post confirmed holistic feedback separately: + +```bash +gh pr comment <PR> --repo docker/docs --body-file <file> +``` + +Use a safely created temporary file or API input so Markdown, backticks, and +shell substitutions are preserved literally. Post exactly the confirmed +text. Verify the resulting review/comments and report their URLs and +placements. If GitHub rejects an inline location, do not silently fall back +to a PR-wide comment; report the failure and prepare a revised placement for +confirmation. + +## Definition of done + +- Verify technical claims with authoritative evidence. +- Evaluate usefulness, placement, duplication, and proportionality. +- Give a decisive verdict and exact actionable wording. +- Choose inline and PR-wide placement based on the feedback's scope. +- Show every exact draft and target before any GitHub mutation. +- Post only after explicit confirmation and verify what was posted. diff --git a/.agents/skills/review-pr/agents/openai.yaml b/.agents/skills/review-pr/agents/openai.yaml new file mode 100644 index 0000000000..05b429a008 --- /dev/null +++ b/.agents/skills/review-pr/agents/openai.yaml @@ -0,0 +1,4 @@ +interface: + display_name: "Review PR" + short_description: "Validate incoming documentation pull requests" + default_prompt: "Use $review-pr to validate this incoming documentation PR and draft maintainer feedback."