This isn’t a Copilot ad — just a dev experiment gone surprisingly well. I built a custom code reviewer powered by GitHub Copilot… for Azure DevOps. Because I thought: 💭 “What if Copilot could help review PRs like a human? On Azure DevOps?”
Turns out, it can — with a bit of hacking and a lot of fun.
🎯 The Goal
I wanted a bot that could:
- Analyze the changes in an Azure DevOps Pull request
- Identify the delta between the source and the destination branches
- Ask GitHub Copilot Chat for feedback on the diffs
- Post review comments back into the PR, as if a human did it
It had to:
- Respect context (don’t comment on unchanged code)
- Be deterministic enough not to annoy reviewers
- Be easy to turn off/delete if not useful
How it was built :
🧱 1. VS Code Extension: Kicks Things Off
This is where the magic starts.
The extension registers a Chat Participant — basically a custom Copilot Chat plugin — which listens for commands like:
“Review this PR for quality, maintainability, and safety.”
Once triggered, the participant passes control to the core logic module — the Code Review Controller.
Reference I used to add a chat participant to your Copilot chat using VS Code extension : https://youtu.be/YI7kjWzIiTM?si=teaE48hIiXlNmB-B
⚙️ 2. Code Review Controller: The Brain
This part handles the heavy lifting:
- Input Validation The method checks if last commit content and file Path are present. If it’s a new file (mainBranchContent === null), it segments the full file for review.
- Diff Calculation Using the diff library, it identifies what changed line-by-line between the main branch and the new commit.
- Chunking and Context Changes are split into chunks of 5 lines (CHUNK_SIZE) with 2 lines of context above and below (CONTEXT_LINES) to ensure GPT gets enough surrounding code to reason about the logic.
- Prompt Creation For each chunk, a prompt is crafted with instructions:
- Copilot Request The code uses the vscode.lm.selectChatModels() API to select a GPT-4 model from the Copilot family and sends the prompt for review.
- Comment Parsing & Publishing Once a response is received, it’s sanitized and parsed. The controller filters out empty suggestions and then uses the AzureDevOpsAPIService to post comments directly on the PR with proper line numbers and offsets.
These are the heart of the review — and where prompt tuning mattered most.
🌐 3. Azure DevOps API: Getting What We Need
To drive all of this, we fetch PR data using Azure DevOps APIs:
- Fetch Pull Request Details
- Get All Commits in the PR
- Get Changes for Commit (i.e., diffs)
- Fetch File Content (to extract surrounding code for context)
- Publish Review Comment (to push Copilot’s review directly to the PR as inline comments)
The Code Review Controller invokes these as needed, based on PR context.
🏢 4. Azure DevOps: Project Config
The extension also fetches:
- Azure DevOps Config for things like repo URLs, project IDs, and access tokens
This config is used across the board to authenticate API calls and map PRs to repos.
🧠 Final Thoughts
This wasn’t just about getting Copilot to “do a review.” It was about bridging the gap between intelligent code assistance and real-world engineering workflows. By stitching together GitHub Copilot Chat and Azure DevOps, I now have a working prototype that reviews pull requests with context , posts inline comments, and actually helps developers catch issues early.
It’s not perfect (yet), but the early results have been promising.
AI won’t replace code reviewers — but it can take the grunt work off their plate. And that’s a win. 🚀