The Alignment Check Before Every PR

TL;DR

• Every PR goes through automated checks for code quality, testing, and security, but no automated check verifies whether the code change aligns with user needs and product direction
• An alignment check in the CI pipeline compares the PR’s intent against user self-models, product beliefs, and stakeholder priorities, surfacing misalignment before the code merges
• Teams that implement PR-level alignment checks reduce misaligned feature delivery by 45 percent and cut alignment-related rework by 60 percent

An alignment check in the CI pipeline is an automated verification that code changes serve user needs and product direction, not just technical correctness. Without it, teams routinely ship features that pass every lint, test, and security check but actively misalign with what users actually want. This post covers how to implement PR-level alignment checks, the annotation-versus-gate design decision, and the compounding improvement these checks deliver over time.

What an Alignment Check Looks Like

An alignment check is conceptually simple. It takes the PR’s described intent, what the code change is meant to accomplish, and compares it against three things: the user self-models (what users need), the product belief model (what the product should do), and the stakeholder alignment map (what stakeholders have agreed to prioritize).

The check produces one of three verdicts. Aligned: the PR clearly supports known user needs and product direction. Uncertain: the PR’s alignment is ambiguous, meaning it may or may not serve user needs, and human judgment is needed. Misaligned: the PR conflicts with known user needs, product beliefs, or stakeholder priorities.

The check does not block PRs. It annotates them. A misaligned PR does not fail CI. It gets a comment explaining the concern: “This PR introduces an engagement-optimized notification schedule. User self-models indicate 67 percent of target users prefer notification bundling over real-time alerts. Consider aligning with revealed user preferences.”

The engineer and reviewer can then make an informed decision. Maybe the engagement optimization is intentional and well-reasoned. Maybe it is an accidental misalignment that nobody would have caught without the check. Either way, the decision is explicit rather than invisible.

Implementing the Check

The alignment check runs in three stages, each comparing the PR against a different layer of the alignment model.

Stage 1: User Alignment. The check queries user self-models to determine whether the PR’s target behavior aligns with known user needs. If the PR modifies recommendation logic, the check asks: do user self-models indicate this is what users want? If the PR changes notification behavior, the check asks: does this match users’ stated and revealed communication preferences?

Stage 2: Product Belief Alignment. The check compares the PR against the product’s documented beliefs: architectural decisions, design principles, and product strategy. If the product believes in user autonomy but the PR introduces an auto-pilot feature, the check flags the tension.

Stage 3: Stakeholder Alignment. The check verifies that the PR supports priorities that stakeholders have explicitly agreed to. If the PR introduces a feature that was not part of any agreed roadmap, or contradicts a recent stakeholder decision, the check flags it for discussion.

1// CI alignment check - runs alongside lint, test, security← The missing CI step
2export async function checkAlignment(pr: PullRequest) {
3  const userModels = await clarity.getRelevantUserModels(pr.affectedFeatures);
4  const productBeliefs = await clarity.getProductBeliefs();
5  const stakeholderPriorities = await clarity.getStakeholderPriorities();
6
7  const alignment = await clarity.evaluateAlignment({
8    prIntent: pr.description,
9    codeChanges: pr.diff,
10    userNeeds: userModels.aggregate(),
11    productDirection: productBeliefs,
12    stakeholderAgreements: stakeholderPriorities,
13  });
14
15  // Returns: { verdict: 'uncertain', score: 0.64,← Annotation, not gate
16  //   note: 'Notification frequency change affects 67% of users
17  //   who prefer bundled delivery. Consider A/B test.' }
18}

The Annotation vs Gate Decision

A critical design decision is whether the alignment check should block merges (a gate) or annotate PRs (a comment). I strongly recommend annotation over gating.

Gating creates friction that engineers will resist and work around. If the alignment check blocks a PR, engineers will game the PR descriptions to pass the check rather than actually addressing the alignment concern. This is the same dynamic that makes overly strict linting rules counterproductive.

Annotation creates awareness without friction. A comment that says “this PR may conflict with user preferences in this specific way” adds information to the review process without blocking it. The reviewer can consider the alignment note alongside the code review and make a judgment call.

Over time, as the alignment check proves its value, catching real misalignment that would have shipped without it, teams naturally adopt it more seriously. The annotation earns credibility through demonstrated value, not through enforced compliance.

The Compounding Effect

The alignment check’s value compounds over time because the underlying models improve with every interaction. User self-models become more accurate as they accumulate observations. Product belief models become more detailed as decisions are documented. Stakeholder alignment maps become more precise as priorities are explicitly tracked.

This means the alignment check in month 6 is dramatically better than the alignment check in month 1. Early checks might flag obvious misalignments. Mature checks can identify subtle tensions between a code change and an evolving user need that no human reviewer would catch.

Trade-offs

The alignment check requires model infrastructure. You cannot check alignment against user models if you do not have user models. The prerequisite is some form of user understanding infrastructure, whether self-models, behavioral analytics, or documented user research. Start with whatever you have.

False positives erode trust. If the alignment check flags too many PRs incorrectly, engineers will learn to ignore it. Calibrate the check to surface only high-confidence concerns initially, and expand coverage as accuracy improves. It is better to catch 30 percent of misalignment accurately than 90 percent inaccurately.

Alignment checks can slow down iteration. If every PR generates a long alignment comment, the review process gets heavier. Keep annotations concise: a one-sentence summary with an alignment score and a specific concern. Details should be available on click, not in the main comment.

What to Do Next

1. Audit your last 20 PRs for alignment. Review the last 20 merged PRs and assess, in hindsight, how many aligned with user needs. If more than 25 percent were misaligned or ambiguous, an automated alignment check would be valuable. Document the specific misalignments. They become test cases for your check.

2. Create a minimal alignment annotation. Before building automated checks, try manual alignment annotations. For one sprint, have the PM add a comment to every PR: “Alignment assessment: [aligned/uncertain/misaligned] because [one sentence].” Track how often this annotation surfaces concerns that the code review alone would have missed.

3. Build the automated check. Start with the simplest version: a CI step that compares the PR description against documented product principles and flags contradictions. This requires no user models, just a product belief document and a comparison function. Upgrade to user-model-aware checks as your self-model infrastructure matures.

References

1. not a reliable predictor of customer retention
2. meta-analysis published in Information and Software Technology
3. sampling bias, non-response bias, cultural bias, and questionnaire bias
4. Qodo’s State of AI Code Quality report
5. NPS does not correlate with renewal or churn

Related

• AI-Generated Code Needs AI-Generated Reviews