AI Project Management vs Traditional Project Management: What's Actually Different

One of the most useful reframes for anyone moving from traditional software projects to AI projects is this: *most of what you know still applies, but the places where it breaks are exactly the places that matter most.*

This article maps the differences directly — role by role, practice by practice — so you can see what transfers and what you need to learn.

## The core difference: deterministic vs. probabilistic systems

Traditional software: *you specify the behavior, engineering implements it, it either does the thing or it doesn't.*

AI software: *you specify the constraints, engineering trains or configures the model, and it produces outputs within a probabilistic range that you have to evaluate.*

This single difference cascades into most of the specific changes below.

---

## Product Manager / PO

| Traditional | AI project | |---|---| | Writes user stories: "As a user, I want to..." | Writes behavioral contracts: "The AI may... The AI must not... When X, a human decides" | | Acceptance criteria: pass/fail | Eval threshold: percentage compliance against rubric | | Owns the PRD | Owns the Specify document (Spec Owner role) | | Signs off when feature is built and QA passes | Signs off when eval threshold is met and GO verdict is issued | | "The system shall display Y" | "The AI may decide Y when confidence ≥ 80%; below that, escalate" | | Manages feature scope | Manages behavioral scope — what the AI is and isn't allowed to do |

**What stays the same**: Stakeholder management, roadmap prioritization, user research, strategic framing, cross-functional coordination.

**What's new**: Spec Owner pattern, ADLC phase ownership, go/no-go decision-making, incident policy ownership.

---

## Business Analyst

| Traditional | AI project | |---|---| | Requirements gathering: what must the system do? | Requirements gathering + 6 additional sections AI systems need | | Functional requirements document | Specify document (with PM) + AI requirements addendum | | "The system shall..." | "The AI must always... The AI must never... When X, handoff to human" | | Acceptance criteria based on behavior | Acceptance criteria based on output quality metrics | | Data requirements: what data feeds the system? | Data quality requirements: what data was the model trained on, and what inputs will cause it to fail? |

**The 6 additional sections** a traditional requirements document misses for AI: 1. Probabilistic output range (what does "correct" mean at each confidence level?) 2. Human-handoff triggers (when must a human take over?) 3. Fallback behavior (what does the AI say when it can't answer?) 4. Guardrail conditions (what can the AI never do?) 5. Data quality requirements (what inputs will cause the model to degrade?) 6. Eval threshold (what pass rate does the model need to ship?)

**What stays the same**: Stakeholder interviews, facilitation, documentation, process mapping, gap analysis.

**What's new**: Writing the 6 additional AI requirement sections, contributing to the Specify document, defining the golden set for QA.

---

## QA Engineer

| Traditional | AI project | |---|---| | Write test cases | Design an evaluation plan | | Pass/fail testing | Probabilistic scoring against a rubric | | "Does it do the thing?" | "Does it do the thing well enough, often enough, safely enough?" | | Bug report | Eval report with dimension scores and verdict | | Regression testing | Red-teaming and golden set regression | | Automated test suite | Automated checks + LLM-as-judge + human review | | Feature is done when tests pass | Feature is done when eval threshold is met and GO/COND/NO-GO verdict is issued |

**The new artifacts QA owns in an AI project**: - **Golden set**: 20–50 representative inputs with expected output criteria - **Rubric**: Scoring dimensions (accuracy, safety, groundedness, tone, consistency) with weights - **Red-team test suite**: Adversarial inputs designed to find failure modes - **Eval report**: Scored results across all dimensions, with verdict

**What stays the same**: Systematic thinking, edge case identification, process rigor, documentation, the instinct that something isn't ready.

**What's new**: The vocabulary and tools of AI evaluation — golden sets, rubrics, LLM-as-judge, eval thresholds, red-teaming.

---

## Delivery Lead / Scrum Master

| Traditional | AI project | |---|---| | Definition of Done: feature is built and QA passes | Definition of Done: eval threshold is met, GO verdict exists, production readiness checklist is complete | | Sprint ceremonies + velocity tracking | Same, plus phase gate checks | | "Are we on track to ship?" | "Has the Specify document been written? Does the eval plan exist? Has the GO verdict been issued?" | | Burndown chart | Burndown chart + eval pass rate | | Incident response: rollback the feature | Incident response: behavioral analysis — did the AI violate its Spec? Pull the model or adjust the guardrails? |

**The new phase gates** delivery leads check in an AI project: - Framing → Architecture: Specify document exists and is signed off - Architecture → Build: Eval plan exists and has been approved - Eval → Production: GO verdict exists with documented evidence - Production: Monitoring rubric is in place and being reviewed

**What stays the same**: Sprint management, stakeholder communication, risk tracking, dependency management, retrospectives.

**What's new**: ADLC phase gates, production readiness checklist, AI-specific incident playbook ownership.

---

## The patterns that trip people up

"We'll figure out the eval criteria at the end."

This is the most common mistake. If you don't define the eval threshold at Framing, you'll negotiate it under launch pressure. The model you've built will influence what threshold you're willing to accept. Set it before the build.

"The demo works, so we're ready to ship."

A model that works in a demo has been shown favorable inputs. The eval phase is specifically designed to stress-test it with the full range of inputs it'll see in production. "Works in a demo" is never a sufficient go/no-go criterion.

"QA just needs to sign off like they always do."

In an AI project, QA's sign-off is a GO verdict with a documented eval report. It's not an implicit green light when bug tickets fall below a threshold. The process is different.

"The PM owns the requirements, so the AI requirements are handled."

The PM owns the Specify document, but the BA contributes the 6 AI-specific sections (the ones traditional requirements docs miss). QA contributes the eval threshold definition. It's a three-way contribution, not a handoff.

---

## The opportunity

Here's the underappreciated flip side of all this: every one of these "what's new" items is a gap in most teams right now. Companies are running AI projects without Specify documents. Without defined eval thresholds. Without production readiness checklists. Without QA engineers who know what a GO verdict looks like.

The PMs, BAs, QA engineers, and delivery leads who learn the ADLC vocabulary and process in 2026 aren't just keeping up — they're ahead of most of the industry.

[Start with the free 90-minute seminar →](https://forwarddeployed.app/seminars)