AI-Augmented Traceability at Industrial Scale: Lessons from SANER 2026

Today, I presented our work on AI-augmented requirements traceability at the SANER 2026 Industry Track in Limassol, Cyprus. The paper, co-authored with Chengrong Lu and Lifu Gong of Molex, is now published in IEEE proceedings.

The central question we explored was simple: can AI meaningfully improve traceability in real industrial systems in practice?

Why Traceability Often Doesn't Happen

In large-scale systems, traceability is widely recognized as critical — for quality, compliance, and maintainability. But in practice, it is often not done.

Why? Because it is too expensive.

In our environment, manual traceability for a single system takes on the order of 10 weeks. At that cost, it becomes a low-priority task which is often skipped entirely.

The result is predictable: unknown gaps between requirements and implementation, hidden risks in production systems, and increasing cost and delay over time.

Industrial Scale Changes Everything

Unlike prior academic approaches that assume pre-structured requirements and clean codebases, we started from unstructured PDF specifications and a production carrier-grade optical networking platform — hundreds of requirements, millions of lines of code, over a decade of evolution.

At this scale, traditional IR approaches achieving 60–75% accuracy are not sufficient.

The Approach: AI-Augmented

We approached traceability as a human–AI collaboration problem, not an automation problem.

The system combines semantic retrieval to narrow candidate code elements, multi-model consensus to classify requirement–code relationships, and human validation applied strategically.

The key insight is: the bottleneck is not AI, it is human validation.

Engineers are already fully occupied with product development. Validation must therefore be prioritized, not maximized. We designed a stratified approach with "unanimous consensus, majority agreement, disagreement" so human effort focuses where it matters most.

Making Quality Measurable

One of the most important shifts is moving from binary traceability ("linked or not linked") to measurable quality. We introduced stratified confidence tiers and semantic classification (IMPLEMENTS, PARTIAL, UNRELATED), allowing teams to understand confidence levels, prioritize validation effort, and manage uncertainty explicitly.

Results

We observed:

•  Approximately 5x speedup in traceability workflows

• 99% extraction faithfulness, validated by independent dual reviewers with 98.7% agreement

• > 80% human-validated correctness on the highest-confidence tier

• Significant improvement in visibility and coverage

The benefits scale with system size: the larger the codebase, the more value the approach provides.

Beyond Traceability: Unexpected Value

What surprised us most was what emerged beyond traceability itself. The system revealed missing implementations, architectural coupling issues, and latent defects not previously documented.

AI-augmented traceability is a compliance tool, it is also a system understanding tool.

Lessons for Practitioners

Three practical lessons stand out:

1. Invest in prompt engineering. Budget 2–3 weeks of iterative refinement. Pilot with your highest-value 20% of requirements before full deployment.

2. Use retrieval before expensive reasoning. Semantic embeddings cost a fraction of LLM calls. Narrow the search space first, then apply classification. This is the key architectural decision that makes the economics work.

3. Track holistic ROI. Direct efficiency gains are measurable, but indirect value (e.g. risk reduction, gap discovery, architectural insight) often exceeds the direct returns.

Limitations and What's Next

This is an initial validation on a single carrier-grade system. We focused human validation on the highest-confidence cases, while other tiers rely on multi-model consensus with targeted review.

Next steps include expanding validation across additional confidence levels, completing the full traceability triangle (Requirement–Code–Test, including backward traceability), integrating into CI/CD workflows, and applying across domains.

Final Thought

AI does not eliminate uncertainty in engineering systems. But it can make that uncertainty visible, measurable, and manageable.

AI scales. Humans ensure correctness.