📉 DevSecOps Metrics: DORA, AppSec Coverage, and Security Debt

Intro: Security programs often drown in raw finding counts. A better operating model combines delivery-flow metrics, control-coverage metrics, and security-debt metrics so that engineering leaders can see whether the organization is simultaneously getting faster, safer, and easier to reason about.

What this page includes

• how to interpret the four DORA metrics in a Product Security context
• how to measure AppSec coverage without pretending every system is equally important
• how to track security debt at the defect and service level
• how to translate these signals into manager and director reporting

Why this page exists

A Product Security team usually gets asked two hard questions:

1. Are we making delivery safer without slowing it down too much?
2. Are we paying down real security debt, or just moving findings around dashboards?

DORA-style delivery metrics help answer the first question.
Coverage and defect-debt metrics help answer the second.

The useful trick is to read them together, not in isolation.

A practical metric stack

Use three layers:

DORA metrics in a Product Security context

The four DORA metrics were not created as “security metrics,” but they are still useful because they show whether security is operating inside delivery flow or outside it.

What a security team should do with DORA

Use DORA to ask these questions:

• Do security checks create useful friction or just hidden queue time?
• Can the team ship a remediation in hours or does it still need a special release ceremony?
• Are security-related breakages concentrated in a few services, modules, or teams?
• Is recovery limited by release process, by missing observability, or by poor ownership?

Security overlays for DORA

DORA alone is not enough. Add a small overlay.

AppSec coverage: two measurements that matter

A lot of teams say “we have coverage” when they really mean “some scanners run somewhere.”

Use two different measurements.

1) Asset coverage

How much of the important estate is actually covered?

Examples:

• percent of tier-1 repos with SAST, secret scanning, and dependency scanning
• percent of tier-1 services with image scanning and runtime owner defined
• percent of public APIs with contract linting and authz review
• percent of critical cloud accounts or subscriptions with posture review enabled

2) Practice coverage

How many expected security practices are actually present per system?

Examples:

• threat model exists
• ownership is defined
• repo protection exists
• CI checks are present
• release evidence exists
• image signing or provenance check exists
• runtime logs are preserved
• exception path exists

A system may have “some scanners” but still have weak practice coverage.

A simple AppSec coverage model

Use weighted coverage rather than flat coverage.

Then score each service against a short list of practices.

Example practice list

• repository secret scanning
• SAST or equivalent code review automation
• dependency or SBOM visibility
• CI quality gate
• release evidence
• threat model or architecture review
• owner and escalation path
• logging and recovery notes

A weighted score is often more honest than “82% of systems covered” because it prevents tiny low-risk repos from dominating the dashboard.

Security debt at the defect level

Security debt is not just “number of open vulns.”
Track the debt in ways that show risk, age, and repair efficiency.

Useful views:

A practical weighted-risk approach

Do not overcomplicate the math. A simple weighted index works:

Weighted Risk Index =
severity_weight × exploitability_weight × asset_criticality_weight

Example weights:

• severity: critical=5, high=4, medium=2, low=1
• exploitability: proven/likely=3, plausible=2, theoretical=1
• asset criticality: tier1=3, tier2=2, tier3=1

This is not a law of nature. It is a prioritization aid.

How to read these metrics together

Good pattern

• deployment frequency stays healthy;
• remediation lead time drops;
• coverage rises in tier-1 systems;
• gate bypass rate is low;
• weighted risk debt trends down.

That usually means security is becoming part of delivery.

Bad pattern

• coverage rises only because more “easy” repos were added;
• critical debt in tier-1 services grows older;
• gate bypasses increase;
• deployment slows but exposure does not meaningfully drop.

That usually means the program is becoming heavier, not better.

Suggested manager dashboard

A manager-friendly monthly view could include:

• DORA four-pack for business-critical products
• percent of tier-1 services with required security practices
• weighted risk debt trend
• median age of critical/high exploitable findings
• gate bypass count and average time to resolution
• secret exposure rate per 1,000 commits
• top five repeat defect classes

Suggested director talking points

Directors usually need business translation more than scanner granularity.

Good statements:

• “Security evidence is now attached to 90% of tier-1 releases.”
• “Median age of exploitable high-risk findings in customer-facing services decreased from 28 days to 11 days.”
• “Coverage growth this quarter came from onboarding tier-1 APIs and regulated repos, not from low-risk estate inflation.”
• “We improved remediation flow without reducing deployment frequency.”

Common mistakes

• using raw finding counts as the main KPI;
• mixing all services into one unweighted average;
• reporting “coverage” without defining coverage of what;
• measuring only scanner activity instead of control adoption and debt trend;
• hiding security-caused delivery friction inside generic pipeline metrics.

Related pages

• 📊 Product Security Maturity, Scale, and Business Translation
• 🧑‍💼 Role-Based KPI Patterns for Product Security
• 🧮 Collecting Product Security Metrics Without ASPM or ASOC
• 📦 Director Packs, Scorecards, and Review Cadence