Methodology: sources, math, and limitations
Research findings on CodeSmellCost.com cite a named source, and every dollar band comes from the cost model set out here. This page is the source list, the per-component formula, the limitations, and the refresh cadence. If a number on the site does not check out against this methodology, email [email protected] and we will correct it.
The methodology rests on three layers: the canonical software-engineering references (Fowler, Martin, Beck, Feathers, McConnell, Ousterhout) for vocabulary and pattern catalogue; peer-reviewed empirical research (Khomh, Palomba, Tufano, Bird, Tornhill and Borg) for the smell-to-defect evidence; and the Stripe Developer Coefficient for the productivity-cost translation that makes the dollar figures defensible to a finance team.
| Source | Type | Refresh | What we take |
|---|---|---|---|
| Fowler, Refactoring (2nd ed., 2018, Addison-Wesley) | Canonical reference | Static (book) | The 22-smell taxonomy, the refactoring catalog, and the prose definitions. Every per-smell page on the site cites the relevant chapter or section page reference. |
| Robert C. Martin, Clean Code (2008, Prentice Hall) | Canonical reference | Static (book) | Naming discipline, function-length guidance, and the Single Responsibility Principle. Cited alongside Fowler on the home page and /the-22-smells, with the explicit caveat that some Clean Code guidance is contested by Ousterhout. |
| Kent Beck, Test-Driven Development: By Example (2002, Addison-Wesley) | Canonical reference | Static (book) | TDD as a precondition for cheap refactoring. Cited in /refactoring-roi as the answer to the CFO question 'how do you refactor safely without breaking everything?'. |
| Michael Feathers, Working Effectively with Legacy Code (2004, Prentice Hall) | Canonical reference | Static (book) | Seams, characterisation tests, and the legacy-code refactoring playbook. Cited in /case-studies and /refactoring-roi for the practical 'how do you start refactoring a smell-dense codebase?' question. |
| Steve McConnell, Code Complete (2nd ed., 2004, Microsoft Press) | Canonical reference | Static (book) | Empirical defect-density data on function length, complexity, and naming. Cited on /pr-review-time and /cost-model for the cognitive-complexity to defect-rate translation. |
| John Ousterhout, A Philosophy of Software Design (2018, Yaknyam Press) | Counterpoint reference | Static (book) | The structural counterpoint to Clean Code's small-function gospel. Deep modules over shallow ones. Cited prominently on /duplicate-code (the DRY counterpoint) and /when-smells-are-ok. |
| Tufano et al., ICSE 2015: 'When and Why Your Code Starts to Smell Bad' | Peer-reviewed research | Static (published) | Mined over 0.5M commits in 200 open-source projects and manually analysed 9,164 smell-introducing commits: most smells arrive when a file is created, during enhancement or new-feature work, and in the month before a release. Cited on /bug-rate-correlations and /when-smells-are-ok. It studies when smells are introduced, not their effect on faults. |
| Bird et al., FSE 2011: 'Don't Touch My Code!' | Peer-reviewed research | Static (published) | On Windows Vista and Windows 7, components with more low-expertise contributors and a smaller top-owner share had more pre- and post-release failures. Cited on /bug-rate-correlations and /god-class for the ownership mechanism. |
| Rahman et al. 2025: 'Software Metric Based Impact Analysis of Code Smells - A Large Scale Empirical Study' | Peer-reviewed research | Static (published) | Large-scale empirical study of how individual code smells affect 25 software quality metrics (size, cyclomatic complexity, coupling). Software: Practice and Experience 55:925-945. Supports the coupling- and complexity-driven mechanism behind God Class and Feature Envy cost. The smell-to-defect association itself rests on Khomh 2012 and Palomba 2018, not on this paper. |
| Khomh et al. 2012: 'An Exploratory Study of the Impact of Code Smells on Software Change-proneness' | Peer-reviewed research | Static (published) | Change-proneness correlation with smell density, especially Shotgun Surgery and Divergent Change. Cited on /the-22-smells and the per-smell change-preventer pages. |
| Stripe Developer Coefficient Report (2018) | Industry research | One-off (2018) | The macro number: $85 billion per year lost globally to technical debt and bad code (about 42 percent of developer time, more than 17 hours per week). Used as the upper-bound sanity check for our per-team cost ranges. Our team-of-8 estimates summed across smells fall within the Stripe range (our per-smell sum is 15-25 percent of developer time). |
| DORA State of DevOps 2024 | Industry research | Annual | The software delivery performance metrics (deployment frequency, lead time, change failure rate, recovery time) and performance tiers. Cited on /pr-review-time for delivery context. We do not take a rework percentage from DORA. |
| Tornhill and Borg, 'Code Red: The Business Impact of Code Quality' (2022) | Peer-reviewed research | Static (published) | 39 proprietary production codebases measured with CodeScene's Code Health score: low-quality code had 15 times more defects, issues in it took on average 124% more development time, and maximum cycle times were 9 times longer. Cited on the home page, /calculator, /onboarding-drag and /refactoring-roi. |
| Cisco / SmartBear, Best Kept Secrets of Peer Code Review (2007) | Industry research | Static (published) | Review no more than 200 to 400 lines at a time, for no more than 60 minutes; defect detection drops at rates faster than 500 lines an hour. Cited on the home page, /pr-review-time and /calculator for the PR review drag formula. |
| GitClear State of the Code 2024 | Industry research | Annual | AI-assistant copy-paste rate research. Cited on /duplicate-code with the explicit caveat that the AI-assistant-era duplication-rate observation is still being validated. |
| SonarSource, SonarQube documentation | Vendor reference | Continuous | The CodeSmell taxonomy reference (Sonar's internal categorisation), Cognitive Complexity definition (Campbell, SonarSource white paper 2018), and the rule set used by most teams in production. Cited on /tools and per-smell detection sections. |
| Software Engineering Institute (CMU SEI), Technical Debt research program | Academic / institutional | Periodic | The standard taxonomy of technical-debt items and the cost-of-deferral framing. Cited on /refactoring-roi and /cost-model. |
| IEEE SWEBOK (Software Engineering Body of Knowledge, v3) | Academic / institutional | Periodic | The discipline-level definitions of software maintainability, quality attributes, and measurement. Cited on /cost-model and /methodology. |
| Empirical Software Engineering journal (Springer) | Peer-reviewed venue | Continuous | Source venue for Khomh 2012 and several Bavota / Palomba follow-up studies. Treated as the gold-standard publication venue alongside ICSE / FSE / ASE. |
| BLS Occupational Employment and Wage Statistics (Software Developers) | Public statistics | Annual | Where to look up the wage for your own region and occupation before setting the fully-loaded cost input. The calculator default is a round model input, not a BLS figure. |
- Dollar cost ranges per code smell, derived from peer-reviewed and industry-research source pattern.
- Refactoring ROI math (NPV, risk-weighted incident cost, velocity reclaim, retention math) in CFO language.
- Empirical research synthesis on smell-to-defect associations (Khomh 2012, Palomba 2018, Bird 2011).
- Named-incident case studies with sourced dollar figures (Knight Capital, Therac-25, plus three positive examples).
- Detection-tool reference for the eight tools that matter in 2026 (SonarCloud, CodeScene, Sourcegraph, GHAS, CodeClimate, ESLint, PMD, Semgrep).
- Counter-narrative coverage (Ousterhout, when not to refactor, prototype and expiring-code exceptions).
- Executive-committee deliverables (memo template, NPV calculator, prioritisation framework, decision tree).
- Vendor-pricing pages (no /sonarqube-pricing, /jetbrains-pricing, /codescene-pricing, /sourcegraph-pricing). Vendor pricing changes frequently and the brand-pricing SEO surface carries trust-ceiling and Lanham Act exposure.
- Specific compliance-grade certifications or audit frameworks (ISO 25010, CISQ, etc.). Adjacent territory but a different audience.
- Language-specific tutorial content (how to write a class in Java, how to use TypeScript generics). Refactoring patterns are language-agnostic at the conceptual level; per-language deep-dives live elsewhere.
- Framework-level architecture rewriting patterns (microservices vs monolith, event-driven vs RPC). Architectural smells are a distinct surface from code smells.
- Performance-only tuning that does not change structure. Cache eviction, query optimisation, and similar work is performance engineering, not refactoring.
- Vendor-preference rankings. The /tools page covers eight tools by what they detect; we do not rank them with affiliate-driven scores or compute a 'best tool' synthetic verdict.
Team-of-8 baseline
Every per-smell dollar band on the site comes from one model, set out with its inputs on /cost-model, and assumes a team of 8 engineers. Time lost and onboarding scale with team size; the defect component does not. The catalog tables show teams of 3, 5, 8, 10 and 20. Team of 8 is a model choice, not a survey median.
Fully-loaded cost
Model input: $180K per engineer fully loaded (base + benefits + equipment + overhead) is a round default, not a published figure. Set your own in either calculator; per-smell badges use $180K.
Team calculator: PR review drag
SmartBear's study of a Cisco team: review 200 to 400 lines at a time, for no more than 60 minutes. Formula: team_size x prs_per_week x review_hours x smell_overhead_percent x 52 x hourly_rate. Smell overhead percent defaults to 30 percent; that default is our assumption, not a published figure.
Team calculator: incident cost
The empirical defect-association ordering (coupling smells strongest) maps to expected-value incident cost. Tornhill and Borg (2022) found low-quality code had 15 times more defects across 39 proprietary codebases; the model uses your own incident frequency rather than that multiple. Multiply by base incident frequency, average incident hours, and responding-engineer count to get annual expected-value cost contribution.
Team calculator: onboarding drag
No published benchmark measures ramp time by smell density, so extra_ramp_weeks is your estimate. Cost: hires_per_year x extra_ramp_weeks x (fully_loaded_cost / 52).
Per-smell severity tiers
Each smell takes the inputs of its severity tier (Critical, High, Medium, Low): hours lost per developer per week and extra defects a year, plus onboarding weeks for God Class and Inappropriate Intimacy. God Class is the only Critical smell. The inputs are our weights, not measured figures; /cost-model lists every one and works the God Class example through.
Each page's Updated date and its schema dateModified come from the site's version history at build, so they move only when that page changes. The Data verified badge at the top of /about and /methodology is a separate record: the last full re-check of numbers, source URLs and tool pricing. It moves only when that re-check is done.
Out-of-cycle refresh triggers:
- Vendor pricing change announcement (SonarSource, CodeScene, Sourcegraph, JetBrains).
- New peer-reviewed publication with effect-size data on a top-5 smell (we watch ICSE, FSE, ASE, ICSME, and Empirical Software Engineering).
- DORA State of DevOps annual release (typically Q3 each year).
- Stripe Developer Coefficient Report refresh.
- Substantive editorial correction submitted via [email protected].
- The smell-to-defect associations reported in the empirical literature (Khomh 2012, Palomba 2018) are correlations, not causation. The literature is honest about this; we are honest about it in /bug-rate-correlations and /faq.
- The team-of-8 baseline normalises across team sizes; the catalog tables show other team sizes, and the smell profile calculator on /catalog takes your own; the per-smell badge figures assume the baseline.
- Fully-loaded cost of $180K is a US anchor. International teams should adjust. The calculator allows the override; per-smell page badges use the US anchor.
- Tool pricing on /tools is checked when the page is reviewed, not continuously. Vendor pages can change between reviews.
- Open-source-study selection bias applies to several of the cited empirical studies (Tufano 2015, Khomh 2012, Palomba 2018). Proprietary-codebase smell distributions and effect sizes may differ; we note this on /bug-rate-correlations.
Found a number that does not check out, a citation that does not resolve, or a smell missing from the catalog? Email [email protected]. Corrections turn around within 5 business days. The corrected version ships with the next monthly pass and rolls LAST_VERIFIED_DATE forward.