UQO Working Papers · WP16 · v1.1
Does the Sales Comparison Approach Actually Work? A Large-Scale Point-in-Time Validation of Comparable Selection, Price Adjustments, and Value Reconciliation
Evidence from 391,000 Single-Family Transactions in Quebec, 2021–2026
- Year
- 2026
- First release
- October 2026
- Latest revision
- October 2026
- Publication status
- Working paper
Adjust, choose similar comparables, and use twenty of them, not three
On 189,354 masked target sales, the best transparent rule frozen in 2024 reaches an 11.1% median absolute error on 2025–2026, within 2.2 points of a quarterly-retrained gradient-boosting model that uses the assessed value.
Abstract
The sales comparison approach is the dominant method of residential valuation, yet its operating rules — how many comparables to use, how far and how far back to search, which adjustments to apply, how to weight and reconcile — rest largely on professional convention. This paper subjects these rules to a population-scale, point-in-time backtest on 390,996 arm's-length single-family transactions in Quebec (2021–2026) matched to the municipal assessment roll. For each of 189,354 target sales in 2024–2026 the price is masked and 2,256 configurations of the approach, together with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, are applied using only information available on the valuation date; configurations are selected on 2024 and evaluated out of sample on 2025–2026, and paired differences are tested with a cluster bootstrap over municipality–quarter cells.
Adjustments dominate: the median absolute error of the five nearest sales falls from 18.1% with a time adjustment only to 13.0% with a hedonic grid, and a stylised conventional grid captures most of that gain. Selecting physically similar rather than merely nearby comparables delivers the same gain without any adjustment. The optimal number of comparables is about twenty once a grid is applied, with no penalty up to fifty; short windows are dominated and the search radius matters little. Weighting and reconciliation rules matter little.
The best transparent configuration chosen in 2024 reaches 11.1% out of sample, against 9.0% for a quarterly-retrained gradient-boosting model using the assessed value; the cost of transparency, 2.2 points, is precisely estimated and narrows in homogeneous submarkets, and the chosen rule ranks second among 1,370 configurations out of sample. Accuracy is governed primarily by submarket liquidity, and a wide-net rule retaining the most similar sales adapts to it on its own.
Key findings
390,996
arm's-length single-family sales, Quebec, 2021–2026
2,256
configurations of the approach backtested point in time
18.1% → 13.0%
five-nearest-sales MdAPE, time-only versus hedonic grid
~20
optimal number of comparables once a grid is applied
11.1%
out-of-sample MdAPE of the best transparent rule
2.2 pt
cost of transparency versus gradient boosting at 9.0%
Scientific contributions
- A population-scale, point-in-time backtest
- Every target sale is valued with only the information available on its valuation date — a point-in-time index, quarterly hedonic grids and quarterly-retrained benchmarks — on 124,559 out-of-sample sales of 2025–2026, with 2,256 rule configurations selected on the separate 2024 validation year.
- Adjustments dominate
- With five nearest sales, moving from a time adjustment only to a hedonic grid lowers the median absolute error from 18.1% to 13.0% (paired difference −5.1 points, 95% CI [−5.26; −4.88]); a stylised conventional grid captures most of that gain.
- Similar beats near
- Five physically similar comparables without any physical adjustment reach the same 13.0% as five nearest with a grid, whereas five nearest within ten kilometres and a time adjustment stay at 17.6%.
- More comparables than convention prescribes
- With a grid, the error falls from 16.5% (k = 1) to 13.8% (k = 3), 13.0% (k = 5), 12.4% (k = 10) and 12.1% at k = 20–50, in sparse and dense markets alike; windows should be wide and the radius barely matters.
- The price of transparency is precisely estimated
- The best transparent rule (compromise selection, k = 30, 25 km, 24 months, hedonic grid) reaches 11.1% [10.8; 11.5] versus 9.0% [8.8; 9.2] for gradient boosting with the assessed value — a gap of 2.2 points (t = −36) that narrows to 1.4 points in the most homogeneous tercile. Validation-to-test rank correlation is 0.996 and the chosen rule ranks 2nd of 1,370.
- Liquidity governs accuracy
- The same hedonic-grid rule yields a 10.2% error where at least 200 sales occur within 2 km per year and 25.3% where fewer than five do; weighting and reconciliation rules change the error by tenths of a point.
Methodology
Methodology (4)
Point-in-time index and quarterly hedonic grids
A provincial and regional hedonic price index and hedonic adjustment grids re-estimated each quarter from 2024-01 to 2026-07 using only sales recorded before the valuation date.
Factorial backtest of the approach
Selection rule (nearest, most similar, compromise, least adjusted) × number of comparables (1–50) × radius × window × adjustment scheme (none, time, conventional grid, hedonic, regional, local ridge, GBM, assessment ratio) × weighting × reconciliation, sharded across five MacLustr nodes.
Selection on 2024, evaluation on 2025–2026
Error surfaces and marginal effects of each rule are mapped on the validation year; the winner is frozen and compared out of sample with hedonic, gradient-boosting, assessment-ratio and hybrid benchmarks, then decomposed by liquidity, region, price stratum and property type.
Cluster-robust inference and selection audit
Paired differences tested with a 999-replication bootstrap over municipality × quarter cells and Diebold–Mariano-type t statistics; sensitivity to four cluster definitions, to the arm's-length screen R8, and to seasonal bias by calendar quarter; stability of the validation-to-test ranking of all configurations.
Research data
- Vrai Prix transaction and assessment-roll snapshot
- Snapshot of 10 October 2026: 766,596 residential transactions and 3.75 million assessment-roll units; single-family sales matched to the roll and cleaned by logged rules R0–R11, from 456,950 to 390,996 sales (January 2021 to September 2026), median price $400,000.
- Backtest splits
- Train 201,642 sales, validation 64,795 (2024) and test 124,559 (2025–2026); 189,354 target sales with masked prices; roll attributes (area, lot, age, storeys, attachment, assessed value) drive selection and adjustment.
Reproducibility
- Pipeline 01_dataset → 06_report run on the MacLustr node M2U64 (about 30 minutes, or 15 shards over five nodes with run_distributed.sh), plus 07_revision_analyses.py on the laptop (about 80 s)
- Every number in the paper is a LaTeX macro regenerated from out/<version>/ and results/revision-2026-10-17/ by make_paper_tables.py; 33 generated tables and 7 figures
- French protocol and report (docs/PROTOCOLE.md, docs/RAPPORT.pdf), full LaTeX source with an anonymous variant for journal submission
- Version 1.0 frozen as docs/main_pre-revision-20261017.pdf; the 2026-10-17 revision audit trail in docs/CHANGELOG-REVISION.md leaves every pipeline number unchanged
Research artifacts
Version history
| Version | Date | Description | File |
|---|---|---|---|
| v1.1current | October 2026 | Current manuscript as published on www.spboucher.ai (revised edition after the 2026-10-17 scientific review for papers 1–17). | PDF · 52 p. |