UQO Working Papers · WP26 · v1.0
Can Autonomous AI Agents Discover the Fundamental Value of Real Estate? Evidence from Quebec Transactions and Multi-Agent Deliberation
- Year
- 2026
- First release
- October 2026
- Latest revision
- October 2026
- Publication status
- Working paper
Deliberation among LLM valuation agents manufactures agreement, not accuracy
In a preregistered experiment on 631,866 Quebec sales, a two-round debate scores 0.158 mean absolute log error against 0.159 for independent agents and 0.157 for a placebo committee; rounds halve disagreement without moving error, and every preregistered contrast includes zero.
Abstract
Committees of role-specialised language-model agents that debate before answering are increasingly proposed as valuation tools. This paper asks whether the debate adds information. In a preregistered experiment on 631,866 arm's-length residential sales registered in Quebec between 2021 and 2026, evaluated under a leakage-audited as-of information set, a hedonic regression, a hierarchical repeat-sales model, gradient boosting and a conformal automated valuation model (AVM) are compared with a single language model, twelve independent role-specialised agents, structured debate of one to four rounds, debate with a blinded skeptic and deterministic verification, a placebo committee and a cost-matched single agent, all fed identical evidence packets.
A model of valuation as decentralised information aggregation shows that opinion exchange contracts dispersion without reducing error, that evidence exchange helps only outside the common view, and that no linear deliberation beats the best round-0 pool. The data agree: on the test subsample the debate reaches a mean absolute log error of 0.158 against 0.159 for the silent committee, 0.157 for the placebo and 0.163 for the cost-matched agent; every preregistered paired contrast includes zero (Holm-adjusted p = 1), and rounds halve the agents' disagreement without moving accuracy.
Committees match the AVM (0.171 on the full test partition) but state intervals about 30 percent too narrow, covering 86–89 percent of prices at a nominal 90 percent, and in the 2026 holdout every agentic arm overvalues by ten log points where the AVM is unbiased. A two-round debate costs about 1.5 US dollars and 5.6 minutes per property. Forecasting accuracy is kept distinct from fundamental value, which is triangulated with 92,089 resales, rents and user cost; the human-appraiser arm is pending.
Key findings
631,866
registered Quebec residential sales, 2021–2026
0.158 vs 0.159
mean absolute log error, debate vs silent committee
[−0.006, 0.005]
95% interval of the debate contrast, Holm p = 1
0.171
AVM mean absolute log error, full test partition
+0.10
log-point overvaluation of every agentic arm in 2026
$1.52
cost of a two-round debate per property
Scientific contributions
- A design that isolates communication
- The independent committee supplies the round-0 estimates of the debate, a cost-matched single agent supplies the same tokens without roles, a placebo committee the same views without mandates; the four primary contrasts were preregistered and frozen before any test property was valued.
- A precise null on deliberation
- Debate improves on the silent committee by 0.001 log points (95% interval [−0.006, 0.005]), on the placebo by nothing ([−0.005, 0.006]), on the cost-matched agent by 0.006 ([−0.017, 0.004]); in the 2026 holdout and prospective set the debated committee is marginally worse.
- Rounds contract dispersion, not error
- As the aggregation theory predicts, the interquartile range of agents' estimates roughly halves after the first round and keeps falling (anchoring index 0.7–0.8) while the error of the median does not move.
- Committees match the AVM but misstate uncertainty
- Agentic arms are statistically indistinguishable from the conformal AVM on the test subsample, yet their intervals are about 30 percent narrower, covering 86–89 percent at nominal 90 (78–81 percent at the small-model tier).
- Common-model drift in the 2026 holdout
- Every agentic arm overvalues by ten log points where the AVM has bias 0.002, a shared error that no committee size or deliberation removes.
- Fundamental value kept separate from price
- On 92,089 sales with a later resale, the boosted baselines lie closer to the market-adjusted resale value than the realised first price does; the same test for the agentic arms is specified and deferred.
Methodology
Methodology (5)
Information-aggregation theory
Eleven propositions, each mapped to a preregistered contrast: correlated-signal pools have an MSE floor, revision toward a committee statistic contracts dispersion by (1−λ)^r without changing error, and no linear deliberation beats the optimal round-0 pool.
Ten arms on identical evidence
Four tuned baselines (hedonic ridge, hierarchical repeat-sales, LightGBM, conformal AVM) and six agentic arms (single LLM, twelve independent roles, 1–4 round debate, debate with blinded skeptic and verification, placebo roles, token-matched single agent) at matched list prices across two providers.
Preregistered paired inference
Primary endpoint mean absolute log error; paired block bootstrap over municipality-by-month cells (2,000 draws) with Holm correction over four primary contrasts; protocol frozen with a digest before any test property was valued.
Calibration and faithfulness audits
Coverage at 50/80/90/95 percent, PIT histograms, verification that every cited evidence identifier exists in the packet, that 96–99 percent of adjustments lie within forty percent of the raw comparable price and that about 99 percent of points lie inside the comparables' hull.
Cost and run-to-run variation
Full cost ledger per arm (about $0.02 for one model, $0.23 for the silent committee, $1.52 and 5.6 minutes for a two-round debate) and measured non-determinism of hosted models (mean run-to-run change 0.007 log points).
Research data
- Registered Quebec residential sales
- 631,866 arm's-length sales (422,542 houses, 122,976 condominiums, 86,348 small plexes) from the public land register joined to the municipal assessment roll, January 2021 to October 2026; chronological splits (train to 2024-06, validation to 2025-03, test 2025-04 to 2025-12, 2026 holdout), a three-region spatial holdout and a prospective set of 1,182 post-freeze registrations.
- Evidence packets and external layers
- Per-property packets with roll attributes, up to twelve comparables sold before the valuation date, local market state and flood, transit and census context; coordinates rounded to three decimals, no addresses, nine leakage checks enforcing a sale-date-minus-one-day cutoff.
- Resales, rents and user cost
- 92,089 sales with a later resale, rental-board leases and a user-cost diagnostic used to triangulate fundamental value independently of the first transaction price.
Reproducibility
- Preregistration in docs/preregistration.md and protocol freeze results/freeze.json (2026-10-28, digest f0e9d6a9…); role prompts released with their hashes
- Full Python pipeline (scripts 01–62: data, features, baselines, runs, evaluation, theory, fundamental value, prospective set) driven by a Makefile; 143 tests on synthetic data with a mock provider
- Labelled synthetic dataset and mock LLM adapter let the whole pipeline run without API keys or private microdata; live-run predictions, manifests and ledgers are committed
- All figures and LaTeX tables regenerated by scripts; full LaTeX source of the 129-page paper with proofs, prompts and output contract in the appendices
Research artifacts
Version history
| Version | Date | Description | File |
|---|---|---|---|
| v1.0current | October 2026 | Current manuscript as published on www.spboucher.ai (revised edition after the 2026-10-17 scientific review for papers 1–17). | PDF · 129 p. |