[us-commits] [ehb54/ultrascan3] a997ae: extrap_c0: fix unweighted GCV wiping out low q; re...

[emre brookes]

  Branch: refs/heads/ehb54-issue-920
  Home:   https://github.com/ehb54/ultrascan3
  Commit: a997aef8d438a5599240a6ef7cbefe853d9604ab
      https://github.com/ehb54/ultrascan3/commit/a997aef8d438a5599240a6ef7cbefe853d9604ab
  Author: ehb54 <brookes at uthscsa.edu>
  Date:   2026-07-20 (Mon, 20 Jul 2026)

  Changed paths:
    M us_somo/develop/src/us_hydrodyn_saxs_iqq_extrap_c0.cpp
    M us_somo/somo/doc/manual/somo/somo_saxs_extrap_c0.html

  Log Message:
  -----------
  extrap_c0: fix unweighted GCV wiping out low q; report real skip causes

Reported: "non weighted seems to not work -- q-points could not be extrapolated
(fewer than 2 valid data points) and were skipped, regardless of merge etc."

Root cause. With unit weights the joint GCV objective on the reciprocal (c/I) axis is
dominated by the high-q points, whose values are orders of magnitude larger than the
low-q ones. GCV therefore selected lambda ~7.7e6 (effective slope dof 9.8, vs 259/19.7
for the same data weighted), over-smoothing alpha(q) into a near-constant until
Iex = (Q - B*alpha)/C went negative starting at the LOWEST q. Every such q was dropped,
so the Guinier region was destroyed and Rg blew up to 111 A. merge_ref cannot help
because it splices the reference at HIGH q while the damage is at low q.

This was previously recorded as "genuine algorithm behavior, validates why sd_weights
defaults ON". That was wrong: the same data unweighted with GCV OFF gives Rg 37.5 A with
1 skip, i.e. unweighted fitting is fine -- the regularization was the problem.

Fixes:
- (C, root cause) In the unweighted case, weight each q by 1/scale^2 where scale is that
  q's mean |y|. The factor is CONSTANT WITHIN a q, so it leaves every per-q fit (slope
  R/M and intercept Iex) algebraically unchanged -- it only rebalances the q against one
  another in the joint penalized solve, restoring the conditioning the 1/sigma^2 path
  gets for free. Equivalent to assuming constant relative error, the standard default
  when error bars are absent or distrusted. The weighted path is untouched.
- (A) Report the ACTUAL skip cause(s) with per-cause counts. All four causes previously
  reported as "fewer than 2 valid data points", which is wrong for three of them and hid
  the real one (non-positive 1/I(0) intercept) -- that message is what sent the original
  diagnosis down the wrong path.
- (B, safety net) If the GCV-smoothed intercept is still non-positive at some q, fall back
  to the unregularized per-q intercept there instead of dropping the point, and tell the
  user how many q that happened at (those points are not regularized).

Validated on the 7-curve alpha-syn set: recip_uw 506 skips / Rg 111 -> 0 skips / Rg 38.4
(in family with recip 38.1, additive 38.4, nogcv 38.3, unweighted-nogcv 37.5); same for
the merge variant. 8 of 9 regression baselines and all 5 SASBDB Table-1 outputs are
byte-identical, confirming the default weighted+GCV path is unchanged. Safety net (B) did
not need to fire in any test -- it is defensive only. Manual updated.

Co-Authored-By: Claude Fable 5 <noreply at anthropic.com>



To unsubscribe from these emails, change your notification settings at https://github.com/ehb54/ultrascan3/settings/notifications