[ehb54/ultrascan3] a997ae: extrap_c0: fix unweighted GCV wiping out low q; re...
Branch: refs/heads/ehb54-issue-920 Home: https://github.com/ehb54/ultrascan3 Commit: a997aef8d438a5599240a6ef7cbefe853d9604ab https://github.com/ehb54/ultrascan3/commit/a997aef8d438a5599240a6ef7cbefe853... Author: ehb54 <brookes@uthscsa.edu> Date: 2026-07-20 (Mon, 20 Jul 2026) Changed paths: M us_somo/develop/src/us_hydrodyn_saxs_iqq_extrap_c0.cpp M us_somo/somo/doc/manual/somo/somo_saxs_extrap_c0.html Log Message: ----------- extrap_c0: fix unweighted GCV wiping out low q; report real skip causes Reported: "non weighted seems to not work -- q-points could not be extrapolated (fewer than 2 valid data points) and were skipped, regardless of merge etc." Root cause. With unit weights the joint GCV objective on the reciprocal (c/I) axis is dominated by the high-q points, whose values are orders of magnitude larger than the low-q ones. GCV therefore selected lambda ~7.7e6 (effective slope dof 9.8, vs 259/19.7 for the same data weighted), over-smoothing alpha(q) into a near-constant until Iex = (Q - B*alpha)/C went negative starting at the LOWEST q. Every such q was dropped, so the Guinier region was destroyed and Rg blew up to 111 A. merge_ref cannot help because it splices the reference at HIGH q while the damage is at low q. This was previously recorded as "genuine algorithm behavior, validates why sd_weights defaults ON". That was wrong: the same data unweighted with GCV OFF gives Rg 37.5 A with 1 skip, i.e. unweighted fitting is fine -- the regularization was the problem. Fixes: - (C, root cause) In the unweighted case, weight each q by 1/scale^2 where scale is that q's mean |y|. The factor is CONSTANT WITHIN a q, so it leaves every per-q fit (slope R/M and intercept Iex) algebraically unchanged -- it only rebalances the q against one another in the joint penalized solve, restoring the conditioning the 1/sigma^2 path gets for free. Equivalent to assuming constant relative error, the standard default when error bars are absent or distrusted. The weighted path is untouched. - (A) Report the ACTUAL skip cause(s) with per-cause counts. All four causes previously reported as "fewer than 2 valid data points", which is wrong for three of them and hid the real one (non-positive 1/I(0) intercept) -- that message is what sent the original diagnosis down the wrong path. - (B, safety net) If the GCV-smoothed intercept is still non-positive at some q, fall back to the unregularized per-q intercept there instead of dropping the point, and tell the user how many q that happened at (those points are not regularized). Validated on the 7-curve alpha-syn set: recip_uw 506 skips / Rg 111 -> 0 skips / Rg 38.4 (in family with recip 38.1, additive 38.4, nogcv 38.3, unweighted-nogcv 37.5); same for the merge variant. 8 of 9 regression baselines and all 5 SASBDB Table-1 outputs are byte-identical, confirming the default weighted+GCV path is unchanged. Safety net (B) did not need to fire in any test -- it is defensive only. Manual updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> To unsubscribe from these emails, change your notification settings at https://github.com/ehb54/ultrascan3/settings/notifications
participants (1)
-
emre brookes