feat(color): PercentileNormalize for percentile-based contrast limits#725
Open
timtreis wants to merge 8 commits into
Open
feat(color): PercentileNormalize for percentile-based contrast limits#725timtreis wants to merge 8 commits into
timtreis wants to merge 8 commits into
Conversation
…t the resolved norm _resolve_continuous_norm preserved the norm subclass (#722) but still derived vmin/vmax from the full data range, so a LogNorm over data containing 0 or negatives produced LogNorm(vmin<=0) and raised 'Invalid vmin or vmax' on render. Derive the range from strictly-positive finite values for LogNorm (mirrors matplotlib's LogNorm.autoscale_None). Once the subclass is preserved, the continuous colorbar sites must attach the resolved norm, not a default linear Normalize: - shapes fill: set_norm(used_norm) instead of set_clim. - labels: imshow of a pre-baked RGB image cannot carry a non-linear norm, so display the RGB without a norm and build the colorbar from a ScalarMappable(norm=used_norm), mirroring the outline path. as_points now carries the upstream colortype instead of re-deriving it from 'source is not None', so an all-NaN ('none') layer is not mislabelled categorical. Adds regression tests feeding LogNorm zeros/negatives/all-NaN (and invoking the norm) plus a shapes colorbar-subclass assertion.
Matplotlib points passed an un-scaled fresh_norm() to ax.scatter and let it autoscale, diverging from shapes/labels: a constant-valued continuous points layer mapped to the cmap floor instead of the [0, 1] reset, and a LogNorm was autoscaled over the raw data (crashing on a 0/negative). Route continuous points through _resolve_continuous_norm so fill and colorbar match the other element types and LogNorm/PowerNorm survive. Kept separate from the core fix as it is the only change that can shift point visual baselines. Adds a regression test for render_points(norm=LogNorm()) over data with a 0.
timtreis
changed the title
Heavy-tailed images (fluorescence, Xenium morphology) render dim because the default per-channel min/max normalization maps the bulk of the signal to near-black — a single bright outlier sets vmax. This addresses the dimness reported in #370. Add PercentileNormalize, a matplotlib Normalize subclass that autoscales vmin/vmax to data percentiles (pmin/pmax) instead of min/max. It plugs into the existing `norm=` argument with no new parameter: a single instance is broadcast and autoscaled independently per channel, and a list applies channelwise limits (length must match the channel count, as already validated). Unify the two normalization code paths: _resolve_continuous_norm (single-channel colorbar + shapes/points/labels) now delegates to the norm's own autoscale_None instead of reimplementing min/max, so PercentileNormalize is honored everywhere. This is behavior-preserving for plain Normalize (min/max) and LogNorm (positive-only domain); an equivalence test locks that. Default behavior is unchanged (still min/max); percentile contrast is opt-in. Usage: import spatialdata_plot as sdp sdata.pl.render_images("morphology_focus", palette=[...], channel=[...], norm=sdp.PercentileNormalize(0, 99.5))
timtreis
force-pushed
the
feat/percentile-normalize
branch
from
1df024f to
bacc738
Compare
timtreis
changed the base branch from
fix/lognorm-norm-and-colorbar-consistency
to
main
…m fill render_points now routes continuous color through _resolve_continuous_norm for the datashader backend too (norm passed at render.py:1459), so the points use the properly scaled norm (degenerate [0,1] reset, LogNorm/PowerNorm preserved, data-range vmin/vmax) instead of an unscaled autoscaled norm. This shifts the datashader continuous-color/reduction/NaN baselines; regenerated from CI.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #725 +/- ##
==========================================
+ Coverage 79.09% 79.20% +0.11%
==========================================
Files 17 17
Lines 4467 4563 +96
Branches 999 1026 +27
==========================================
+ Hits 3533 3614 +81
- Misses 593 599 +6
- Partials 341 350 +9
🚀 New features to boost your workflow:
|
…r autoscaling d5a811a routed continuous points through _resolve_continuous_norm, but the resolved norm was passed to BOTH backends. For datashader the displayed value is the per-pixel aggregate (count/sum/reduction), whose range differs from the per-point color vector — applying a color-vector-derived vmin/vmax mis-scaled the colorbar and painted empty pixels (purple background) instead of leaving them transparent. Scope the resolved norm to the matplotlib backend (where each point is colored by its own value); datashader keeps the un-scaled fresh_norm so _apply_ds_norm autoscales to the aggregate (honoring any explicit vmin/vmax) — the pre-d5a811a behavior. Reverts the datashader point baselines regenerated in error.
Lock the rendered output of percentile-based contrast: a single PercentileNormalize broadcast across channels, and a channelwise list. Baselines generated from CI.
…omments - PercentileNormalize.autoscale_None now honors masked arrays (matplotlib parity): masked/NaN/inf are excluded from the percentile computation. Regression test added. - Document that the datashader backend autoscales to the aggregate, not the percentiles. - Add PercentileNormalize to the API docs (it was cross-referenced but unrendered). - Trim comments that restated the code in _render_points / _resolve_continuous_norm and hoist the duplicated PercentileNormalize import in the image tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the dimness reported in #370. Heavy-tailed images (fluorescence, Xenium morphology) render dim because the default per-channel min/max normalization maps the bulk of the signal to near-black — a single bright outlier sets
vmax.Adds
PercentileNormalize, amatplotlib.colors.Normalizesubclass that autoscalesvmin/vmaxto data percentiles (pmin/pmax) instead of min/max. It plugs into the existingnorm=argument — no new parameter:Also unifies the two normalization code paths:
_resolve_continuous_norm(single-channel colorbar + shapes/points/labels) now delegates to the norm's ownautoscale_Noneinstead of reimplementing min/max, soPercentileNormalizeis honored everywhere. This is behavior-preserving for plainNormalize(min/max) andLogNorm(positive-only domain) — locked by an equivalence test.Default behavior is unchanged (still min/max); percentile contrast is opt-in. No deprecation needed.