Data flagging (`ps_eor.flagger`)#

Configurable frequency- and spatial-mode flagging for data cubes.

Flaggers inspect a data cube and a noise proxy, usually a Stokes-I cube paired with Stokes V, then either remove outlying channels / spatial modes (action='filter') or retain them with zero weight (action='zero_weight'). FlaggerRunner applies an ordered pipeline and records the combined selection so it can be replayed on another cube:

runner = FlaggerRunner.load('flagger.ini')
filtered_i, filtered_v = runner.run(i_cube.copy(), v_cube.copy())
filtered_noise = runner.apply_last(noise_cube)

Flaggers may modify their input cubes; pass copies when the originals must be preserved. Pipelines can be assembled in Python or loaded from an INI file.

class ps_eor.flagger.BaseFlagger(**kargs)[source]#

Bases: SimpleConfig

Base configuration shared by all flaggers.

Every flagger has a display name and an action: 'filter' removes selected modes, while 'zero_weight' keeps their data and sets their weights to zero.

do_flag(i_cube, v_cube, verbose=True)[source]#

Apply this flagger to a data cube and its noise proxy.

Parameters:

i_cube – data cube used for Stokes-I statistics and geometry.
v_cube – matching noise-proxy cube, commonly Stokes V.
verbose (bool) – report the number of selected modes.

Returns:

the flagged (i_cube, v_cube).

Return type:

tuple

is_applicable(i_cube)[source]#: Whether this flagger supports the type of i_cube.

class ps_eor.flagger.BaseUVFlagger(**kargs)[source]#

Bases: BaseFlagger

Base class for flaggers that select spatial modes.

do_flag(i_cube, v_cube, verbose=True)[source]#

Apply the spatial-mode mask returned by get_outliers.

get_outliers returns True for modes to flag. Filtering removes those columns; zero-weighting retains the data and changes both cubes’ weight arrays in place.

Returns:: the flagged (i_cube, v_cube).
Return type:: tuple

class ps_eor.flagger.BaseFreqsFlagger(**kargs)[source]#

Bases: BaseFlagger

Base class for flaggers that remove or zero-weight channels.

do_flag(i_cube, v_cube, verbose=True)[source]#

Apply the channel mask returned by get_outliers.

get_outliers returns True for channels to flag. Filtering returns frequency-sliced cubes; zero-weighting retains the channels and changes both cubes’ weight arrays in place.

Returns:: the flagged (i_cube, v_cube).
Return type:: tuple

class ps_eor.flagger.FixedFreqsFlagger(**kargs)[source]#

Bases: BaseFreqsFlagger

Flag configured frequencies or frequency ranges.

Frequencies are specified in MHz as comma-separated values or half-open ranges, for example '121.3,123.1-124.0'. By default every configured channel is flagged. ratio_min_v_var can instead retain a configured block unless its foreground-subtracted noise-proxy variance exceeds the variance outside all configured blocks by that factor.

get_outliers(i_cube, v_cube)[source]#

Channels matching the configured ranges and variance criterion.

Returns:: boolean mask of length n_freqs; True means flag.
Return type:: ndarray

class ps_eor.flagger.UVDirectionFlagger(**kargs)[source]#

Bases: BaseUVFlagger

Flag Cartesian UV cells along a configured angular direction.

The direction is interpreted modulo 180 degrees. extend controls the half-width of the selected UV line relative to the median U-cell spacing.

get_outliers(i_cube, v_cube)[source]#

UV samples intersecting the configured direction.

Returns:: boolean mask of length n_vis; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: Only Cartesian cubes provide the required UV coordinates.

class ps_eor.flagger.SigmaClipper(**kargs)[source]#

Bases: SimpleConfig

Shared robust detrending and one-sided sigma clipping.

The statistic is optionally polynomial-detrended after excluding an initial set of extreme samples. Only positive residual outliers are selected.

get_sigma_clip_mask(x, y)[source]#

Detrend y and select high positive residuals.

Parameters:

x – coordinate used for polynomial detrending.
y – statistic to clip.

Returns:

boolean outlier mask with the shape of y.

Return type:

ndarray

class ps_eor.flagger.FreqsSigmaClipFlagger(**kargs)[source]#

Bases: SigmaClipper, BaseFreqsFlagger

Sigma-clip channel noise or variance.

For each frequency, clip either the estimated SEFD (sefd=True) or the standard deviation across spatial modes. stokes='I' uses i_cube; other values use the noise-proxy v_cube.

get_outliers(i_cube, v_cube)[source]#

Channels with anomalously high SEFD or variance.

Returns:: boolean mask of length n_freqs; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: SEFD estimation requires a Cartesian cube; variance does not.

class ps_eor.flagger.FreqsWeightsFlagger(**kargs)[source]#

Bases: BaseFreqsFlagger

Flag channels whose median weight falls below a smooth trend.

A polynomial is fitted after excluding zero and five-MAD weight outliers. Channels whose median effective weight divided by that trend is below ratio are selected.

get_outliers(i_cube, v_cube)[source]#

Channels whose median effective weight is below its fitted trend.

Returns:: boolean mask of length n_freqs; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: Only Cartesian cubes are supported.

class ps_eor.flagger.UVWeightsFlagger(**kargs)[source]#

Bases: BaseUVFlagger

Flag UV samples with insufficient weight across the band.

For each UV sample, take its minimum effective weight over frequency and compare it with the largest such minimum. Values below threshold times that reference are selected.

get_outliers(i_cube, v_cube)[source]#

UV samples below the relative minimum-weight threshold.

Returns:: boolean mask of length n_vis; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: Only Cartesian cubes are supported.

class ps_eor.flagger.UVSigmaClipFlagger(**kargs)[source]#

Bases: SigmaClipper, BaseUVFlagger

Sigma-clip SEFD or variance across Cartesian UV samples.

The clipped statistic is evaluated per UV sample and detrended against baseline radius. stokes selects 'I', 'V', 'dI', or 'dV'; the d variants first difference adjacent frequency channels.

get_outliers(i_cube, v_cube)[source]#

UV samples with anomalously high SEFD or variance.

Returns:: boolean mask of length n_vis; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: Only Cartesian cubes provide the required UV statistics.

class ps_eor.flagger.LMThetaMaxFlagger(**kargs)[source]#

Bases: BaseUVFlagger

Flag spherical modes beyond a variance-derived angular boundary.

The boundary is inferred from frequency variance as a function of m / ell. th_in defines the inner region used to set the reference variance; relative_threshold selects the low-variance modes from which the limiting angle is estimated.

get_outliers(i_cube, v_cube)[source]#

Spherical modes beyond the inferred angular boundary.

Returns:: boolean mask of length n_modes; True means flag.
Return type:: ndarray

is_applicable(i_cube)[source]#: Only spherical-harmonic cubes are supported.

class ps_eor.flagger.Flag(freqs, uu, vv, idx_uv, idx_uv_zero_weights, idx_freqs, idx_freqs_zero_weights)[source]#

Bases: object

Serializable frequency and spatial masks produced by a flagging run.

idx_freqs and idx_uv select the channels and spatial modes kept from the original cube. The corresponding *_zero_weights masks refer to the retained grid and select samples whose weights are set to zero.

save(filename)[source]#

Save the masks and their coordinate grids to HDF5.

Parameters:: filename – Destination HDF5 filename.

static load(filename)[source]#

Load a Flag from HDF5.

Parameters:: filename – HDF5 file created by save().
Returns:: The stored Flag.

apply(cube)[source]#

Apply these masks to a compatible cube.

The input is copied. Rejected channels and spatial modes are removed, then the zero-weight masks are applied to the retained grid.

Parameters:: cube – Cube with the same original frequency and spatial layout as the cube used to create this flag.
Returns:: A flagged copy of cube.

class ps_eor.flagger.FlaggerRunner(verbose=True, min_weights=2)[source]#

Bases: object

Run an ordered flagger pipeline on two matched cubes.

Every flagger derives a mask from the Stokes-I and noise cubes and applies it to both, keeping their sampling identical. The runner retains the inputs, outputs, and combined Flag from the most recent run for plotting or reuse.

The pipeline can modify the supplied cubes. Pass copies to run() when their original state must be retained.

add(flagger)[source]#

Append a flagger to the pipeline.

Parameters:: flagger – A BaseFlagger instance.

run(i_cube, v_cube)[source]#

Run the pipeline on two cubes with matching sampling.

Flaggers that do not support the supplied cube type or metadata are skipped. Filtering and zero-weight selections are applied identically to both cubes.

Parameters:

i_cube – Stokes-I data cube.
v_cube – Noise or Stokes-V cube with the same geometry.

Returns:

Tuple containing the flagged Stokes-I and noise cubes.

Note

The input cubes can be modified. Use cube.copy() when their original state is needed later.

apply_last(cube)[source]#

Apply the most recently generated flag to a compatible cube.

Parameters:: cube – Cube with the same original geometry as the last inputs.
Returns:: A flagged copy of cube.

plot(figsize=(10, 12), **fig_kargs)[source]#

Plot diagnostics from the most recent run.

Removed samples are marked with crosses and retained samples whose weights were set to zero with plus signs.

Parameters:

figsize – Matplotlib figure size.
**fig_kargs – Additional arguments passed to matplotlib.pyplot.subplots().

Returns:

The Matplotlib figure containing frequency and spatial diagnostics.

static load(filename)[source]#

Build a flagger pipeline from an INI configuration file.

The [flagger] section defines an ordered, comma-separated pipeline. Each named section must provide a type matching a flagger class from this module. Its remaining values are passed to the class as keyword arguments.

For example:

[flagger]
pipeline = bad_channels, noisy_baselines

[bad_channels]
type = FixedFreqsFlagger
freqs = 115.0-115.2, 121.5
action = filter

[noisy_baselines]
type = UVSigmaClipFlagger
nsigma = 5
action = zero_weight

Parameters:: filename – Path to the configuration file.
Returns:: A FlaggerRunner containing the configured pipeline.

Data flagging (ps_eor.flagger)#

Data flagging (`ps_eor.flagger`)#