Analysis tools

LineStacker.analysisTools tests what a stack shows and prepares cubes or spectra for stacking.

Bootstrapping

Bootstrapping stacks random draws, with replacement, of the stacked sources. The spread of the results shows how much the stack depends on which sources it contains; a few bright sources dominating it give a skewed or multi-peaked distribution.

import LineStacker.analysisTools as A

boot = A.bootstraping_OneD(spectra, nRandom=10000, method='median', save='amp')
boot = A.bootStraping_Cube(coords, fEm=fEm, stampsize=16, N_chans=31,
                           nRandom=1000, save='ampAndWidth')

Both return a list of nRandom + 1 entries, the first one for the stack of all sources. save decides what is kept from each draw: 'all' (the whole stack), 'amp' (its maximum), 'ampAndWidth' (amplitude and width of a Gaussian fit; fits that do not converge are stored as NaN), and for cubes 'outflow' (the six parameters of a two-Gaussian fit). For cubes the line is the spectrum summed over the stamp.

Subsampling

subsample_OneD() stacks random subsets of the spectra and grades every stack; each spectrum collects the grades of the stacks it was part of. Spectra that make stacks better get higher totals.

grades = A.subsample_OneD(spectra, nRandom=10000, maxTest=A.maximizeSNR)
best = sorted(grades, key=grades.get, reverse=True)[:10]

The grade function is maximizeSNR() by default (fitted amplitude over the standard deviation of the valid channels outside the line; 0 when the fit fails or fewer than 2 such channels are left), maximizeAmp() (maximum), maximizeOutflow() (amplitude × width² of the fainter component of a two-Gaussian fit), or any function of a stacked spectrum. minSize and maxSize bound the sizes of the subsets.

Random positions and noise

stack_estimator() repeats a cube stack at random positions, between lowerLimit and upperLimit radians from each source (5 and 10 beams by default), keeping each source’s redshift. The distribution of these stacks is what noise alone gives:

std, mean, stacks = A.stack_estimator(coords, fEm=fEm, nRandom=2000, N_chans=31)

noise_estimator() measures the noise of images or cubes from random pixels, optionally outside a mask, for the whole cube or channel by channel. Blank (NaN) pixels are never drawn; pixels that are exactly 0 are data. LineStacker.randomCoords() draws random pixel positions anywhere in a set of images, and LineStacker.randomizeCoords() moves positions randomly by one to a few beams.

Line widths and redshifts

Lines of the same rest-frame width are narrower at higher redshift, and lines of different widths blur a stack. LineStacker can bring them to a common width before stacking:

  • regridFromZ1D() (spectra) and LineStacker.image.stack(..., regridFromZ=True) (cubes, in memory) resample every source to the same rest-frame channel width; regridFromZ() writes the regridded cubes (stamps around each source) to disk, to stack as returned with spectralMethod='z'.

  • rebin_OneD() and rebin_CubesSpectra() rebin every source so that its line spans as many channels as the narrowest line, measured by Gaussian fits (or given as widths). This needs lines visible source by source.

Rebinned and regridded spectra keep their line centres, spectral axes and per-channel weights in step with the new channels. Blank (NaN) channels are never used to make a value: rebin_OneD and rebin_CubesSpectra interpolate linearly, and a new channel interpolated from a blank channel is blank; regridFromZ1D and regridFromZ interpolate with a cubic spline through each stretch of valid channels separately, so only new channels between a valid and a blank channel are blank. New channels beyond the last channel of a source (when its channels are made narrower) are blank too.

Channels made wider: conserveFlux

These tools make the channels of some sources wider, by a factor s:

  • s = (1 + z_max)/(1 + z) when regridding to the highest redshift (regridMethod='scaleToMax', the default of LineStacker.image.stack);

  • s = line width / narrowest line width in rebin_OneD and rebin_CubesSpectra.

A new channel is then the mean of all the channels it covers, partly covered channels counting by their fraction (conserveFlux=True, the default since 2.0.1). Every channel of the data is used, and the integrated flux of each source is kept.

LineStacker 1.x and 2.0.0 sampled instead: a new channel took the value of the spectrum at its centre, interpolated between the two nearest channels, and the channels in between were not used. conserveFlux=False still does that.

One spectrum made 3 times wider by sampling and with flux conservation, and the noise of both against the widening factor

For a source widened by s, the noise of sampling divided by the noise with flux conservation is:

s

per stack channel (average, worst case)

integrated line flux (average, worst case)

1.25

1.07, 1.10

1.01, 1.01

1.5

1.14, 1.22

1.03, 1.07

2

1.28, 1.63

1.16, 1.41

3

1.49, 1.73

1.42, 1.72

4

1.72, 2.14

1.65, 2.04

The worst case is a line centred exactly on a channel. What differs from sampling:

  • Noise. It is lower, as in the table. Up to s of about 1.5 the line flux gains little, because the interpolation still uses every channel.

  • Line flux. The integrated flux of each source (the sum of its channels times their width) is kept exactly. Sampling keeps it only for lines much wider than a channel: it always samples the line at its centre, so a line narrower than the new channels comes out too bright (47 % for a line of FWHM 1.5 channels at s = 3; 0.2 % at FWHM 4 channels).

  • Line peak. It is that of the wider channels, slightly below the peak of the original channels: 2 %, 4 % and 10 % at s = 1.5, 2 and 3 for a line of FWHM 4 channels.

  • First channel. A rebinned spectrum or cube keeps its channel centres, so half of the first new channel lies outside the data: it is the mean of the half inside.

Channels made narrower, and sources that keep their channels, are interpolated either way. These functions take the option: image.stack, OneD_Stacker.Stack, regridFromZ, regridFromZ1D, rebin_OneD, rebin_CubesSpectra, and the tools that repeat a stack (bootStraping_Cube, stack_estimator, subsample_OneD).

Automatic weights (weighting='sigma2'…) are computed on the resampled stamps, so they follow the lower noise. Weights given with the coordinates or the spectra are not changed.

Speed, parallelism and reproducibility

The Monte-Carlo tools (bootstraping_OneD, bootStraping_Cube, subsample_OneD, stack_estimator) take:

  • n_jobs: the number of CPU cores to use (-1 for all);

  • backend: 'process' or 'thread'. A suitable one is chosen per task: threads for NumPy-bound work (mean and median stacks), processes for Python-bound work (Gaussian fits);

  • seed: a seed for reproducible draws. With seed=None the draws come from NumPy’s global random state, so np.random.seed() also makes runs reproducible.

All random draws are made before the work is split between workers, so the results do not depend on n_jobs or backend. Worker processes are started with spawn; scripts that use them on macOS or Windows need their code under if __name__ == '__main__':. Two process-backed jobs started at the same time from different threads run one after the other (each uses all its workers).

Fitting

LineStacker.tools.fit has the Gaussian fits the tools use: GaussFit(), DoubleGaussFit() and DoublePeakFit(). They ignore NaN channels and raise FitError (a RuntimeError) when a fit cannot be done or does not converge.