Analysis tools¶
LineStacker.analysisTools tests what a stack shows and prepares
cubes or spectra for stacking.
Bootstrapping¶
Bootstrapping stacks random draws, with replacement, of the stacked sources. The spread of the results shows how much the stack depends on which sources it contains; a few bright sources dominating it give a skewed or multi-peaked distribution.
import LineStacker.analysisTools as A
boot = A.bootstraping_OneD(spectra, nRandom=10000, method='median', save='amp')
boot = A.bootStraping_Cube(coords, fEm=fEm, stampsize=16, N_chans=31,
nRandom=1000, save='ampAndWidth')
Both return a list of nRandom + 1 entries, the first one for the stack of
all sources. save decides what is kept from each draw: 'all' (the whole
stack), 'amp' (its maximum), 'ampAndWidth' (amplitude and width of a
Gaussian fit; fits that do not converge are stored as NaN), and for cubes
'outflow' (the six parameters of a two-Gaussian fit). For cubes the line
is the spectrum summed over the stamp.
Subsampling¶
subsample_OneD() stacks random subsets of
the spectra and grades every stack; each spectrum collects the grades of the
stacks it was part of. Spectra that make stacks better get higher totals.
grades = A.subsample_OneD(spectra, nRandom=10000, maxTest=A.maximizeSNR)
best = sorted(grades, key=grades.get, reverse=True)[:10]
The grade function is maximizeSNR() by
default (fitted amplitude over the standard deviation of the valid channels
outside the line; 0 when the fit fails or fewer than 2 such channels are
left),
maximizeAmp() (maximum),
maximizeOutflow() (amplitude × width² of the
fainter component of a two-Gaussian fit), or any function of a stacked
spectrum.
minSize and maxSize bound the sizes of the subsets.
Random positions and noise¶
stack_estimator() repeats a cube stack at
random positions, between lowerLimit and upperLimit radians from each
source (5 and 10 beams by default), keeping each source’s redshift. The
distribution of these stacks is what noise alone gives:
std, mean, stacks = A.stack_estimator(coords, fEm=fEm, nRandom=2000, N_chans=31)
noise_estimator() measures the noise of
images or cubes from random pixels, optionally outside a mask, for the
whole cube or channel by channel. Blank (NaN) pixels are never drawn; pixels
that are exactly 0 are data. LineStacker.randomCoords() draws
random pixel positions anywhere in a set of images, and
LineStacker.randomizeCoords() moves positions randomly by one to a few
beams.
Line widths and redshifts¶
Lines of the same rest-frame width are narrower at higher redshift, and lines of different widths blur a stack. LineStacker can bring them to a common width before stacking:
regridFromZ1D()(spectra) andLineStacker.image.stack(..., regridFromZ=True)(cubes, in memory) resample every source to the same rest-frame channel width;regridFromZ()writes the regridded cubes (stamps around each source) to disk, to stack as returned withspectralMethod='z'.rebin_OneD()andrebin_CubesSpectra()rebin every source so that its line spans as many channels as the narrowest line, measured by Gaussian fits (or given aswidths). This needs lines visible source by source.
Rebinned and regridded spectra keep their line centres, spectral axes and
per-channel weights in step with the new channels. Blank (NaN) channels are
never used to make a value: rebin_OneD and rebin_CubesSpectra
interpolate linearly, and a new channel interpolated from a blank channel is
blank; regridFromZ1D and regridFromZ interpolate with a cubic spline through each
stretch of valid channels separately, so only new channels between a valid
and a blank channel are blank. New channels beyond the last channel of a
source (when its channels are made narrower) are blank too.
Channels made wider: conserveFlux¶
These tools make the channels of some sources wider, by a factor s:
s = (1 + z_max)/(1 + z) when regridding to the highest redshift (
regridMethod='scaleToMax', the default ofLineStacker.image.stack);s = line width / narrowest line width in
rebin_OneDandrebin_CubesSpectra.
A new channel is then the mean of all the channels it covers, partly covered
channels counting by their fraction (conserveFlux=True, the default since
2.0.1). Every channel of the data is used, and the integrated flux of each
source is kept.
LineStacker 1.x and 2.0.0 sampled instead: a new channel took the value of
the spectrum at its centre, interpolated between the two nearest channels,
and the channels in between were not used. conserveFlux=False still does
that.
For a source widened by s, the noise of sampling divided by the noise with flux conservation is:
s |
per stack channel (average, worst case) |
integrated line flux (average, worst case) |
|---|---|---|
1.25 |
1.07, 1.10 |
1.01, 1.01 |
1.5 |
1.14, 1.22 |
1.03, 1.07 |
2 |
1.28, 1.63 |
1.16, 1.41 |
3 |
1.49, 1.73 |
1.42, 1.72 |
4 |
1.72, 2.14 |
1.65, 2.04 |
The worst case is a line centred exactly on a channel. What differs from sampling:
Noise. It is lower, as in the table. Up to s of about 1.5 the line flux gains little, because the interpolation still uses every channel.
Line flux. The integrated flux of each source (the sum of its channels times their width) is kept exactly. Sampling keeps it only for lines much wider than a channel: it always samples the line at its centre, so a line narrower than the new channels comes out too bright (47 % for a line of FWHM 1.5 channels at s = 3; 0.2 % at FWHM 4 channels).
Line peak. It is that of the wider channels, slightly below the peak of the original channels: 2 %, 4 % and 10 % at s = 1.5, 2 and 3 for a line of FWHM 4 channels.
First channel. A rebinned spectrum or cube keeps its channel centres, so half of the first new channel lies outside the data: it is the mean of the half inside.
Channels made narrower, and sources that keep their channels, are
interpolated either way. These functions take the option: image.stack,
OneD_Stacker.Stack, regridFromZ, regridFromZ1D, rebin_OneD,
rebin_CubesSpectra, and the tools that repeat a stack
(bootStraping_Cube, stack_estimator, subsample_OneD).
Automatic weights (weighting='sigma2'…) are computed on the resampled
stamps, so they follow the lower noise. Weights given with the coordinates
or the spectra are not changed.
Speed, parallelism and reproducibility¶
The Monte-Carlo tools (bootstraping_OneD, bootStraping_Cube,
subsample_OneD, stack_estimator) take:
n_jobs: the number of CPU cores to use (-1for all);backend:'process'or'thread'. A suitable one is chosen per task: threads for NumPy-bound work (mean and median stacks), processes for Python-bound work (Gaussian fits);seed: a seed for reproducible draws. Withseed=Nonethe draws come from NumPy’s global random state, sonp.random.seed()also makes runs reproducible.
All random draws are made before the work is split between workers, so the
results do not depend on n_jobs or backend. Worker processes are
started with spawn; scripts that use them on macOS or Windows need their
code under if __name__ == '__main__':. Two process-backed jobs started at
the same time from different threads run one after the other (each uses
all its workers).
Fitting¶
LineStacker.tools.fit has the Gaussian fits the tools use:
GaussFit(),
DoubleGaussFit() and
DoublePeakFit(). They ignore NaN channels and
raise FitError (a RuntimeError) when a fit
cannot be done or does not converge.