QUaD May 2008 Notes

Subject: Re: .newdat format
From: Carlo Contaldi 
Date: Tue, 20 May 2008 17:04:55 +0100
To: Clem Pryke 

hi Clem

glad to hear the eagerly awaited data files are going to come out soon! 
The format for the newdats looks complicated but it contains a lot of 
useful information for doing params with small sky band power 
experiments like ours. As you pointed out it's not written up anywhere 
so here goes...

1) first line is the root filename for the window functions it will look 
for in data/windows

2) then a line of 6 integers stating how many bins in each pol type in
the order TT EE BB EB TE TB (note this is different from cosmomc's order 
for historical reasons - cosmomc didn't expect to introduce other xcorrs)

3) Optional band selection set of 7 lines e.g.

BAND_SELECTION
  1 24
  1 7
  1 7
  0 0
  1 9
  0 0

this would use TT 1-24, EE 1-7, BB 1-7, and TE 1-9 etc. If These 7 lines 
are not present it will use all available bands but this feature is nice 
to test sensitivity to high or low-l bands

4) below that (or 4th line if BAND_SELECTION is not present) there is a 
calibration error option line with
  calon/off(1/0), recal value (in temp), cal error (in power)

note that cal error in power and recal value are in different units for 
historical (stupid) reasons! usually recal is 1 unless you want to test 
against recalibrations or simply update the calibration without changing 
anything else

5) beam error option line with
  beamerr on/off(1/0),  beam FWHM (arcmin), beam err (arcmin)
This assumes a gaussian beam with errors evaluated at weighted average 
of l for each bin.

6) likelihood option line
  liketype
  0 == treat all bands gaussian
  1 == treat all bands offset lognormals (using offsets provided)
  2 == treat according to flag in last column of bandpower (0== 
gaussian, 1==offset lognormal)

option 2 is useful if you are including TE as those might be better 
approximated by a gaussian

7) 'TT', 'EE' etc. flag telling the code which pol type is listed

8) list of band power info:
band num per type, C_b, dC_b+,dC_b-, noise offsets, lmin, lmax, likeflag

lmin, lmax are not used and are included just for plotting purposes, 
noise offsets should be provided if liketype > 0. If liketype < 2 then 
last column may be omitted

9) dump of block diagonal normalized covariance matrix for that 
particular pol type, dimension must be the same as list immediately 
above. This is included purely for the practical purpose of reading off 
bin-to-bin correlations without having to dig them out of the full 
covariance matrix. It is not used by cosmomc and I guess I should make 
this optional at some point but this information is always very handy 
for people to see.

10) repeat 7)-9) for each pol_type included, bin numbers must agree with 
values given in 2)

11) dump of the whole all-with-all covariance matrix (e.g. inverse 
Fisher) across all pol types included and in same order as 2). The 
covariance matrix is always that of the C_Bs (not of the offset 
lognormal vars even if liketype > 0). The transformation to offset 
lognormals or mixed gaussian/offset lognormal is done internally in 
Readdataset_bcp.

12) optional SZ freq line in GHz e.g.
#SZ 30
This was for our internal (not included in public version) way of 
including a freq dependent SZ template for the various experiments with 
automatic scaling e.g. acbar + cbi. Antony has now included his own 
method to deal with SZ templates in the latest cosmomc version but with 
his you have to give a separate template for each experiment.

finally - windows, these follow the cosmomc format with each bin in a 
separate file. Each file will have l, TT, TE, EE, BB contribution to 
that particular bin - note this is the cosmomc ordering! you have to 
list each l between an lmin and lmax of your choosing.

The windows must be normalized and the number actually listed is 
W_l^B/l. The theoretical band power is

C_B = \sum_l (l+1/2)/l/(l+1) W_l^B C_l

let me know if you need anything else

Carlo

Back To QUaD May 2008 Product Info