****************** * Kast Reduction * (Or: My Shitty Notes, a Mystere!) ****************** 23 May 2002 THIS IS FAR FROM THE KAST FAINT-OBJECT REDUCTION GOSPEL. I'M REALLY JUST MESSING AROUND HERE. 1. Trim 'em all. Keep [1:1190,10:169]. ls *fits > list.all :.,$s/.fits//g @names.idl Check a few: cd .. display raw/b178.fits 1 display redux/b178.fits 2 2. Make n1/n2 bias, compare and wave hands. imhead r*fits | grep bias --> list.bias_rn1 and list.bias_rn2 imhead b*fits | grep bias --> list.bias_bn1 and list.bias_bn2 First experiment with the statistics of one of the frames: iterstat b207 b207: mean=0.06858139 rms=1.885055 npix=189235 median=0.006478287 mode=-0.004605293 Try imcombine with the following parameters (suggests by the "Reducing your CCD data" handbook): imcombine ("@list.bias_bn1", "bias_bn1", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="minmax", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) display bias_bn1 4 iterstat bias_bn1 bias_bn1: mean=-0.4108614 rms=0.5028479 npix=188259 median=-0.3998171 mode=-0.3745111 Woah! The mean/median in in the combined frame is slightly negative! This would result in *adding* some small number, which doesn't feel right. And that small number add is less than the RMS, which also doesn't feel right. I'll check the other bias frames, but I get the feeling I'm going to skip this step. (Except that the bias frame shows some funny columns. I suppose these will flatten out, or could be masked out? Hmmmm.) (Of course, you could also argue that evidently KAST slightly oversubracts when it writes out the blue side, which is why these bias frames are running under, and which should be corrected by putting that small number back in. Small beans, probably.) imcombine ("@list.bias_bn2", "bias_bn2", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="minmax", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) imcombine ("@list.bias_rn1", "bias_rn1", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="minmax", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) imcombine ("@list.bias_rn2", "bias_rn2", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="minmax", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) Now check 'em: display bias_bn2 1 display bias_rn1 2 display bias_rn2 3 iterstat bias* bias_bn1.fits: mean=-0.4108614 rms=0.5028479 npix=188259 median=-0.3998171 mode=-0.3745111 bias_bn2.fits: mean=-0.5625234 rms=0.613769 npix=188877 median=-0.5650827 mode=-0.5593288 bias_rn1.fits: mean=2.934422 rms=0.7281235 npix=189238 median=2.911753 mode=2.953817 bias_rn2.fits: mean=1.875294 rms=0.7189943 npix=188945 median=1.876704 mode=1.755012 Huh. Forget the blue side bias. The red side bias change by a couple sigma from night 1 to night 2. I think I'll subtract the red side bias, each night separately, and scrap the blue side. 3. So bias subtract, already. ls r*fits > list.obj_rn1 ls r*fits > list.obj_rn2 vi list.obj_rn1 :.,$s/.fits//g vi list.obj_rn2 :.,$s/.fits//g imhead @list.obj_rn1 imhead @list.obj_rn2 imarith @list.obj_rn1 - bias_rn1 @list.obj_rn1//b imarith @list.obj_rn2 - bias_rn2 @list.obj_rn2//b iterstat r467.fits r467.fits: mean=8464.403 rms=4372.092 npix=190400 median=9261.8 mode=551.6046 iterstat r467b.fits r467b.fits: mean=8462.512 rms=4372.086 npix=190400 median=9258.35 mode=549.8265 Good -- nothing crazy happened. 4. Think about flattening. a) Start with red side. As a first pass, just combine all the flats across the night and make one big flat. If the data ends up looking shitty, you could use just the flats taken right after the science frame and see if that helps things. imhead r*b.fits | grep flat --> list.flat_rn1 Some of these frames are saturated and such. Better check 'em individually. iterstat @list.flat_rn1 Yup. Ditch the first two frames... r105b: mean=-1.308212 rms=2.35533 npix=189661 median=-1.316603 mode=0.0235796 r106b: mean=30904.48 rms=15.48011 npix=146803 median=30901.29 mode=30892.63 r113b: mean=13847.91 rms=7053.31 npix=190400 median=15470.21 mode=947.9731 r114b: mean=13551.98 rms=6918.821 npix=190400 median=15162.96 mode=908.3257 r115b: mean=13451.42 rms=6882.425 npix=190400 median=15033.37 mode=910.1982 ...etc... You know what? The RMS looked so high that I displayed and blinked a bunch of these. I think the fringes moved around enough that you really ought to just flatten with the local flats (i.e. I mean, those dome flat taken right after the science frame should get applied to that science frame, and none other.) I'm gonna go for it. (This might not apply on the blue side.) Sort 'em: ls r113b.fits >> list.flat_rn1_J1429 ls r114b.fits >> list.flat_rn1_J1429 ls r115b.fits >> list.flat_rn1_J1429 ls r116b.fits >> list.flat_rn1_J1429 ls r117b.fits >> list.flat_rn1_J1429 ls r118b.fits >> list.flat_rn1_J1429 ls r134b.fits >> list.flat_rn1_J1432 ls r135b.fits >> list.flat_rn1_J1432 ls r136b.fits >> list.flat_rn1_J1432 ls r137b.fits >> list.flat_rn1_J1432 ls r138b.fits >> list.flat_rn1_J1432 ls r152b.fits >> list.flat_rn1_J1430 ls r153b.fits >> list.flat_rn1_J1430 ls r154b.fits >> list.flat_rn1_J1430 ls r155b.fits >> list.flat_rn1_J1430 ls r156b.fits >> list.flat_rn1_J1430 ls r171b.fits >> list.flat_rn1_J1426_27 ls r172b.fits >> list.flat_rn1_J1426_27 ls r173b.fits >> list.flat_rn1_J1426_27 ls r174b.fits >> list.flat_rn1_J1426_27 ls r175b.fits >> list.flat_rn1_J1426_27 Combine 'em. Use average with pclip, and scale by the median: cl < flatcombine.cl Watch the scaling! Some look really bad, so ax those frames and run again. Check 'em: flat_rn1_J1426_27.fits: mean=8647.064 rms=4460.806 npix=190400 median=9670.357 mode=541.7512 flat_rn1_J1429.fits: mean=13291.59 rms=6799.197 npix=190400 median=14895.04 mode=905.5116 flat_rn1_J1430.fits: mean=10569.88 rms=5414.396 npix=190400 median=11716.28 mode=705.8002 flat_rn1_J1432.fits: mean=9883.612 rms=5087.125 npix=190400 median=10927.95 mode=672.7629 flat_rn2_J1426.fits: mean=13404.85 rms=6790.309 npix=190400 median=14926.91 mode=915.7684 flat_rn2_J1428.fits: mean=13119.14 rms=6673.138 npix=190400 median=14462.27 mode=871.2921 flat_rn2_J1431.fits: mean=12084.97 rms=6164.697 npix=190400 median=13287.37 mode=804.6075 flat_rn2_J1434.fits: mean=9720.686 rms=5005.554 npix=190400 median=10912.64 mode=631.9708 flat_rn2_J1435.fits: mean=13009.74 rms=6597.681 npix=190400 median=14448.39 mode=853.408 Run response. I think those wiggles at the left side are in the CCD and not in the lamp, so don't fit them out! I liked a spline, order 6. response flat_rn1_J1426_27 flat_rn1_J1426_27 flat_rn1_J1426_27nrm interac=no response flat_rn1_J1429 flat_rn1_J1429 flat_rn1_J1429nrm interac=no response flat_rn1_J1430 flat_rn1_J1430 flat_rn1_J1430nrm interac=no response flat_rn1_J1432 flat_rn1_J1432 flat_rn1_J1432nrm interac=no response flat_rn2_J1426 flat_rn2_J1426 flat_rn2_J1426nrm interac=no response flat_rn2_J1428 flat_rn2_J1428 flat_rn2_J1428nrm interac=no response flat_rn2_J1431 flat_rn2_J1431 flat_rn2_J1431nrm interac=no response flat_rn2_J1434 flat_rn2_J1434 flat_rn2_J1434nrm interac=no response flat_rn2_J1435 flat_rn2_J1435 flat_rn2_J1435nrm interac=no Now flatten. Notice I got both nights goin' here: imarith r100b / flat_rn1_J1429nrm r100bf imarith r102b / flat_rn1_J1429nrm r102bf imarith r121b / flat_rn1_J1432nrm r121bf imarith r123b / flat_rn1_J1432nrm r123bf imarith r139b / flat_rn1_J1430nrm r139bf imarith r141b / flat_rn1_J1430nrm r141bf imarith r157b / flat_rn1_J1426_27nrm r157bf imarith r160b / flat_rn1_J1426_27nrm r160bf imarith r161b / flat_rn1_J1426_27nrm r161bf imarith r163b / flat_rn1_J1426_27nrm r163bf imarith r361b / flat_rn2_J1434nrm r361bf imarith r363b / flat_rn2_J1434nrm r363bf imarith r379b / flat_rn2_J1435nrm r379bf imarith r381b / flat_rn2_J1435nrm r381bf imarith r397b / flat_rn2_J1426nrm r397bf imarith r399b / flat_rn2_J1426nrm r399bf imarith r414b / flat_rn2_J1428nrm r414bf imarith r416b / flat_rn2_J1428nrm r416bf imarith r434b / flat_rn2_J1431nrm r434bf imarith r436b / flat_rn2_J1431nrm r436bf imarith r438b / flat_rn2_J1431nrm r438bf imarith r440b / flat_rn2_J1431nrm r440bf b) Now look at blue side. First figure out what's going on in some flats. Just look at n1 for now: lo> imstat @list.flat_bn1 # IMAGE NPIX MEAN MAX b104 190400 0.1973 3924. b107 190400 14246. 30986. b108 190400 8600. 26215. b109 190400 8631. 26275. b110 190400 8690. 26461. b111 190400 8749. 26760. b112 190400 8906. 27116. b130 190400 8020. 24521. b131 190400 7936. 24139. b132 190400 7932. 24216. b133 190400 7774. 23771. The first two are problems; the rest look okay. I'm told that the UV counts are low, so let's splot some. Sho' nuff, the counts are way down on the blue side. Not much I can do about that except combine all the flats I have. No fringes -> not as big a deal to use flats from across the night. imcombine ("@list.flat_bn1", "flat_bn1", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="pclip", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="median", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) imcombine ("@list.flat_bn2", "flat_bn2", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="pclip", project=no, outtype="real", offsets="none", masktype="none", maskvalue=0., blank=0., scale="median", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) Run response - spline3 order 20 imarith @list.obj_bn1 / flat_bn1nrm @list.obj_bn1//f imarith @list.obj_bn2 / flat_bn2nrm @list.obj_bn2//f 5. Cosmic ray zap. szap ("@list.zap_r", "@list.zap_r//z", inter=no, restore=yes, verbose=yes, inimglist="", outimglist="") szap ("@list.zap_b", "@list.zap_b//z", inter=no, restore=yes, verbose=yes, inimglist="", outimglist="") 6. Background subtract. Test out one red side to set the parameters, then run non-interactively. I like a chebyshev order 9. background ("@list.back_b", "@list.back_b//s", axis=2, interactive=no, sample="*", naverage=1, function="chebyshev", order=9, low_reject=3., high_reject=2., niterate=3, grow=1., graphics="stdgraph", cursor="") Same with blue. background ("@list.back_r", "@list.back_r//s", axis=2, interactive=no, sample="*", naverage=1, function="chebyshev", order=9, low_reject=3., high_reject=2., niterate=3, grow=1., graphics="stdgraph", cursor="") 7. Check 'em: display r100bfzs.fits 1 display r102bfzs.fits 2 display r121bfzs.fits 3 display r123bfzs.fits 4 display r139bfzs.fits display r141bfzs.fits 2 display r157bfzs.fits 3 display r160bfzs.fits 4 display r161bfzs.fits display r163bfzs.fits 2 display r361bfzs.fits 3 display r363bfzs.fits 4 display r379bfzs.fits display r381bfzs.fits 2 display r397bfzs.fits 3 display r399bfzs.fits 4 display r414bfzs.fits display r416bfzs.fits 2 display r434bfzs.fits 3 display r436bfzs.fits 4 display r438bfzs.fits display r440bfzs.fits 2 display b101fzs.fits 1 display b103fzs.fits 2 display b122fzs.fits 3 display b124fzs.fits 4 display b140fzs.fits display b142fzs.fits 2 display b158fzs.fits 3 display b159fzs.fits 4 display b162fzs.fits display b164fzs.fits 2 display b360fzs.fits 3 display b362fzs.fits 4 display b378fzs.fits display b380fzs.fits 2 display b396fzs.fits 3 display b398fzs.fits 4 display b413fzs.fits display b415fzs.fits 2 display b433fzs.fits 3 display b435fzs.fits 4 display b437fzs.fits display b439fzs.fits 2 Blue side looks like shit! Oops -- didn't normalize blue flatfield.... That's better. 8. Think about combining... display r438bfzs 1 display r440bfzs 2 Shift is 189.7 --> 210.0 in y. Shift is 142.5 --> 142.7 in x. So shift file is: 0 0 0 20 9. Make the various frames. Cut and paste in "combs.cl" to hit the right object files / target names. Here are some helpful VI cut 'n' pasters. Combs.cl is reproduced below. Watch out -- I bet the shift direction changes... :.,$s/r438/r434/g H :.,$s/r440/r436/g H :.,$s/J1431+3346+0.04/J1429+3359-0.41/g H :.,$s/b437/b433/g H :.,$s/b439/b435/g H :x cp combs_old.cl combs.cl vi combs.cl ### Fringe-subtracted frame - red ### imarith r438bfzs - r440bfzs junk1 imarith r440bfzs - r438bfzs junk2 imcombine ("junk1,junk2", "J1431+3346+0.04r", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="none", project=no, outtype="real", offsets="list.shift", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) imdel junk1 yes ver- imdel junk2 yes ver- ### Fringe frame - red ### imcombine ("r438bfzs,r440bfzs", "J1431+3346+0.04r.fringe", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="none", project=no, outtype="real", offsets="list.shift", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) ### Sky frame - red ### imcombine ("r438bfz,r440bfz", "J1431+3346+0.04r.sky", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="none", project=no, outtype="real", offsets="list.shift", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) ### Fringe frame - blue ### imcombine ("b437fzs,b439fzs", "J1431+3346+0.04b.fringe", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="none", project=no, outtype="real", offsets="list.shift", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) ### Sky frame - blue ### imcombine ("b437fz,b439fz", "J1431+3346+0.04b.sky", rejmask=" ", plfile=" ", sigma="", logfile="STDOUT", combine="average", reject="none", project=no, outtype="real", offsets="list.shift", masktype="none", maskvalue=0., blank=0., scale="none", zero="none", weight="none", statsec="", expname="", lthreshold=INDEF, hthreshold=INDEF, nlow=0, nhigh=2, nkeep=1, mclip=yes, lsigma=3., hsigma=3., rdnoise="0.", gain="1.", snoise="0.", sigscale=0.1, pclip=-0.682, grow=0.) ### Display ### display J1431+3346+0.04r 1 display J1431+3346+0.04r.fringe 2 display J1431+3346+0.04b.fringe 3 display J1431+3346+0.04b.sky 4