----------------------------------------------------- sensor_3.9 out of range 28 Oct 2003 ----------------------------------------------------- "Sensor" refers to temperature sensor in the dewer. All temperatures in the dewer were climbing; sensor 3, having the most limited operation range, triggered the alarm first. As usual, stop the noise with "alarm," check the alarm log with "galarm," then plot the sensor values for antenna 9 with the xplotwatch. Sho' enough, the temperatures were climbing at all four sensors. To watch this in real time, hit "9r" in the xstatus window. Ultimately, we also disabled the alarm for all sensors on antenna 9 (with xfixwatch). First action was to cycle the power on the compressor. (Climb the telescope; open the big silver box to your immediate right; cycle either the white or black switch on the cold head.) You could hear the compressor turn off and turn on. This did not solve the problem. Rick hypothesized that the problem was with the lens inverter (down and to the left of the cold head). The lens inverter ordinarily sits in slow mode; we tried fast mode to see if it would bring the temperatures down. It did, the friction generated in fast mode limits how low you can go, so this was a temporary solution. Ultimate solution was to swap the lens inverter. The new inverter needed to be dialed down to 26 RPMs in its slow mode. That did the trick! With the temperatures normal, we re-enabled the sensor alarms on antenna 9. Final action is to let the script owner know that the dewer warmed up around 10:30 PM when making the quality assessment. ----------------------------------------------------- antenna will not lock up 29 Oct 2003 ----------------------------------------------------- A new script started, and a bunch of crazy shit happened, including antennas pointing all around and turning off, and some bullshit about phases not locking, etc. The first thing I tried was to get all the telescopes on and pointing to reasonable positions. This did not fix the situation, so I turned my attention to the phase lock problem (after, frankly, trying a LOT of other things). Antennas 1 and 9 were affected; in the xstatus "d" window, the evidence was "-" instead of "L" under the LOCK column. The notes in the "harmonic" file honed in on the problem, so I followed them dutifully. After trying all possible harmonic numbers, the antennas would still not lock up, so -- per instructions -- I took them offline and started the script. The script proceeded smoothly. But now here's some more info: 1. The script called for FREQ=72.4, which is very low. This is probably why the receivers would not lock -- though that's not entirely clear. 2. The correct thing to do IS in fact to proceed with these antennas turned off. 3. Unfortunately, ant 9 is a collision antenna in C-array. This means it must be stowed in its safe position in order for the script to proceed without incurring a collision: > csafe 9 This command drags all collision antennas (4, 6, 9) along with 9 to the safe position, and then bring 4 and 6 back, ready for action. 4. So now the script is terminated; ant 1 is off (and it's parked at az=0 el=7); ant 9 is off and in its safe position (az=175 el=8). The thing to do is to restart the master script: > c03.csh | & tee -ia c03.log 5. But now here's another problem. This script is okay with ant 1 and 9 off, but we'd like them on again when the next script starts. This can be achieved by editing the master script by hand. I used xpeek to find the current script's name. It's the line in front of the next script that needs editing. Here's what I did: > cd /obs/obs/oper > vi c03.csh 36.3750: #-------------------------------------------------------------- project name=t860c072.n7538 start=2242 stop=0900 date=30oct03 if ($status != 0) goto 999.9999 36.6250: #-------------------------------------------------------------- cunsafe 9 ; aon 1 ; ants ants=1 az=0 el=45 options=wait project name=t853c115.n3862 start=0900 stop=1500 date=30oct03 if ($status != 0) goto 999.9999 36.3750 refers to the current script -- the one that must be run with ant 1 and 9 turned off. 36.6250 is the script BEFORE which the antennas must be turned back on. That line beginning with "cunsafe" does the job. First, cunsafe turns 9 back on, then drags 4 and 6 around with it to prevent collisions. Presumably, it returns them to a reasonable pointing. Next, we turn on ant 1, but that's pointed at the ground, so we move him by hand to a reasonable position. The "wait" tells "ants" to get the move finished and settled before the next project begins. ----------------------------------------------------- Bringing a telescope back into the fray 30 Oct 2003 ----------------------------------------------------- Let's say you had to turn off an antenna. If it was a collision antenna, you'd need to follow the csafe / cunsafe instructions described above. Cunsafe is not very efficient though, so you could try doing this by hand. Note that all of the following can/should be performed with the master script cranking away in the red xterm. Commands to be issued to the shell can be entered in some other xterm. Again, let's pretend it's ant 9 that you're bringing back in. 1. Ant 9 is off and in its safe position. First go out to the antenna, flip the drive control to manual, and drive it (carefully) so that it's pointing in approximately the same place as everybody else. Flip the control back to computer. 2. Ant 9 is off as far as the software is concerned. In some window: > aon 9 > halt There are some important details here. "aon 9" -- duh -- just turns the antenna on from the point of view of the software. However, the antenna won't be pointed in *exactly* the right place. The "halt" tells the software to begin a new scan. That new scan will see that ant 9 is now back in the game, and it'll point it (along with everybody else) at the same location in the sky. 3. The last thing to do is to let the control software know that ant 9 is back, and is pointing in the correct place. (How is this different from what we did in step 2? I don't totally know.) Anyway, now that all telescopes are in the game and are pointing in the right place: > fixall This kills and restarts the antenna driving software. If you had done this before the "halt" above, an alarm would have gone off, because the antenna driving software would have noticed collision antennas which were out of alignment. CAVEAT: I have not actually tried this; it was merely described to me. I used cunsafe, which did the job. ----------------------------------------------------- SETFREQ: SETFOCUS finds stalled motor on ant 4 30 Oct 2003 ----------------------------------------------------- You can ignore this. The motor which tunes ant 4 is shot. Fortunately, you never need to change the frequency. This may or may not get fixed, someday. ----------------------------------------------------- Ant 3 azim has stalled 30 Oct 2003 ----------------------------------------------------- If the azim stalls, the alarm will trip. Here's what I got: 31Oct 02:40|Ant 3 azim has stalled at 53.981214, try 1 31Oct 02:40|Ant 3 azim has stalled at 53.980998, try 2 31Oct 02:40|Ant 3 azim has stalled at 53.980964, try 3 31Oct 02:40|Ant 3 azim has stalled at 53.980916, try 4 31Oct 02:40|Ant 3 azim has stalled at 53.980884, try 5 31Oct 02:40|ALARM: FATAL:An active antenna has been turned OFF All I did was head out to ant 3, check that no LEDS were red at the control panel, then cycle the power (hit the big red button, wait, hit the big black button). That seemed to do it, as the following sequenced aimed at bringing it back online did fine: > aon 3 # turn the damn thing on > ants ants=3 az=147.9 el=64.8 # point it where everybody else is pointed > aoff 3 # turn it back off, for tuning > tuneant 3 # do a minimum tuning > aon 3 # turn it on, to join the array > halt # force a cal observation, so that ant is picked up All this was done with the masterscript toiling away with ant 9, in a different shell window.