Click on these topics to find out what to do when ...
The TOI reports "Position Rejected - Nothing Done"
The TOI reports that one or more of the Axes have halted
The TOI reports a "Could not read channel n" Error
The TOI reports "Axis not parked" error, or one or more axis are not parked at startup
There is a power outage during observing
The TOI reports a Limit Switch error
The Dewar LN2 hold time seems very short (< 18 hours)
The TOI reports a Rotator Servo error
You point to an object and it is not there!
The TOI crashes during operation
PSF or Flats appear to have Contours or Intensity Histogram has "spikes"
You need to manually park the telescope
The TCC reports that it "Failed to set the Earth block"
This error is supposed to be generated when you request that the telescope slew to a position which beyond the range of travel of the elevation axis, ie.too near the zenith or below the horizon limit. Unfortunately, it also tends to occur sometimes when a valid star coordinate is entered. In the case when invalid star coordinate is entered, the message truly means what it says, the position is rejected and the telescope halts and remains in its current position. If the star position is actually valid the TCC will accept and process the coordinates, in spite of what the error message says. When you see this error message you should clear the error and then look at the axes status boxes on the main TOI window. If they say "Halted" then you have entered invalid coordinates and they were rejected. If they say "Slewing" then the position was actually accepted and the telescope is slewing to the requested coordinates.
This typically happens because one of the axes has tracked into a forbidden zone. The three different cases and their remedies are listed below.
The elevation axis tracks into the exclusion zone near the zenith - You can either wait until the object leaves the zone above 85 degrees elevation and then press OK to resume tracking (note that you will probably need different offsets) or you can switch to another object.
The elevation axis tracks to the horizon limit - If you are paying attention, you should stop observing an object long before this happens, but if it does, it is time to switch to another object.
The telescope tracks into the rotator or azimuth wrap positions at +/- 270 degrees - This can happen if you track an bject for a long time. If you want to continue to observe this object, simple press the OK button on the main TOI window and the TCC will unwrap the appropriate axis and resume tracking. This could take 6 minutes because of the long distance the axis must travel.
Sometimes this is a problem and sometimes it is not. The first thing that you should do is to clear the error(s). If there are a small number of these errors reported < 10 and then they stop, you should look at the scrolling status window and watch for the TCC /Telescope coordinates. If they are scrolling by and they still appear synchronized then everything is OK. If they are not synchronized then the axes have probably halted and this should be reported on the main screen. If this is the case, record your offsets (because they will be reset to zero) and press OK. Once the telescope axes are tracking, re-enter your offsets and continue observing.
If you try to clear the errors, but they keep reappearing, then there is very likely a serious communications error between the TCC and the Ziatech servo controller. Press the STOP button on the main TOI window and exit the TOI. Under these conditions, you will not be able to control the telescope from the control room and you will have to check the status from the control room at the base of the pier. The first thing to do when you enter the control room is to check the bottom of the status screen on the Ziatech monitor. Record any error which is reported and enter it into the observing log. Next, try to switch axes using the F7 key. If you are lucky enough to get a response, you should press F4 to reset all axes. After you hear the relays latch you should press F2 on each axis screen to restart the axes and then press F6 to manually park the axes. Unfortunately you will probably not be this lucky. You will likely find that the processor does not respond to any of the keys indicating that it has locked up. At this point you should press the emergency stop key and then go up to the observing level and park the telescope manually.
Once the telescope has been parked, either manually or under servo control, you need to hard reset the servo processor by cycling the Servo Control power on the control panel. Once the Servo Controller has booted you can return to the control room and restart the TOI
There are four common causes of this error, none of which are serious:
1. Someone was working on the telescope and did not park the axis properly when they were done.
2. One of the rotators jumped out of the park position at system shutdown due to the vagaries of the power shutdown sequence.
3. An axis did not park properly the last time the telescope was parked, usually due to an encoder error.
4. The telescope was parked manually and it was not positioned properly.
The first thing you should do is open the limit switches window under the Engineering menu. Examine the limit switch states and determine which axis is not parked. Once you have identified the axis see the section on manually parking the telescope to park the appropriate axis. Once the offending axis has been parked, you will probably need to reset the servos because you invariably will move the axis too fast and trigger an overspeed. To reset the axis, simply press the Reset All (F4) key on the Ziatech console. You should then press F8 and review the limit switch status on the console to make sure that the axes are all parked and than any overspeeds or ESD errors are cleared.
A limit switch error can occur for a number of reasons. When you first see the error, press the STOP button in the main window, clear the error message and open up the limit switches window under the Engineering menu. Check for errors and the use the following list to correct the problem.
ESD or No Air Pressure - This means that someone hit one of the emergency shut down switches, the brake control pendant was left in an improper state or the air compressor has failed. First, check all of the ESD switches (one on the control panel in the electronics room and 4 on the telescope skirt at the observing level. If you find one of the switches depressed, you can release it by giving it about 1/8 turn counterclockwise. Next, check the Brake Control Pendant, if the LED is illuminated, then one of the brakes has been released, identify the offending switch(s) and change the state to turn off the LED. If an ESD button was depressed, make sure that you determine the reason and resolve any problems before restarting the telescope. If you determine that it is safe to restart the telescope, you can reset the axes using the Reset All (F4) button on the Ziatech console and then start each axis individually using F2. Once all of the axes have been restarted, you can return to the control room and enter a set of object coordinates.
If none of the ESD buttons is depressed then the compressor has probably failed. You can check this by pressing one of the main axis brake release buttons and listening for the hiss of escaping air. If there is no hiss, or it is very quite compared to normal, then the compressor is not working. You will need to park the telescope by hand and determine the reason for the failure.
<axis name> Overspeed - For some reason one of the axes moved too fast resulting in an over speed condition. This can happed to a rotator if a cable gets caught on the cable wrap. It can also occur on the main axes in very high wind or for an unknown reason. In either case, you should go to the observing level and check visually inspect the entire telescope paying particular attention to the cable wraps and anything which may have interfered with the motion of the telescope. If the wind is very high you should park the telescope manually with at least two people. Otherwise, once you have eliminated the problem which caused the over speed or your examination has not found any problems, you can proceed to restart the axes. Someone should remain on the observing level clear of the telescope to watch the telescope reaction during the axes restart. Go to the electronics room at the base of the pier and press the Reset All (F4) key on the Ziatech Console. Next, start each axis individually and have the person on the observing levels look for sudden axis jumps. If an axis jumps and generates another over speed then you should park the telescope by hand, otherwise once all axes have been restarted you can return to the TOI and enter new object coordinates.
Axis in Limit - This should not occur since the TCC should know better than to drive the axis into a limit, but it can happen. Once an axis is has triggered a limit switch it is necessary to park the telescope by hand and then reset all of the axes using the Reset All (F4) command on the Ziatech console. It is then necessary to restart all of the axes before resuming observing.
This is usually caused because the dewar does not have an adequate vacuum. The typical reason is that it was not pumped long enough before filling with LN2. The higher internal dewar pressure will increase conductive heat flow from the outside of the dewar to the inside causing the LN2 to boil off more rapidly. It will also cause the outside of the dewar to be colder than normal. If this happens to happen on a humid night, you may also observer condensation on the dewar window.
To solve this problem, you need first let the LN2 evaporate and then re-pump and re-cool the dewar per the instructions in the operators manual.
NOTE: you MUST let the LN2 dissipate and the interior of the dewar come to room temperature before pumping, otherwise
The first thing you should do is to press the stop button on the main screen to halt the axes.
The most common is that the monofilament which is used to drive the cable wrap gets caught on the wrong side of the posts. This will be immediately obvious if you go up to the observing level and look at the instrument. The monofilament will be caught on the wrong side of one of the posts and it will be pulled very tight. If this is the problem, carefully rotate the instrument and the cable wrap pulley to relieve some of the tension on the line. Next, carefully pull the monofilament around to the correct side of the post(s).
The second most common cause of this error is again related to the cable wrap. On rare occasions, one of the cables will become caught or wedge someplace on or near the instrument. If this occurs, it will also be obvious. As with the previous problem, the first thing to do is to rotate the instrument and the cable wrap to relieve the tension on the cable. Next, reposition the cable and, if necessary tie it into the proper position.
If neither of these problems appears to be the cause of the failure, try rotating the instrument by hand slowly and make sure that it rotates freely. If it does not, look carefully around the instrument to find the cause. If it does rotate freely, then something more serious may be wrong. In either case, if you cannot identify the problem you should park the telescope manually and restart the axes with someone observing the rotator during startup and the initial slew.
If you successfully identify and remedy the problem without parking the telescope, there is a good chance you can recover servo operation without parking and reinitializing everything. Proceed to the electronics room at the base of the pier. On the Ziatech main axes console, press the Reset All function key (F4). You should hear the relays reset, if not, press it once again. After the relays have reset, use the Start (F2) function key to restart each axis individually (you will have to toggle between screens using F7). Eac time you restart an axis you should notice that the servo error is near zero and there is some "life" in the encoder readings. If you are successful, you should be able to return to the TOI and press the OK button on the main screen and the telescope should return to your previous object. Note that you probably will have lost your offset values, so you might want to write them down before starting this procedure.
Normally this error will occur immediately after you start the telescope although it can also occur after a limit switch or servo error. The cause is improper zeroing of one or more of the encoders, or a time base error between the axes. When this happens, you should park all of the axes and then restart them so their encoders and clocks are reinitialized.
Although the frequency of this problem has been reduced significantly, it does still happen. The symptom is that the TOI will simply disappear from the screen. This usually happens when you are editing a text field, but it can happen at other times as well. When the TOI crashes, the TCC will continue to run and send commands to the telescope, therefore if the telescope is tracking when the TOI crashes, it will continue to track. This is fortunate, because it means that the telescope will remain under control while you get the TOI restarted. Also, if you are in the middle of an exposure, you can wait until the exposure is complete before you do anything. One caveat is that if you were using the adaptive tertiary servo, it will no longer feed back offsets to the main axes servo system so you may loose lock.
1. Restart the TOI.
2. Once the TOI is running and has retrieved status information from the TCC and the servo controllers you will notice that the servos are in the same state as when the TOI crashed. If they were tracking before, they will still be tracking, if they were halted they will still be halted. If they happened to have been slewing, the telescope has probably reached the object and they will report that they are tracking. At this point you must stop the axes by pressing the STOP button on the main screen. It may seem that the recovery process is complete, but unfortunately the state of certain parts of the TOI are incorrect. For example, even though the rotator servo may be running, the TOI thinks that it is off. The TOI also does not know the status of the filter changer or the tertiary servo system and it may even rotate the tertiary to the wrong instrument, therefore it is necessary to park and restart the axes.
3. If the tertiary servo was on, you must turn it off before parking. Open the HRCam window and toggle the Servo On switch to the ON and then back to the OFF state.
4. Park all of the axes.
5. Once the axes report that they are parked, you can restart the telescope as documented in the startup procedure.
First check that the control power switch at the top of the stairs is in the on position (the indicator light will be on). If the control power is on and the building still will not open it is possible that high winds have moved the building slightly and caused the park pin to jam in it's locking hole. To free the pin, press the building close button for about 30 seconds. If the pin is jammed this should release the pressure on the pin and the building should open. If neither of these solutions solves the problem, you will need to contact the company who installed the controller for service.
This problem indicates that the 5V supply for the CCD controller needs to be adjusted. See Adjusting HRCam Controller Power Supply Voltage in the Maintenance manual section.
The TCC requires a current Earth Orientation Block data file to operate. This file contains the minor pertubations to the earth's rotation rate for fine tuning the telescope pointing. The file is located on the Alpha system in the TDAT directory. The name of the file is EARTHPRED.DAT and contains 120 days of data. This file is updated weekly by the command script IERS_SUBMIT.COM on the Alpha. This script is submitted as a batch job and it will continue to execute as long as the Alpha keeps running. If there is a power failure the job will not auto-execute at startup. Four months later when the predictions expire you will see the following error when you start up the TOI (and therefore the TCC):
TCC>0 0 F Modu="tut_CompEarth"; Text="data file is too old"
TCC>
TCC>0 0 W Modu="exe_Slow"; Text="Failed to set the Earth block. The telescope
!!CANNOT BE USED!! until you update the earth orientation predictions and
issue the SET TIME command."
When you get this error you must quit the TOI, log onto the Alpha and resubmit the batch job. The new earth orientation block will be loaded immediately and you can then restart the TOI and resume observing.
The instructions for resubmitting the Earth Orientation Update script are as follows:
Telnet to charon and log in as TCC
Type the following commands:
$ SET DEF TDAT
$ SUBMIT IERS_SUBMIT.COM
You can now check the status of the job by listing the IERS_SUBMIT.LOG file by issuing the command:
$ TYPE IERS_SUBMIT.LOG
The result should look like:
$ Set NoOn
$ VERIFY = F$VERIFY(F$TRNLNM("SYLOGIN_VERIFY"))
Your process name is BATCH_4
220 maia FTP server (Version 1.7.212.5 Wed May 30 12:19:42 GMT 2001) ready.
Connected to MAIA.USNO.NAVY.MIL.
331 Guest login ok, send ident as password.
230 Guest login ok, access restrictions apply.
200 PORT command successful.
150 Opening ASCII mode data connection for /ser7/ser7.dat (33997 bytes).
226 Transfer complete.
local: DKA0:[TCC.LOCAL.DAT]IERSBULLA.DAT;63 remote: /ser7/ser7.dat
33997 bytes received in 00:00:00.62 seconds (52.86 Kbytes/s)
221 Goodbye.
120 days of predictions were written
Normal Processing Completion.
16-SEP-2002 17:36:47
www@calypso.org
Last updated: 17-SEP-02 - BET