XRT HOWTO --- Debug HK at SAO

K. Reeves; 2008-Nov-12

If the XRT HouseKeeping (HK) appears to have stopped working, or is giving strange values, this process will help you evaluate if the problem is with the housekeeping software at SAO.

Overview

There are cronjobs at SAO that parse ascii files created by xskim and plot the hk values on the SAO HK website. These cronjobs run about once per hour. Some of the files are only accessible to those with access to SAO computers. A description of the HK pipeline process is given below.

Housekeeping pipeline

On the machine "norway" at ISAS, the housekeeping data stream is processed via the script /home/sbusxrt/install/hk_auto.sh. This script creates ascii files that contain the housekeeping data.
The ascii files are kept at ISAS on /soda/solarb/xrt/status/yyyy/mm/dd and are named things like "xrt_sts1_yyyymmddt000000.txt.merge." You can get to these files from the gateway machines, or the xrtco machine.
The ascii files are rsynced over to SAO, and live in /archive/hinode/xrt/hk/yyyy/mm/dd.
The files are then processed by another cron job on machida (/data/solarb/XRT/hk_pro/make_hk_plots_merge.cron) that parses them out into two-column text files including the time and the HK value. These files can be found in /data/solarb/XRT/hk_cron/yyyy/mm/dd/. They have names like TMP04C_yyyymmdd_hh.txt. These are the files that are being plotted on the SAO website.
The plots are made (via the same cron job on machida that makes the two column text files) using a batch idl process. The main plotting routine is plot_all_hk_vs2.pro, and it can be found in /data/solarb/XRT/hk_pro/programs/.

Procedure

1. If you have access to the SAO machines, you can identify problems by looking first at the two column text files on the SAO machines at /data/solarb/XRT/hk_cron/yyyy/mm/dd/. Looking at the ones from 11/11/2008, for example, P_P28IV__20081111_00.txt , you see:

UTC_Time P_P28IV_
[...]
2008/11/11_00:57:22.2 29.1797
2008/11/11_00:57:24.2 29.1797
2008/11/11_00:57:26.2 29.1797
2008/11/11_00:57:30.2 29.1797
2008/11/11_03:43:43.5 :03:21:30:07.747
2008/11/11_03:43:45.5 :03:21:30:09.748
2008/11/11_03:43:47.5 :03:21:30:11.748
2008/11/11_03:43:49.5 :03:21:30:13.749
2008/11/11_03:43:51.5 :03:21:30:15.75
2008/11/11_03:43:53.5 :03:21:30:17.75
2008/11/11_03:43:55.5 :03:21:30:19.751
2008/11/11_03:43:57.5 :03:21:30:21.751
2008/11/11_03:43:59.5 :03:21:30:23.752
2008/11/11_03:44:01.5 :03:21:30:25.753
[...]

2. Even if you don't have access to SAO machines, you can check the master ascii file (in /archive/hinode/xrt/hk/yyyy/mm/dd/ at SAO, /soda/solarb/xrt/status/yyyy/mm/dd in Japan) for that day to determine if the strange values are contained in the ascii file.

Creative use of the grep command is your friend when checking the ascii file - these files are huge. However, say you were worried about TMP11. You can do a command that looks something like this:

prompt: grep "Temp 11" xrt_sts1_20081111t000000.txt.merge

And you will get a list of all the TMP 11 values :

Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
Temp 11 (1-5): 46.571442 C (Mirror Support -X)
et cetera.....

3. Email xrt_manager@head.cfa.harvard.edu.