PROCESSFOS (Jul00) stpoa.poa_fos PROCESSFOS (Jul00) NAME processfos -- Pre-process and re-calibrate Faint Object Spectrograph (FOS) data USAGE processfos input output DESCRIPTION The 'processfos' task performs the pre-processing of the raw HST data as well as the re-calibration of these data within one task. It is a script which runs the tasks 'poa_preproc_fos' and 'poa_calfos' one after the other. The pre-processor checks to see whether the FOS data fits the POA processing criteria. If the criteria are matched, then the pre-processor updates the input header to prepare it for calibration processing; 'poa_calfos' is executed after the pre-processor is finished. Running 'processfos' on data for which the POA processing criteria are not met, will issue a WARNING to STDOUT, telling the user that there is no POA correction for that particular dataset. Instead of running the POA version of the pipeline, the current version of 'calfos' (from stsdas.hst_calib.fos) will be run on the data instead. The result will be the standard set of 'calfos' output products, with no POA changes in the calibration. Please see the 'poa_preproc_fos' help file for more information on the processing criteria and the pre-processor itself. See the 'poa_calfos' help file for more information on the calibration processing. This task can also be used to run on a list of FOS datafiles; see the 'input' parameter description below for more information. The POA FOS pipeline relies on the STSDAS and TABLES packages; therefore, we advise that one adds the following to their IRAF login.cl file (before the 'keep' statement): tables stsdas hst_calib fos ctools stpoa poa_fos If these packages are not loaded via the login.cl file, then one must open them on the command line prior to running the POA FOS tools. PARAMETERS input [string] The rootname of the input FOS observation data set. One can also run on a list of rootnames, using an ascii file as input, with suffix ".list". There must be only one rootname per line; the input file MUST have a '.list' ending, or else it will not be recognized as an ascii input list file. The last entry in the list file must have a carriage return. The '.list' file itself as well as all the FOS data files associated with this list must be present in the working directory in order to process these datasets properly. When using a list file as input, each 'output' name is the same as the input root name listed. output [string] The rootname of the output FOS data set. If no value is specified for this parameter, 'output' will default to the value passed to 'input'. For multi-processing using a '.list' file, 'output' is automatically named the same as the 'input'; the user is unable to set the 'output' to anything other than ""; if the 'output' parameter is set to anything, the string is nullified during processing. (internal_ref = yes) [boolean] The POA reference files are added to the raw headers using the pre-processor; the STPOA internal reference files are used by default (internal_ref = yes). However, the user also has the option to use external CDBS POA calibration reference files, by setting this parameter to "no". It is up to the user to check that the stpoa reference files are also available in CDBS. If the pre-processor does not find the reference files in CDBS, it will issue a warning and set the reference files to the internal files. Unless stated otherwise in the release notes or on the POA web pages, the FOS POA calibration reference files internal to stpoa should be identical to the external CDBS files. CDBS ingestion of the FOS POA ref files can take some time; therefore, these files are always made available internally to match a given release, while CDBS can only be updated after a given release. (force_poa = no) [boolean] Prior to processing, the data are checked to see if the POA pre-processor has already been run; if not, then this tool will run the pre-processor and pipeline; if yes, then this tool will exit, stating that one can run the pipeline directly or force re-pre-processing. This option is for the unusual case of needing to re-run the POA FOS pre-processor. If the FOS data have already been pre-processed, then it is unlikely that one would want to re-run the pre-processor. Perhaps in the event that the header has been manually changed; the user may want to 'force' the POA pre-processor to be re-run. If so, then set this option to 'yes'. This option has built into the system mostly for the POA project, for testing the POA criteria. If the 'poa_calfos' pipeline is updated with further improvements (say for other FOS modes) one can issue the pre-processor to accept a FOS dataset even if it does not yet fall into the POA processing criteria. Also, if one is used to using the 'all' processor and are running on data that has been pre-processed, it is "harmless" to re-preprocess, unless there has been a software or reference file change since the last time you ran on these data. (rootlist = "") [null string] Dummy parameter needed to run on a list of FOS rootnames. The user should always leave this parameter blank. It is automatically nullified at runtime. Do not put the name of the '.list' file here; it should go in the 'input' parameter field. EXAMPLES 1. Run the pre-processor and perform calibration for observation 'y0k4510dt'. All data files for this observation are in the directory 'poa_spec$' and are copied to the current working directory. Output files are to have the same root name as the input files. st> copy poa_spec$y0k4510dt.* . st> processfos y0k4510dt "" 2. Pre-process and calibrate the observation 'y3ee2804t' and produce output files with a root name of 'test186' in the subdirectory 'temp'. st> copy poa_spec$y3ee2804t.* . st> mkdir temp st> processfos y3ee2804t temp/test186 3. Pre-process and calibrate a list of observations (in one step), using an ascii text file of the root names to be processed; output file rootnames will be the same as the input. The sample data are in the poa_spec$ directory; as is the sample list file, which is called 'y0.list'. st> copy poa_spec$y0cw0109t.* . st> copy poa_spec$y0cw010it.* . st> copy poa_spec$y0cw0110t.* . st> copy poa_spec$y0k4510dt.* . st> copy poa_spec$y0ue0103t.* . st> copy poa_spec$y0.list . st> processfos y0.list "" NOTES Once the POA pre-processor has been run on the FOS data, one should not use 'calfos' on these pre-processed data. The wavelength calibration files and the flat field files have been changed in the header to point to the POA version; calfos should not use these files for processing. Therefore, we highly recommend that one does not use 'calfos' on POA pre-processed data. Keep a copy of the original non-pre-processed data, if you want to run calfos. In the unusual case where one would like to reset the header back to the way it was before the POA pre-processing, one can use hedit to manually change the header values. Use the POA pre-processor STDOUT messages as the source of what the original values were, prior to pre-processing the data files. An example for changing header keyword values after the pre-processor has been run, is as follows. If the following changes took place during the pre-processing: OFF_CORR=OMIT ===> OFF_CORR=PERFORM SCT_CORR=PERFORM ===> SCT_CORR=OMIT FL1HFILE=yref$i1k1220cy.r1h ===> FL1HFILE=pref$l7m1413ly.r1h FL2HFILE=yref$N/A ===> FL2HFILE=yref$N/A CCS6=ytab$e5v11576y.cy6 ===> CCS6=pwav$l611655oy.cy6 The user can change the values back to the original values: st> hedit root.*h OFF_CORR "OMIT" verify- update+ st> hedit root.*h SCT_CORR "PERFORM" verify- update+ st> hedit root.*h FL1HFILE "yref$i1k1220cy.r1h" verify- update+ st> hedit root.*h CCS6 "ytab$e5v11576y.cy6" verify- update+ Once the header edits the done, one should not run either the 'poa_preproc_fos' or the 'processfos' tasks, since they will overwrite the changes which have just been made. The only reason to set the header back to the original values, is to be able to run 'calfos' on these data. The POA pipeline will not run on data which have not been POA pre-processed. BUGS When running on a list of rootnames, if there is an error in processing any of the datasets in the list, the rest of the list will not be processed. This is an IRAF bug, which exits processing if there is an error of any kind while processing any list. REFERENCES References to the Post Operational Archive system for HST data can be found on: "http://www.stecf.org/poa" HELP For assistance using this particular task, please contact ecf-poa@eso.org, or stdesk@eso.org (see http://www.stecf.org/poa). For assistance using this or any other tasks, please contact help@stsci.edu or call the help desk at 410-338-1082. SEE ALSO poa_preproc_fos, poa_calfos, hedit, chcalpar Type "help poa_fos opt=sys" for a higher-level explanation of the 'poa_fos' package and the process of re-calibrating a FOS data set. Type "help poa_preproc_fos" for an explanation of the FOS data pre-processor. Type "help poa_calfos" for an explanation of the 'poa_calfos' task for re-calibrating a FOS data.