Menu for the package stpoa.poa_fos poa_calfos - Post Operational Archive Pipeline calibration for FOS. poa_preproc_fos - Task for preprocessing FOS data, for additional keywords needed by poa_calfos. processfos - Task for running the FOS preprocessing and pipeline back-to-back. pfos_pix2wav - Task to calculate wavelength positions from FOS pixel positions. pfos_dispfit - Obtain dispersion coefficients for FOS ccs6 reference file. POA_FOS (Jul00) stpoa.poa_fos POA_FOS (Jul00) NAME poa_fos -- Post Operation Archive Calibration of Faint Object Spectograph (FOS) Observations. INTRODUCTION The 'poa_fos' package contains routines to re-calibrate FOS data with improved recipes (algorithms, reference data) by the Post Operational Archive Project (STECF/POA). The STSDAS 'calfos' task was updated to 'poa_calfos'; we have also added a few tools to work on POA fos data. All other fos tasks can still be found under the STSDAS hst_calib/fos package tree. Below is the 'fos' helpfile with some POA additions. The STSDAS fos package contains routines to generate the calibration reference files and tables required by 'poa_calfos', routines that were needed to generate the parameters needed for observation planning and instrument configuration, and routines to display and reduce FOS data. The POA FOS pipeline 'poa_calfos', maintained by the STECF/POA project, provides consistent calibration of FOS observations with algorithms and reference data based on a comprehensive review of the entire FOS data archive. CALIBRATION REFERENCE FILES AND TABLES The following table summarizes the FOS calibration parameters needed in tools in the 'poa_fos' package. For each parameter, the table lists the STSDAS tasks that generate the parameter, and the name of the CDBS relation (if any) in which the parameter is stored. Table 1. FOS Calibration Parameters Required by RSDP ------------------------------------------------------------------ | Parameter Description | Primary STSDAS task(s) | CDBS Relation | ------------------------------------------------------------------ |deadtime constants | deadtimey | CYCCG2R | |aperture parameters | aperlocy, apscale | CYCCS0R | |aperture positions | ybase | CYCCS1R | |emission lines | | CYCCS2R | |filter widths | | CYCCS3R | |Wollaston/Waveplate | | CYCCS4R | | parameters | | | |sky shift parameters | | CYCCS5R | |wavelength coefficients| linefind, dispfity, | CYCCS6R | | | waveoffset, fitoffsety | | |GIMP-correction scale | | CYCCS7R | | factors | | | |predicted background | | CYCCS8R | | count rates | | | |scattered light | | CYCCS9R | | parameters | | | |OTA focus history | | CYCCSAR | |Relative aperture | | CYCCSBR | | throughputs | | | |Aperture throughput | | CYCCSCR | | vs. OTA focus | | | |Time change in | | CYCCSDR | | sensitivity | | | |background reference | parthity | CYBACR | | files | | | |flat field reference | flatfield | CYFLTR | | files | | | |inverse sensitivity | absseny, absfity | CYINVR | | reference file | | | |average inverse | absseny, absfity | CYAISR | | sensitivity file | | | |retardation reference | | CYRETR | | files | | | |disable diode files ! | CYDDTR | |quality initialization | parthity | CYQINR | | reference files | | | |post-COSTAR polarimetry| | CYPCPR | | correction files | | | ------------------------------------------------------------------ POA CALIBRATION CODE -- POA_CALFOS The calibration process consists of the following steps: 1) Convert to count rates. 2) Apply dispersion direction offset correction. (POA_CALFOS v1.0+) 3) Correct for diode non-linearities. 4) Subtract background. 5) Subtract scattered light. 6) Apply flat fielding. 7) Subtract sky. 8) Assign wavelengths. 9) Correct for changes in OTA focus. 10) Normalize to reference aperture. 11) Convert to absolute flux units. 12) Correct for time change in sensitivity. 13) Produce the output statistical error array. 14) Special mode processing. Raw counts are converted to count rates by dividing each data point by the exposure time and correcting for disabled diodes. If the science data header keyword 'DEFDDTBL' is true, then the disabled diodes are taken from the Unique Data Log (UDL), otherwise, disabled diodes are found in the file given by the keyword 'DDTHFILE'. This file is selected from the reference relation 'CYDDTR'. Raw count rates are corrected for saturation in the detector electronics. Deadtime constants (table 'CCG2') are used to correct for the non-linear channel response of each diode. These constants are not expected to change through the lifetime of the FOS instrument. Analysis of prelaunch data indicate that the same constants can be used for both FOS detectors. After conversion from counts to count rates and the correction for paired-pulse loss, the raw data is shifted in memory location according to the sum of offsets determined from all effects listed above. The correction is applied to the raw data, errors and data quality values. For FOS header keyword GRNDMODE = SPECTROSCOPY, IMAGE and RAPID-READOUT, a correction is determined for each data group (or ystep) based on the orbital position of the spacecraft at the mid-point of the observation for each subintegration. While the correction is calculated to sub-pixel accuracy, it is applied as an integer value and is therefore accurate only to the nearest integral pixel. This is done to avoid resampling the data in the calibration process. Special handling is required for data obtained in ACCUM mode (GRNDMODE=SPECTROSCOPY, OPMODE=ACCUM) since each data frame contains the sum of all frames up to that point. In order to apply a unique correction to each frame, data taken in this mode are first unraveled (deaccumed) into separate frames. Each frame is then corrected individually, and the corrected frames are then recombined to match the input data format. The correction is applied by simply shifting pixel values from one array location to another. For example, if a correction for a particular data group is calculated to be +2.38 pixels (STDOUT message when running 'poa_calfos'), the data point originally at pixel location 1 is shifted to pixel 3, pixel 2 is shifted to pixel 4, pixel 3 to pixel 5 and so on. Pixel locations at the ends of the array that are left vacant by this process (e.g., pixels 1 and 2 in the example above) retain their unshifted values and are assigned a data quality value of 700. The particle-induced background can be subtracted from the sky and object spectra. A default reference background is indicated by the keyword 'BACHFILE'. Note that this version of poa_calfos includes a refined method for scaling the 'BACHFILE'. The L-shell geomagnetic parameter which describes the expected particle flux for a given geomagnetic longitude and latitude, is used in place of simply the geomagnetic longitude and latitude alone. The expected count rate versus L-shell relation was calibrated with data covering the whole of the FOS lifetime. In addition a bug affecting the way that accumulated datasets had the background subtracted has also been fixed. Contributions from (non-dispersed) scattered light can be measured and removed from object and sky spectra obtained using gratings that have a region of no sensitivity to dispersed light within the recorded spectrum. The mean light level is computed within this no sensitivity region of the spectra and is then subtracted (as a constant) from all regions of the object and sky spectra. The diode ranges of the no sensitivity region for each grating are stored in the reference table 'CCS9'. Diode-to-diode sensitivity variations and fine structure are removed by multiplying by the flat field response. Requires the flat field response file 'FL1HFILE'. A second flat field file, 'FL2HFILE', is required for paired-aperture or spectropolarimetry observations. These files are selected from the reference relation 'CYFLTR'. Please note that the flat fields have been reprocessed using the POA correction. The current version of the POA FOS software (v1.2 Aug 2001), has the POA reference files internal to the STPOA package. Therefore, the flat field reference files location will be set to "pref$". However, these same POA ref files are likely to be ingested into the STScI CDBS archive, later to be available via the standard "yref$" location. The sky can be subtracted from the object spectra. The emission line relation ('CCS2') gives regions where a sky spectrum should not be smoothed before subtraction from the object spectrum; the detector parameters ('CCS3') contain the filter widths (mean and median) for smoothing the sky spectra. The sky shift parameter ('CCS5') gives an integer shift in pixels that is to be applied to the sky spectrum before subtraction. The aperture parameters ('CCS0') contain aperture areas required to scale a sky spectrum taken in one aperture of a paired aperture before subtraction from the object spectrum taken in the other aperture. Values for this relation are required only for the paired apertures. The aperture position parameters (CCS1) are used to determine which aperture (UPPER or LOWER) of an aperture pair was used for observing an object or sky spectrum. The wavelength parameters ('CCS6') give the dispersion coefficients used to generate wavelength scales. Separate coefficients are required for each detector, disperser, aperture and polarizer mode. The 'ccs6' wavelength calibration file has been completely changed from it's original 'calfos' format. This file is an internal reference files to the STPOA package; it should also be visible in the standard CDBS reference file area once STScI has integrated it into it's archival system. The POA pre-processor 'poa_preproc_fos' updates the CCS6 header keyword with the POA file name and location; a message is printed to STDOUT, telling the user that the wavelength calibration file has been changed in the header. The new 'ccs6' file contains 10 coefficients to calculate the wavelength dispersion. For all POA criteria data, all 10 coefficients are being used for the solution. These coefficients used for processing are printed to STDOUT. Object spectra are converted to absolute flux units by multiplying by the inverse sensitivity vector. This step requires the inverse sensitivity file found in header keyword 'IV1HFILE'. A second inverse sensitivity file, 'IV2HFILE', is required for paired aperture or spectropolarimetry observations. These files are selected from the reference relation 'CYIVSR'. Pre-COSTAR object spectra can utilize the following four enhanced flux calibration techniques (available only in CALFOS versions 2.0 and higher): Object spectra are corrected for changes in aperture throughput due to changes in OTA focus. This step requires the OTA focus history table found in header keyword 'CCSA' and the focus change in aperture throughput table found in header keyword 'CCSC'. Object spectra are normalized to the throughput of the reference aperture for which the average inverse sensitivity (AIS) curve was derived. This step requires the relative aperture throughput table found in header keyword 'CCSB'. Object spectra are converted to absolute flux units by multiplying by the average inverse sensitivity vector for the reference aperture. This step requires the average inverse sensitivity file found in header keyword 'AISHFILE'. This file is selected from the reference relation 'CYAISR'. Object spectra are corrected for changes in instrumental sensitivity over time. This step requires the time change in sensitivity table found in the header keyword 'CCSD'. The propagated error can be calculated at each point in the spectrum. The final step performs ground software mode dependent reductions for time-resolved, spectropolarimetry, and rapid-readout observations. The spectropolarimetry reductions require the the Wollaston and Waveplate parameter table, 'CCS4', and the retardation reference file, 'RETHFILE'. The retardation file is selected from the reference relation 'CYRETR'. If the spectropolarimetry data were acquired during the post-COSTAR epoch as indicated by the Science header keyword/value pair, KYDEPLOY=T, and a post-COSTAR calibration is available, the post-COSTAR polarimetry correction reference file, 'PCPHFILE', selected from the relation 'CYPCPR' is also required for proper calibration. However, it should be noted that if no post-COSTAR polarimetry calibration file for the 'PCPHFILE' calibration file keyword is provided, 'calfos' will not abort. Instead, warning messages will be issued and the data in the output 'c3h' file will still be calibrated, but the additional post-COSTAR correction will not have been applied. Only versions 3.0 and higher of 'calfos' include the post-COSTAR polarimetry correction capability. The calibration process is controlled by the values of certain header keywords. These keywords may be examined and modified using the task 'chcalpar'. All FOS data from the archive require pre-processing using the task 'poa_preproc_fos' in order to make their keywords and new reference file(s) and locations compatible with 'poa_calfos'. OBSERVATION PLANNING AND INSTRUMENT CONFIGURATION In addition to the calibration parameters needed originally by by the Routine Science Data Processing software (RSDP), there are other FOS calibration parameters that were needed to configure the instruments, and compute target acquisition parameters. The relevant STSDAS tools are references here for completeness. Table 2. Non-RSDP FOS Calibration Parameters ------------------------------------------------ | Parameter Description | Primary STSDAS tasks | ------------------------------------------------ | ybases | ybase | | discrimator values | phay | | target acquisition | aperlocy, apscale, | ------------------------------------------------ Ybases are the Y positions required to center a spectrum on the diode array for any detector, disperser, aperture, and polarizer mode. These must be closely monitored as they affect the quality of the scientific data. Improper Ybases will decrease the amount of flux recorded by the diode array. The detector descriminator values are computed by the 'phay' task, which fits a Gaussian function with a polynomial baseline to the differential pulse height distribution. The 'aperlocy' and 'apscale' tasks are used to analyze FOS target acquisition images. DATA REDUCTION AND DISPLAY The tasks listed in Table 3 are used to display or prepare the data for further analysis. Table 3. Display and Reduction Tasks ---------------------------------------------------------------- | STSDAS tasks | Task Description | ---------------------------------------------------------------- | bspec | compute scattered light spectrum using FFTs | | countspec | calculate countrate spectrum from flux | | deaccum | unstack ACCUM mode data into separate reads | | fwplot | plot flux vs. wavelength | | foswcorr | correct blue wavelengths for zero-point shift | | *gimpcor | show the magnitude of the calfos gimp | | mkmultispec | Create a Multispec MWCS for the spectra | | rapidlook | display multigroup image as 2-d image | | rcombine | average or sum rapid-readout data with | | | propagation of errors and data quality | | pcombine | average multiple data sets with propagation | | | of errors and data quality | | resample | resample data to a linear wavelength scale | | vac2air | convert vacuum wavelengths to air | | yd2p, yp2d | pixel <--> diode conversions | | ymkmu | Place wavelength information in headers | | yddintplot | analyze dead diode locations | | yfluxcal | reset calibration switches for flux cal. | | ypeakup | Analyse ACQ/PEAK observations | | yratio | Take ratios of each group of ACCUM obs. | ---------------------------------------------------------------- *gimpcor - Should only be used on data for which 'poa_calfos' is currently unable to make a GIMP correction (please see the 'poa_calfos' help file for a list of the current functionality). HELP For assistance using this POA related tasks, please contact ecf-poa@eso.org, or stdesk@eso.org (see http://www.stecf.org/poa). For assistance using fos or any other tasks, please contact help@stsci.edu or call the help desk at 410-338-1082.