SWAS Data Reduction and Data Products

Data are downlinked from the spacecraft twice daily.  NASA is responsible for concatenating successive 12 h datasets, performing some quality control and formatting chores, and sending full 24 h datasets to the SAOSOC daily via the internet.  Datasets are sent from NASA daily at about 3 am.  Transmissions  include 24 h of Level 0 and spacecraft telemetry data.  Level 0 data are processed automatically within fifteen minutes of their arrival at SAOSOC producing the Level 0.5 data product. At this point, the data are reduced from Level 0.5 to Level 1 interactively via the IRAF-based SWAS pipeline.

---

1.0 SWAS Pipeline Processing

The output of the SWAS spectrometer, containing all of the simultaneously observed spectral lines, is read out every 10  milliseconds, co-averaged for 2 seconds (i.e. 200 spectra), and each 2-second co-average is stored on-board.  These data, along with engineering and housekeeping data, are downlinked to a NASA controlled ground station twice per day.  The two downlink data sets are checked for transmission errors, time ordered, and merged at GSFC, then transferred to SAO once each day.  Upon receipt of the data at SAO, the data are automatically reformatted into separate FITS files for each observation and into a separate enigneering database.  The IRAF-based SWAS pipeline data data reduction package then processes the data through several steps by: aligning the spectra in velocity space to correct for Doppler shifts induced by spacecraft motion around the Earth; separately co-averaging segments of 30 seconds ON-source and  OFF-source data for an object; performing the ON minus OFF subtraction for adjacent ON/OFF pairs; folding in the appropriate calibration load, zero signal, and blank sky measurements; checking the frequency calibration using data from the on-board comb-line generator; and creating flux and frequency calibrated spectra.

A number of quality control steps are performed along the way:

1) A check for mode-hops in the laser diode of the AOS that can result from temperature changes and will affect the frequency calibration derived from the comb spectra.  So far, we have never encountered a mode hop.

2) A check for time gaps in the data.  Time gaps can occur due to ground supports that occur completely within a segment, or due to bad data product from DPS.  In the latter case, SAO requests a new product.

3) If the last segment from the previous day was broken by the day boundary, we put it back together and run it through the pipeline.

4) The AOS has been found to have a problem with hysteresis after the blanking switch has been disengaged.  The blanking switch is used during ground supports AND during ZEROs.  All waypoints used to have ZEROs, but after Misson Day 284 only special waypoints (called "zwaypoint" on the timeline summary sheet) have ZEROs.  In order to give the AOS time to settle down after the blanking switch has been disengaged, we have decided to drop the 50 scans that immediately follow (about 2 minutes worth of data).

- For segments where a ground support occurs entirely within a science segment, the pipeline will *automatically* drop 50 scans following the use of the blanking switch.

- For segments which have already started while the blanking switch is still engaged, we correct for the hysteresis by hand by dropping  the following 50 scans.

5) A visual inspection of each spectrum to ensure its validity.

Once we are confident that the data are all of the highest quality the IRAF-FITS data are converted to CLASS format.

For more details on the SWAS pipeline processing, consult our SWAS Data Pipeline page.
 

2.0 Post-Pipeline Processing

SAO creates two final data products for release to the community.  One is what we refer to as the pointing-based archive for IRAF-FITS formatted Level 1.5 data.  The other archive is in CLASS format.  CLASS is the data reduction package for single-dish radio telescopes from IRAM.  The CLASS executable binaries can be downloaded from http://iram.fr.  Details on both data products are given in Sections 2.1 and 2.2 below.
 

2.1 The IRAF-FITS Pointing-Based Archive

A pointing is defined as an attitude at which the spacecraft dwells long enough to acquire spectra of a target.  A pointing can be towards a target's coordinates, or offset from a target, as will be the case for mapping observations. By definition, maps contain many pointings.  The pointing-based archive is therefore the set of all science observations, coadded over the lifetime of the SWAS mission, for each pointing.

The IRAF data for a particular source are all contained within a single directory whose name is that of the source. Each directory contains a number of IRAF-FITS files with each file corresponding to one pointing of the SWAS telescope (i.e. if SWAS has made a 9 point map of this source, there will be 9 individual files). Each of these files is an IRAF-FITS file containing 6 separate IRAF extensions.  These extensions correspond to the 6 SWAS sidebands (i.e. CI - O2, H2O - 13CO, and H218O - other).  Each extension contains its own secondary header (with information about the spectra within the extension) followed by a series of spectra which comprise the "image".  Each spectrum (i.e. each row of the image) contains the co-added spectrum for an individual orbit (usually about 20 to 40 minutes of integration time).  The FIRST row of each extension, however, contains the TOTAL co-added spectrum of all the orbits.  An index table on our Download Data page lists information on available data.
 

2.2 The CLASS Archive

All CLASS data for a particular source are stored in a single data file named all.dat within a directory which is given the same name as the source (e.g. the data for S140 are contained in the datafile all.dat in the directory s140).  The data can be accessed from our Download Data page.

It should be noted that some of the sourcenames have a number appended to the end of the root sourcename (e.g. omc-1+15).  This describes data taken with an alternate LO (local oscillator) setting.  In the example of OMC-1+15, we adjusted the VLSR of the source by +15 km/s.  Data taken with multiple LO settings can be co-added manually by the user.

Solar system objects (e.g. Comet Lee or c_lee) are also handled somewhat awkwardly.  Since these are moving targets, they have no fixed RA or DEC values.  For these targets, the offset fileds in the CLASS headers are meaningless.

Description of the Contents of the Data File :

Performing a LIST operation on the data file (within CLASS) will provide a list of the file's contents.  The resulting list has 10 columns but really only 7 search fields.

For Example :

  12032; 14    S140   H2(18)O   s457074 NOD      0.0         0.0  Eq.    13032
|--------------|--------|------------|--------------------|----------|-------------|----------|
  Field 1        Field 2   Field 3            Field 4         Field 5      Field 6     Field 7
 

Field 1 - The first number is a sequential scan number.  The second number (after the semicolon) is a version number which can be ignored. One can select a particular scan or a range of scans with the SET NUMBER command.  Note - each scan contains approximately 5 minutes of ON-source integration time.

Field 2 - The source name.

Field 3 - The line name.  There are 6 possible lines to choose from.  CI, O2, H20, 13CO, H2(18)O, and OTHER (the H2(18)O image sideband).  One may select any of the lines using the SET LINE command which does accept wildcard characters.  NOTE - YOU MUST PLACE AN * AFTER LINE NAME!!!  E.g. to select the CI line type SET LINE C* or SET LINE CI* but NOT SET LINE CI.

Field 4 - A description of the observation.  For example, s457074 NOD means that the observation occured in orbit segment 074 (i.e. was the 74th source observed) on day 457 of the mission, and that the observation was done in nodding mode.  The other possible modes are MAP (mapping mode), and CHP (chopping secondary engaged).

Field 5 - The mapping offset (in RA) in whatever units (degrees, arcminutes, or arcseconds) you are displaying the data.

Field 6 - The mapping offset (in DEC) in whatever units (degrees, arcminutes, or arcseconds) you are displaying the data.  The Eq. means the coordinates are equatorial.

NOTE - One may select any position using the SET OFFSET command.

Field 7 - Another sequential scan number.  One can select a particular scan or a range of scans with the SET SCAN command.

Either plotting a spectrum or using the HEADER command will provide information about the observation.  Refer to the "CLASS Reference Manual" for details on the CLASS headerparameters.

       - General information line:
         Scan number and version
         Source name
         Line name
         Day of Mission, segment number, and observing mode
         Date of observation
         Date of last reduction
       - Position information line:
         Right ascension
         Declination
         Epoch of RA and DEC
         Offsets in current units
         Type of coordinate (Eq, Ga, Az)
       - Quality information line:
         Quality of calibration (as defined by a SET QUALITY)
         Tau: opacity at zenith (set to 0 for SWAS)
         Tsys: system temperature (outside atmosphere).
         Time: total integration time on source (minutes)
         El: elevation of source (set to 0 for SWAS)
       - Spectral information lines (2):
         N: number of channel
         I0: reference channel (real)
         V0: velocity at reference channel
         Dv: velocity resolution (signed)
         F0: rest frequency at reference channel
         Df: frequency resolution (signed)
         Fi: image frequency at reference channel
       - Calibration information line:
         B_ef : Beam efficiency of SWAS
         F_ef : Forward efficiency of SWAS
         G_im : Gain in image band
       - Atmospheric information lines (2):  SET TO 0 FOR SWAS
         H2O : millimeter of precipitable water vapor
         Pamb: ambient pressure
         Tamb: ambient (receiver cabin) temperature
         Tchop: Chopper temperature
         Tcold: Cold load temperature
         Tatm: Atmospheric temperature in the signal band
         Tau: Zenith opacity in the signal band
         Tatm_i: Atmospheric temperature in the image band
         Tau_i: Zenith opacity in the signal band

     followed by the list of Scan numbers added in the observation.
 

2.3 Notes on Continuum Levels

In some sources, the spectral baselines are not at zero but show an offset due to the background continuum of the source.  It must be noted that these continuum offsets do not necessarily reflect the true level of the continuum flux due to instrumental instabilities and the fact that the spectra are calibrated to a single sideband.
 

3.0 Glossary of Terms

AOS -- Acoustico-Optical Spectrometer.  The detector used by SWAS.

APID -- Application ID.  A grouping of mnemonics in the SWAS telemetry.  For example, APID 20 contains the continuum detector data, APID 15 contains AOS spectra, and APIDs 16-19 contain housekeeping data.

CLASS-FITS -- A SWAS data product optimized for use by CLASS.

IRAF-FITS -- A SWAS data product optimized for use by IRAF.

Level 0 Data -- Science data sent to SAO by GSFC.  These data are the "raw" data provided from the spacecraft, reorganized into separate files according to "ApID" ("Application ID" - GSFC's arrangement of data according to content) and encapsulated according to a NASA standard called CCSDS packetization.

Level 0.5 Data -- This is our term, not a standard NASA term.  By it, we mean that we have not actually changed any of the content of the Level 0 data, but we have reformatted it considerably, uncompressed the spectra, and reorganized them into a number of separate files, each of which corresponds to one roughly 30-40 minute observation of a target, i.e., one segment.  Level 0.5  files are in IRAF-FITS format, suitable for input to our IRAF-based pipeline processing package. The planetary chop data are also put into special IRAF-FITS files.

Level 1.0 Data --  In standard NASA parlance, these are data from which the instrumental signature has been removed.  For example, instead of counts per pixel, the Level 1.0 data are expressed in terms of flux per frequency unit.  According to SWAS usage, the Level 1.0 Data are provided as 5 minute averages of the reduced spectra.

Level 1.5 Data -- This is our term, not a standard NASA term. By it, we mean that we have now coadded spectra to at least include the final spectra for each 30-40 minute observation of a target.   The data are arranged so that it is easy to derive coadds of multiple pointings to a given target.

Level 2 Data -- In NASA parlance, this is meant to indicate targets' physical properties, inferred from the Level 1.0 data: line strengths, velocities, velocity widths, chemical abundances, etc.

Lines -- Emission from one of the species to which SWAS is sensitive, i.e, carbon ([CI]), oxygen (O2), water (H216O), 13CO (J = 5-4), or isotopic water (H218O).

Mission Day -- Elapsed days from the day SWAS is launched.  Our convention is to define midnight of 1998 November 1 (the expected SWAS shipping date) as the beginning of mission day 0001.  This is equivalent to setting UNIX time 909,878,400 seconds equal to zero hours and zero seconds on mission day 0001.  Note that this is a revision (effective 1998 October 1) of the definition used during instrument testing.

Pointing -- An attitude at which the spacecraft dwells long enough to acquire spectra of a target. A pointing can be towards a target's coordinates, or offset from a target, as will be the case for mapping observations.

SAOSOC -- The Smithsonian Astrophysical Observatory Science Operations Center.  The operations center for SWAS.

Segment -- The data obtained while the spacecraft is observing one target during one orbit.  Segments are typically 30-40 minutes long.  They may include multiple pointings, as when mapping, but only one target.  Each segment has a unique associated number, the segment index.

Target -- An object of interest.  W33, the Orion Nebula, and S140 are examples of scientific targets. Jupiter is a planetary target.  Each target has a well-defined name and position in the file.