This is an old revision of the document!

PyWiFeS Documentation

Running the Data Reduction

Data reduction is generally run in two steps:

  1. Gather Metadata - in the 'scripts/' directory of your PyWiFeS installation directory you will find the script 'generate_metadata_script.py'. Use this to organize your metadata in the following steps:
    1. Run './generate_metadata_script.py /home/mjc/wifes/20120101/raw_data/' from the command line, with the argument being the directory where your raw data is located. This will automatically figure out what type of data each file is and arrange it in the preferred metadata format.
    2. Check the metadata scripts 'save_red_metadata.py' and 'save_blue_metadata.py' which were automatically generated by the previous step. Make sure that the data are arranged correctly! In particular, make sure standard star observations were correctly identified. If changes need to be made, consult the 'Metadata Format' section below for help.
    3. Run 'python save_red_metadata.py' (and similarly for blue) from the command line. This will save your metadata in a python dictionary which gets pickled into a file e.g. 'wifesR_20120101_metadata.pkl'
  2. Reduce the Data - again in the 'scripts/' directory you will find 'reduce_red_data.py' (and similarly for blue). It is strongly recommended that the first time you use the reduction script you proceed one step at a time (see the 'Reduction Script' section below). This script is run from the command line with the pickled metadata file passed as argument: './reduce_red_data.py wifesR_20120101_metadata.pkl'

Metadata Format

The observation metadata is stored as a Python dictionary. Dictionaries are very nice objects because they let you access stored values (or other objects) by name (called a “key”), rather than by index.

Master calibration files for the night are stored as lists under appropriately named keys in the metadata dictionary. For example, if you are using the Python interpreter and load the metadata pickle file into an object called 'metadata', you can see the biases for the night like this: 

  >> print metadata['bias']
  ['r0001.fits', 'r0002.fits', 'r0003.fits']

Science and standard star observations (identified by the header keyword 'IMAGETYP' being 'OBJECT') are stored in the “metadata['sci']” and “metadata['std']” keys, but are stored differently than calibration files. Instead “metadata['sci']” is a list of Python dictionaries rather than a list of filenames.

This format was chosen so that individual science observations could have particular calibration files associated with them, e.g. if you took a bias just before or after which you want to use for bias subtraction, etc. This format is also critical for associating sky frames with the science frames they are intended to be subtracted from.

Let's again suppose you're inspecting the metadata in the Python interpreter, then each science observation dictionary would look like this: 

  >> print metadata['sci'][0]
  {'sci' : ['r0005.fits'],
   'arc' : [],
   'bias' : ['r0006.fits'],
   'sky' : ['r0007.fits']}

Where the 'sci' file is the main science image. Note that this calibration and sky frame association is the main reason you should check the auto-generated metadata scripts, which by necessity cannot know these associations! You as the observer must tell the script which calibrations or sky frames go with specific observations.

The standard star metadata is very similar, but each dictionary also has the additional key 'type', which tells the reduction script whether this star is a flux standard or telluric standard (or both). 

  >> print metadata['std'][0]
  {'sci' : ['r0008.fits'],
   'arc' : [],
   'type' : ['flux', 'telluric']}

Also important to note here is that an object frame only get automatically classified as a standard star if the 'OBJECT' header field is filled with a name from the standard star list defined in 'wifes_calib.py' in the 'src/' directory.

Reduction Script

Layout and details about the reduction script…

Individual Function Documentation

Basic Preprocessing Functions

Image coaddition / arithmetic

Cosmics Rays

Wavelength Solution

Wire Solution

Flat-fielding

WiFeS Data Formatting

Flux Calibration

Back to top