apero_trigger¶

Contents¶

1. Description
2. Schematic
3. Usage
4. Optional Arguments
5. Special Arguments
6. Output directory
7. Output files
8. Debug plots
9. Summary plots

1. Description¶

SHORTNAME: TRIGGER

The apero trigger runs continuiously and tries to accurately reduces night data.

It requires a master night to be processed BEFORE starting.

How it works¶

It works as follows:

copies files from a “live directory” into a sym-linked directory (the live directory is defined be –indir, the sym-lined directory is the raw directory defined in installation – i.e. DRS_DATA_RAW)

tries to figure out what has been done previously

this is done in two steps

first step: calibrations

it uses the log database and the trigger_night_calibrun.ini file to work out (per obs_dir) whether at least one of each recipe has been run (it counts QC failures as done)

if all steps are not complete it runs apero processing with the trigger_night_calibrun.ini run.ini file with the standard skips in apero_processing

apero_processing will stop if a recipe finds no runs (this is only true in TRIGGER_RUN=True mode

second step: science

this step is only done once all calibrations are deemed to be completed

it uses the log database, index database and the trigger_night_scirun.ini file to work out whether all recipes with science DPRTYPES have been run (per obs_dir)

if there aren’t the same number of raw science files as recipe runs (in the log database) it will attempt to re-run apero processing with the trigger_night_scirun.ini run.ini file with the standard skips in apero_processing

apero_processing will stop if a recipe finds no runs (this is only true in TRIGGER_RUN=True mode

Caveats¶

results may not be optimal - we recommended running all nights together in an offline manner (after having all nights) for optimal results, for example:
- calibrations may be sub-optimal (missing/using wrong night etc)
- telluric correction may not be using all hot stars
- templates may be sub-optimal
- polar recipes cannot be produced online
A master night must be run before running apero_trigger.py
Adding files for older nights after newer nights could result in unwanted behaviour (especially when it comes to calibrations)
Removing files may result in new calibrations being generated with less calibrations than before (apero_processing skip only works with the same number of files) and apero will use the most recently reduced calibration
Once calibrations are finished for a night remove/adding calibration files will not re-trigger the calibration sequence (unless –reset is used)
All obs_dir and files in the –indir will be processed, use –ignore to remove certain obs_dirs from the obs_dirs list

2. Schematic¶

No schematic set

3. Usage¶

apero_trigger.py {options}

No optional arguments

4. Optional Arguments¶

--indir[STRING] // [STRING] The input directory to scan for new data. (This is not the apero defined raw directory)
--reset // Reset the trigger (default is False and thus we use cached files to speed up trigger). This means after nights are marked done (calib/sci) they will not be reprocessed. Thus --reset to avoid this.
--ignore[STRING] // [STRING] Ignore certain obs_dir (observation directories) by default all directories in --indir are reduced. Using ignore will ignore certain directories and not add them to the the sym-linked (DRS_DATA_RAW) directory.
--wait[1>INT>3600] // [INTEGER] Number of second to wait between processing runs. Should not be too low (below 10s its too fast) unless testing, or too high (above 3600s)
--calib[STRING] // [STRING] The run.ini file to use for calibration trigger run
--sci[STRING] // [STRING] The run.ini file to use for science trigger run
--trigger_test // Active test mode (does not run recipes)

5. Special Arguments¶

--xhelp[STRING] // Extended help menu (with all advanced arguments)
--debug[STRING] // Activates debug mode (Advanced mode [INTEGER] value must be an integer greater than 0, setting the debug level)
--listing[STRING] // Lists the night name directories in the input directory if used without a 'directory' argument or lists the files in the given 'directory' (if defined). Only lists up to 15 files/directories
--listingall[STRING] // Lists ALL the night name directories in the input directory if used without a 'directory' argument or lists the files in the given 'directory' (if defined)
--version[STRING] // Displays the current version of this recipe.
--info[STRING] // Displays the short version of the help menu
--program[STRING] // [STRING] The name of the program to display and use (mostly for logging purpose) log becomes date | {THIS STRING} | Message
--recipe_kind[STRING] // [STRING] The recipe kind for this recipe run (normally only used in apero_processing.py)
--parallel[STRING] // [BOOL] If True this is a run in parellel - disable some features (normally only used in apero_processing.py)
--shortname[STRING] // [STRING] Set a shortname for a recipe to distinguish it from other runs - this is mainly for use with apero processing but will appear in the log database
--idebug[STRING] // [BOOLEAN] If True always returns to ipython (or python) at end (via ipdb or pdb)
--ref[STRING] // If set then recipe is a reference recipe (e.g. reference recipes write to calibration database as reference calibrations)
--crunfile[STRING] // Set a run file to override default arguments
--quiet[STRING] // Run recipe without start up text
--nosave // Do not save any outputs (debug/information run). Note some recipes require other recipesto be run. Only use --nosave after previous recipe runs have been run successfully at least once.
--force_indir[STRING] // [STRING] Force the default input directory (Normally set by recipe)
--force_outdir[STRING] // [STRING] Force the default output directory (Normally set by recipe)

6. Output directory¶

DRS_DATA_REDUC // Default: "red" directory

7. Output files¶

N/A

8. Debug plots¶

No debug plots.

9. Summary plots¶

No summary plots.