This web document is intended to provide information on the VLBA data calibration pipeline. That is, we are currently running data of selected VLBA projects correlated at the VLBA correlator in Socorro through an automated process using AIPS to provide a best effort a-priori calibration of the data.
What is the VLBA data calibration pipeline?
The VLBA data calibration pipeline is a 'runfile' in AIPS which is primarily based on the AIPS VLBAUTIL procedures. The VLBAUTIL procedures are described in Appendix C of the AIPS Cookbook. The VLBA data calibration pipeline performs calibration tasks and makes lots of plots and some simple images to find out whether the calibration has worked - the VLBA data calibration pipeline is not intended to produce high fidelity images. The VLBA data calibration pipeline will be available to AIPS users as part of the AIPS distribution and the Midnight Job.
Restrictions for use of the VLBA data calibration pipeline
The VLBA data calibration pipeline is an effort to streamline and simplify customary calibration of data from the Very Long Baseline Array (VLBA). The VLBA is operated by the National Radio Astronomy Observatory (NRAO) as a facility of the National Science Foundation (NSF) operated under cooperative agreement by Associated Universities, Inc. (AUI).
Current possibilities and limitations of the VLBA data calibration pipeline
Below we have collected some current 'can-do' and 'cannot-do' features of the VLBA data calibration pipeline, which will be updated after new releases.
Updates and older versions of the VLBA data calibration pipeline scripts
We intend to archive older versions of the VLBA data calibration pipeline, along with older versions of the VLBAUTIL scripts. This should accommodate sites not running the AIPS 'Midnight Job' (MNJ) and make it possible to recreate 'features' (that is, check the results in one version with a different version if one does not trust the results). We will provide the collected older versions and keep track of new updates and their revision history at this spot in the near future. We anticipate however, that the sites running the MNJ get updates and new releases through the MNJ so that they always have the hottest versions (and their bugs? ;-) without any special intervention.
|Latest pipeline version in AIPS:||updates with DOUVCOMP, messages not suppressed, new HELP, PLOT index|
|Pipeline version before 25NOV02:||updates, and NPOINTS->NCHAV in IMAGR; first MNJ version|
|Pipeline version before 25JUN02:||overhaul to new VLBAUTIL (frequency ID split)|
|Pipeline version before 25MAR02:||added pol/spectral-line, surveys|
|Pipeline version before 25JAN02:||plain continuum version, including self-cal imaging|
Please contact Loránt Sjouwerman if you have data in the correlator queue (i.e. your data has been observed recently) and you would like to obtain your pipeline calibrated data. Highest priority will be given to projects of novice users for which the VLBA data calibration pipeline is known to work; though the latter may change on a regular basis! Lowest priority will be projects out of range of the current capabilities, projects of experienced users, and projects observed in the more distant past.
If you are running the MNJ, you are almost set to go. If not, you have to get the two VLBAPIPE files in the table below and put them in the appropriate places. We highly recommend that you use a local environment to place your aips pipeline files. However, if you don't want to keep the files in your local environment (see section 3.10 (page 3-16) of the AIPS Cookbook) and make the VLBA data calibration pipeline script available to other users at your institute, it is easiest to source the AIPS LOGIN.[C]SH file, and then type $CDTST to find the variable $RUNFIL and $HLPFIL (usually, if your AIPS installation is, for example, in /home/AIPS, then $RUNFIL is usually /home/AIPS/RUN, and $HLPFIL would be /home/AIPS/31DEC02/HELP). HOWEVER, the latter is highly discouraged in case you want to edit the procedure to your own likings and/or run the MNJ - always ONLY do that in your local environment! (You also wisely may not have the privileges to do that). Alternatively, pre-defined variables such as $HOME, $FITS, $DA01 (your first AIPS data disk), etc, can be used as a local environment (but a separate AIPS-runfile directory is preferred). After placing the files in one of these directories, define the non-predefined directory as a variable (e.g. in csh: setenv MYAREA `pwd` before starting AIPS!) - skip this step for HOME, FITS, etc, and if you have the MNJ enabled. Start up AIPS as usual, and if you do not use the default $RUNFIL area, but e.g. your $MYAREA area to place the pipeline scripts, remember to initialize the 'local version'. That is, within AIPS you make the files in the local directory available to AIPS by typing (e.g.) version 'myarea' or version 'home'. Then type explain vlbapipe within AIPS to read the help available.
The AIPS 'helpfile' contains information on how to use the 'runfile' and is maintained properly. In case it bombs on you, it may be necessary to also fetch the 'frozen utilities' - a version of the VLBAUTIL procedures that are known to work with the VLBAPIPE script. If it still fails, it may be that the individual underlying AIPS tasks have changed. Either update your AIPS installation, or send us a bug report (depending on your AIPS version and latest overhaul of the AIPS MNJ version) - if you use an AIPS version older than 31DEC02, don't expect the pipeline 'runfile' to work.
|Current VLBA pipeline runfile script:||VLBAPIPE.001||Put this in your $MYAREA (or $RUNFIL) directory|
|Current VLBA pipeline helpfile text:||VLBAPIPE.HLP||Put this in your $MYAREA (or $HLPFIL) directory|
|Temporary frozen VLBAUTIL procedures:||VLBAUTIL.001||You may need this too (in $MYAREA, or $RUNFIL)|
For both the MNJ and newly fetched files, you have to type run vlbautil and run vlbapipe (each on a separate command line) in your AIPS window to make sure you are using the latest 'runfiles'. If you forget this, either you might end up using an older version, or it will complain about not finding or recognizing the command. The same will happen if you forget to place the 'helpfile' or 'runfiles' in the appropriate directories (or misplace them).
After defining the procedure in AIPS, read the 'helpfile' for its use (type explain vlbapipe) and type inputs vlbapipe to review its current parameter settings. Invoke the VLBA data calibration pipeline by simply typing vlbapipe; it will check the inputs for possible problems, and if none are found, tell it that the tape is ready to be read (by typing 0 when the tape drive has settled down) and/or sit back and let the VLBA data calibration pipeline do it's work. Do not forget to check the plots and images it has made, as they may point to specific problems with your data set (dead IF channels, RFI, failing fringe-fitting, etc). If something is definitely wrong, usually these problems are solved by flagging the bad data (in flag/FG table one!) and rerunning the VLBA data calibration pipeline. Other common problems are that you selected a time range for your bandpass calibrator in which it has too few visibilities (e.g. when it is late on source), in which the source is below the horizon for one or more antennas, or the timerange does not correspond to the scan on your bandpass calibrator - change either or both. For other problems contact Loránt Sjouwerman
|Pipeline version before 25NOV02:||VLBAPIPE.001_25nov02||VLBAPIPE.HLP_25nov02||VLBAUTIL.001_25nov02|
|Pipeline version before 25JUN02:||VLBAPIPE.001_25jun02||VLBAPIPE.HLP_25jun02||VLBAUTIL.001_25jun02|
|Pipeline version before 25MAR02:||VLBAPIPE.001_25mar02||VLBAPIPE.HLP_25mar02||VLBAUTIL.001_25mar02|
|Pipeline version before 25JAN02:||VLBAPIPE.001_25jan02||VLBAPIPE.HLP_25jan02||VLBAUTIL.001_25jan02|
Please consult one of the following if you have any questions regarding parts of the VLBA data calibration pipeline. Most likely you want to contact Loránt Sjouwerman, but if your question obviously regards one of the other items listed in the table below, the relevant addresses are given as well.
|Loránt Sjouwermanfirstname.lastname@example.org||VLBA data calibration pipeline scripts & data|
|Amy Mioduszewskiemail@example.com||General VLBA utility (VLBAUTIL) scripts|
|Cast of AIPS||AIPS Cookbook Index||VLBA data reduction in AIPS|
|Eric Greisenfirstname.lastname@example.org||AIPS data reduction package|
|Claire Chandleremail@example.com||VLBA scientific services|
|Data firstname.lastname@example.org||VLBA logistics (schedules/correlation/shipments)|
Again, when you have calibrated your data using the VLBA data calibration pipeline, it would be appreciated if you would mention the VLBA pipeline next to an acknowledgment to NRAO in the publication of your scientific results; e.g. "The VLBA data was calibrated using NRAO's "VLBA data calibration pipeline" in AIPS (Sjouwerman, Mioduszewski & Greisen, in preparation). Thanks!
We on our side acknowledge the efforts of the Cast of AIPS, in particular Eric Greisen, for making and maintaining the AIPS-tasks and environment needed to reduce VLBI data in AIPS over the years. Chris Flatters started collecting the tasks into the VLBAUTIL procedures, which was continued by Amy Mioduszewski; the VLBAPIPE procedures heavily depend on these VLBAUTIL procedures. Finally, Greg Taylor allowed me to work on this pipeline.
Pipeline ideology / Copyrights, etc. / Current status / Scripts & releases / Pipelined projects / Points of contact
VLBA Home Page
The VLBA data calibration pipeline processing webpage originated June 5th, 2002.
Maintained by Loránt Sjouwerman: email@example.com.