20000222: creating McIDAS MD files from archived text (cont.)

>From: Mark Anderson <address@hidden>
>Organization: University of Nebraska-Lincoln
>Keywords: 200002212031.NAA01679 McIDAS archive data

Mark,

Previously you asked:

>I was wondering where you could get archived data for McIDAS. I'm
>looking for the raobman and raobsig data sets for 17 and 18 Feb 2000.
>You used to be able to get them from Wisconsin, but there are not in the
>neartime data stream anymore.
>
>Any help would be appreciated.

I responded:

>>I put "raw" data files for 11, 12, 13, and 14Z on 17 and 18 Feb in
>>the pub/mcidas directory of anonymous FTP on our FTP server,
>>ftp.unidata.ucar.edu.  The files to grab are:
>>
>>2000021711_DDPLUS.wmo
>>2000021712_DDPLUS.wmo
>>2000021713_DDPLUS.wmo
>>2000021714_DDPLUS.wmo
>>2000021811_DDPLUS.wmo
>>2000021812_DDPLUS.wmo
>>2000021813_DDPLUS.wmo
>>2000021814_DDPLUS.wmo
>>
>>In order to create McIDAS MDXX files from these data files, you will
>>have to run them through the McIDAS-XCD upper air data monitor,
>>DMRAOB.  The absolute easiest way to do this is to insert the products
>>into the LDM's queue and let your system's XCD setup handle the rest.
>>The procedure would involve using pqinsert and may take some fiddling
>>with the feed type and pqact.conf entries.  Since I have never done
>>this before, I am not sure of exactly what it will take.  I will try to
>>see if I can come up with the procedure for doing this in the next day
>>or so.  The good news is that we have the reports themselves so the
>>rest is just engineering.

It turns out that running the old text data through the XCD decoders
was more involved than I thought/hoped.  I _believe_ that I was able to
correctly decode the data, but I need you to check the MD files to make
sure.  I put the files MDXX0017, MDXX0018, MDXX0019, MDXX0027,
MDXX0028, and MDXX0029 in the pub/mcidas directory of anonymous FTP on
our FTP server, ftp.unidata.ucar.edu.

You should:

o grab these files in binary mode
o put them in a directory that will not intefere with your realtime XCD
  data ingestion/decoding
o rename them if needed (like MDXX0017 -> MDXX1017, etc.)
o create an ADDE dataset that contains the files
o try using them in McIDAS and see if the decoded data looks reasonable

The procedure for using the XCD decoders was reasonably ugly.  What I
had to do was:

o modify ingetext.k and dmraob.k to add support for a DAY= keyword
  (the routine gets data from the realtime clock otherwise)

o start the RAOB data monitor, dmraob.k on a machine that had a user
  'mcidas' but did not have XCD running (so I wouldn't intefere with
  realtime data decoding)

  cd workdata
  dmraob.k DAY=2000048 DEB=15

  The DEB= keyword just set a debug output level so I could see the
  decoder do its thing.

o send the "raw" text files through the XCD ingetext.k ingestor:

  ingetext.k DDS DAY=2000048 < 2000021711_DDPLUS.wmo
  ingetext.k DDS DAY=2000048 < 2000021712_DDPLUS.wmo
  ingetext.k DDS DAY=2000048 < 2000021713_DDPLUS.wmo
  ingetext.k DDS DAY=2000048 < 2000021714_DDPLUS.wmo

  This left me with MD files for the 16th and 17th of February.

o kill and restart dmraob.k and ingest the next day's data:

  dmraob.k DAY=2000049 DEB=15

  ingetext.k DDS DAY=2000049 < 2000021811_DDPLUS.wmo
  ingetext.k DDS DAY=2000049 < 2000021812_DDPLUS.wmo
  ingetext.k DDS DAY=2000049 < 2000021813_DDPLUS.wmo
  ingetext.k DDS DAY=2000049 < 2000021814_DDPLUS.wmo

  This added some obs to the 17th MD files and created ones for the 18th.

I purposely chose raw text files for what I considered would hold
pertinent upper air obs for the days you were after, the 17th and 18th,
so I am a little puzzled by the existence of obs for the 16th.

I need you to take a look at the data and see if it makes sense.  If
need be, I could recreate the surface MD file so that you could see if
it has data that makes sense.

Since I did not see you FTPing the raw data files, I am uncertain of
whether or not you got my previous email.  Please let me know if/when
you get the MD files (and perhaps the text files) and the results of
your tests with them.

Tom