[McIDAS #DJR-180829]: Mcidas xcd vs ldm mcidas

Hi again Randy,
 
> Under the oper account there is a crontab which has several commands
> which delete MD , GRID etc files if they are older then X-days long.

OK.  The Unidata McIDAS release contains a Bourne shell script, mcscour.sh,
that scours McIDAS files (MD, GRID, *.XCD, *.IDT) and is also run from
cron.

> Last Friday, I changed the command which deletes the MD files older than
> 7 days to 30 days. 

Ouch.  I would have told you that you can NOT do this _if_ the data files
are kept in the standard McIDAS named files (e.g., MDXXnnnn).  The reason
for this is the XCD decoders always write to the end of the daily file
AND they use a naming convention that allows for 10 days of data:

MDXX0001 - MDXX0010 -- Surface files
MDXX0011 - MDXX0020 -- Mandatory Level Upper Air files
etc.

The second part of the naming convention is the last digit of the file
name is the same digit of the Julian day (e.g., today, February 14, 2006
is Julian day 2006045, so today's surface MD file is MDXX0005, the mandatory
level upper air file is MDXX0015, etc.).  This is one of the "warts" of
McIDAS that users have to learn how to deal with.

Now, if you wanted to keep 30 days of MD data of any type online, there
is a "solution":  simply rename the files to something other than what
is used by XCD decoders.  The tricky part if doing this is that the
current file can not be renamed until it is no longer being written
to (but a soft link to the file would work), and scouring of the data
can no longer be done by qrtmdg.k.

> Today 3 days after I made the change (00z) it broke.
> The MD files which were being written today from the LDM/ingestor could
> not be correctly read by McIDAS. It interpreted those files as 10 day
> old files.

It interpreted the data to be 10 days old because it was 10 days old.
The new data (today's data) gets written to the end of the file, not the
beginning.  Writing data to the end of the file will only work for a
short time, however, since most MD files have a limited size.  Once the
file reaches that size, the decoder writes will fail.

> As it turns out there is only room for 10 positions of MD files / data
> type (e.g., SFCHOURLY). So the program I guess got confused when it saw
> 30 days. The program name by the way is qrtmdg.k

Yup.  This is not fragility of XCD, but, rather, adherence to the
old and strict McIDAS naming convention for files.  Again, by renaming
the data files you can keep as many online as you like.

> The fix actually was simple. I just deleted todays MD files and after
> doing a decinfo.k SET DMSFC INACTIVE and then ACTIVE again. I need to be
> careful what I say about ssec support over this email.

Yup.

> Different subject:
> I have also downloaded and installed IDV and have been playing with it.
> Since my noaapxcd machine is decoding  grib data I wanted to load that
> up in IDV. So far no luck. The users guide does not specifically talk
> about model data.

I am uncertain if the IDV can access model output served through the
remote ADDE interface.  It should, however, be able to read the GRIB files
directly (I think).  I certainly can read netCDF files with model output.
You could, if desired, download our netCDF decoders package, and configure
it to decoded GRIB into netCDFs.  The IDV could then be used to read that
data directly.

> There was some mention of a catalogue which I tried to
> access from  your site but could not. Likely due to the Northrop Grumman
> Firewall.

I imagine that a firewall is the reason you could not access the THREDDS
catalog of model data.

> Is there an easy way to bring this data into IDV?

Try reading the GRIB files directly.  Alternatively, decode the GRIB data
into netCDF and then read those files directly.  Finally, get your IT group
to allow port 80 access (http) so you can get at the THREDDS catalog.


Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: DJR-180829
Department: Support McIDAS
Priority: Normal
Status: Closed