20050127: Unidata McIDAS-XCD Trouble (cont.)

>From: "James T. Brown" <address@hidden>
>Organization: MSU
>Keywords: 200501201933.j0KJXOv2020815 McIDAS-XCD ADDE

Hi Jim,

Sorry for the lateness of this reply, but I was out of town until
yesterday afternoon.

>For the most part, everything seems to be running just fine now.

I _always_ like to hear comments like this :-)

>I made the following change to my "pqact.conf" and it seems to
>be creating the MDxx files just fine - the regular expression
>seems to have done the trick:
>
>WMO     ^([^HIJOLMPSYZ]|S[^D]) 
>        PIPE    xcd_run DDS
>
>HRS     ^.*    
>        PIPE    xcd_run HRS
>
>IDS|DDPLUS      ^.*    
>        PIPE    xcd_run DDS

I am concerned that there is two separate invocations of ingetext.k
(you should see 4 ingetext.k executions as two are parents and two are
children).  I would try eliminating the IDS|DDPLUS and only using the
WMO entry for text products:

WMO     ^([^HIJOLMPSYZ]|S[^D]) 
        PIPE    xcd_run DDS

Since WMO is a mnemonic for the combination of IDS|DDPLUS and HDS, you
should still get all of the data being sent from your NOAAPORT system
and any data ingested from your IDD IDS|DDPLUS feed that is not
eliminated as being a duplicate.

As to the entry to use for HDS data, I believe that it should look something
like:

WMO     ^[HIJOLMPSYZ]
        PIPE    xcd_run DDS

Since you are not (yet) decoding model output into McIDAS GRID format,
however, this change is not very important.

>I set up the DATALOC definitions as user mcidas on the "ADDE" machine
>and pointing TOPO to "accas.geo.msu.edu" instead of "<LOCAL-DATA>"
>is allowing all of the general mcidas user accounts to access the
>TOPO data group.

Sounds good.

>I am currently adding DATALOC references as I setup local data
>products.  Currently, I am processing and have access to the
>following groups (entires have been added to the "pqact.conf"
>file as well as the "LSSERVE.BAT" file on my McIDAS ADDE
>server:
>
>   RTIMAGES   accas.geo.msu.edu
>   RTNEXRAD   accas.geo.msu.edu
>   RTPTSRC    accas.geo.msu.edu
>   RTWXTEXT   accas.geo.msu.edu
>   TOPO       accas.geo.msu.edu

Yup, you are processing data in a way that creates data files that
populate these ADDE datasets.

>If I turn on the proper decoders, I should be able to setup local
>access to:
>
>   RTGRIDS

Yes.  This would be easy enough.  You are already running the first
part of the decoding process: ~ldm/etc/pqact.conf entry for 'xcd_run HDS'.
The second part is to turn on decoding in McIDAS:

<as 'mcidas'>

cd ~mcidas/workdata
decinfo.k SET DMGRID ACTIVE

Then, you need to uncomment the definitons for the RTGRIDS dataset in
LSSERVE.BAT and rerun 

cd ~mcidas/workdata
batch.k LSSERVE.BAT

Again, this is done by the user 'mcidas'.

>and possibly:
>
>   CIMSS

Yes.  The CIMSS dataset is composted of UNIWISC (aka MCIDAS) IDD
datastream products.  If you have turned on decoding of these images,
then you will be populating the data backend for the CIMSS dataset.

>I am not real sure about the NEXRCOMP group.  You mentioned that the
>NWSTG NOAAPort channel feed included NNEXRAD products - what about
>the FNEXRAD products?

The FNEXRAD data products are not entirely available through a NOAAPORT
reception system.  The FNEXRAD IDD datastream is currently composed of
composite images created from NEXRAD Level III products and three
"floater" NEXRAD sites.

The floater NEXRAD sites are going away since users can easily get the
same set in the IDD NNEXRAD stream.  We will be replacing the Level III
products with Level II full volume scan data that will be colocated
with the UNIWISC Educational Floater and FNEXRAD NEXRAD Level III
composite floater.  In addition, we will be running a regional model
(the workstation ETA at this point) over a Region of Interest (ROI)
that is objectively selected by a set of GEMPAK processes that is
evaluating where the most active weather area is in the Continental
US.  We will be moving towards the ROI procedure in the coming month or
so.  The reason for doing this is to simply demonstrate one of the
basic premises in LEAD; another is to automatically create sets of data
that can be viewed as realtime casestudies.

>Are those products included?

You would need to ingest the FNEXRAD feed through the IDD.

>If so, I
>will try to capture and setup local access, otherwise I will modify
>the DATALOC definitions to point to a remote server.

Either way will work.  The ldm-mcidas example pqact.conf entries
show how to setup decoding of FNEXRAD products that are the data
backend of the NEXRCOMP dataset.

>At any rate, everything is looking good.   Unless you need to login
>again to take a look at some of the recent changes, I may be able
>to "lock-down" the "mcidas" account.

I don't need to login unless you want me to.

>Thanks again for all your help!

No worries, glad to help.

>PS: You mentioned something about a change to the NOAAPORT system
>(change to DVB-S).   I talked to Dr. Jeff Andresen who was responsible
>for getting the NOAAPORT system here at Michigan State University and
>he wasn't even aware of the pending change.

Hmm...  I am suprised that he or someone else at MSU has not been
informed about the upcoming change by the vendor of your NOAAPORT
system!

>Is this being discussed
>on the LDM mailing list?

It has been mentioned a bit in the ldm-users email list.  It is a hot
topic among folks that are eager to get their feet wet in ingesting
using the DVB-S technology.  We are just about finished with
modifications needed to our NOAAPORT ingest software suite, and I am
happy to report that it appears to be working very nicely with the test
broadcast currently being conducted by the NWS.  Kudos go to Steve
Chiswell for his very nice work!

>Are there any other mailing lists that
>Jeff or myself need to be included in so that we can keep up-to-date
>with this pending change.

If you want to stay with your current NOAAPORT system, then someone
there should contact your vendor to see what they recommend/have to
offer.

>If a new system is going to be expensive,
>we may have to drop back to a single source (IDD) of data.

The beauty of the DVB-S system is that it is _not_ expensive.  We
followed the lead of the NWS and purchased a Novra S75 DVB-S receiver.
We have been modifying our software to work with the UDP multicast
stream that comes out of the Novra and inserts products into an LDM
queue correctly tagged in IDD datastreams.  It is mostly our systems
that are inserting the data that you have been receiving through the
IDD.

If you find that you can not afford to work with your current NOAAPORT
ingest system vendor, we can advise you on how to upgrade your system
to ingest the data from the DVB-S NOAAPORT broadcast.  In addition to
the hardware needed to ingest the DVB-S stream being cheap (the Novra
S75 is about $330; a second Ethernet interface for your ingest machine
can be purchased for about $15), you will be able to ingest _all_ of
the data being transmitted by the NWS.  I must warn you, however, that
this is a firehose of data compared to what you are receiving
currently.

Cheers,

Tom
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.