[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20010417: Data feed to UCR (cont.)
This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
- Subject: 20010417: Data feed to UCR (cont.)
- Date: Mon, 23 Apr 2001 14:53:35 -0600
>From: =?X-UNKNOWN?Q?Jimmy_Mejia_Fern=E1ndez?= <address@hidden>
>> >Organization: UCR
>> >Keywords: 200104170329.f3H3T4L28470 IDD MCIDAS
Jimmy,
I am sending you the following message as an addition to the reply you
should have gotten from Anne:
re: result of a notifyme
>OK, I dit it and I got this output:
>
>[ldm@inti ldm]$ notifyme -vxl- -f MCIDAS -o 3600 -h inti.efis.ucr.ac.cr
>Apr 18 03:20:01 notifyme[2556]: Starting
>Up: inti.efis.ucr.ac.cr: 20010418022001.630 TS_ENDT {{MCIDAS, ".*"}}
>Apr 18 03:20:01
>notifyme[2556]: NOTIFYME(inti.efis.ucr.ac.cr): 4: RPC: Unable to
>receive; errno = Broken pipe
>Apr 18 03:20:26
...
>Apr 18 03:23:21 notifyme[2556]: NOTIFYME(inti.efis.ucr.ac.cr): 7: Access
>denied by remote server
>Apr 18 03:23:46 notifyme[2556]: NOTIFYME(inti.efis.ucr.ac.cr): 7: Access
>denied by remote server
...
>That is strange because I remember that one time I ran a similar command
>and I was able to get data, I do not know what happened here.
OK, I was wrong in what I asked you to do. If you login to the machine
running the LDM as the user 'ldm', then the notifyme command is better
run as:
notifyme -vxl- -f MCIDAS -o 3600
This contacts 'localhost' and 'localhost' is enabled explicitly in ldmd.conf:
# UNIDATA
allow ANY
^((localhost|loopback)|(127\.0\.0\.1\.?$)|([a-z].*\.unidata\.ucar\.edu\.?$))
Since all Unidata machines are also enabled by this 'allow' line, I am
used to being able to specify the LDM host in the '-h hostname' flag
to notifyme. If you explicity 'allow' inti.efis.ucr.ac.cr in your
~ldm/etc/ldmd.conf file, you could also use the '-h inti.efis.ucr.ac.cr'
flag to notifyme. If not, you need to let the host default to 'localhost':
notifyme -vxl- -f MCIDAS -o 3600
>I ran:
> /etc/rc.d/init.d/portmap stop
> /etc/rc.d/init.d/portmap start
>
>and I got the same output.
The above problem was my fault, and the explanation is above.
I logged onto the upstream host you are trying to get MCIDAS imagery
and gridded data from (thelma.unidata.ucar.edu) to see if there is
anything that would prevent you from getting the imagery. The LDM log
file on thelma shows that the problem is most likely a very slow
connection between thelma and inti.efis:
Apr 23 03:24:14 5Q:thelma inti(feed)[1930]: RECLASS: 20010423022414.294 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:26:27 5Q:thelma inti(feed)[1930]: RECLASS: 20010423022627.016 TS_ENDT
{{HDS, ".*"}}
...
Apr 23 03:32:51 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023251.056 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:34:09 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023408.862 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:35:20 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023520.487 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:36:30 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023629.590 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:37:36 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023736.502 TS_ENDT
{{HDS, ".*"}}
Apr 23 03:39:31 5Q:thelma inti(feed)[1930]: RECLASS: 20010423023930.858 TS_ENDT
{{HDS, ".*"}}
...
Apr 23 13:42:01 3Q:thelma inti(feed)[1927]: h_clnt_call: inti.efis.ucr.ac.cr:
BLKDATA: time elapsed 38.216653
Apr 23 13:43:40 3Q:thelma inti(feed)[1927]: h_clnt_call: inti.efis.ucr.ac.cr:
BLKDATA: time elapsed 21.690497
The RECLASS messages are showing us that the LDM is skipping a number
of products (grids in this case (HDS feed type)) in order to catch up.
The 'h_clnt_call:' notices are letting us know that it is taking an
exceedingly long time to send pieces of a large product. It is likely
that the product that is being sent is one of the MCIDAS images as
these are the largest products that are sent in the IDD.
A 'traceroute' from thelma to inti.efis.ucr.ac.cr shows, however, that
the connection to your machine from thelma is not terrible (at least not
when I ran it).
Given this we have the following recommendation:
<login as 'ldm'>
ldmadmin stop
cd etc
<edit your ldmd.conf file and change your requests for data from:
request UNIDATA ".*" thelma.unidata.ucar.edu
to:
request DDPLUS|IDS|HRS ".*" thelma.ucar.edu
request MCIDAS "^pnga2area Q[01]" 192.52.106.21
Also, add an allow line for thelma.ucar.edu in addition to the general one
for Unidata machines:
allow ANY thelma.ucar.edu
The changes are:
o change request for data from thelma.unidata.ucar.edu to thelma.ucar.edu.
thelma.ucar.edu is the primary name of this machine
o split your request for data into two: request DDPLUS. IDS, and HRS
through thelma.ucar.edu, and MCIDAS from thelma through its IP address
o allowing thelma to do 'notifyme's to your machine
Changing the request to thelma's primary name will be more in line with
a standard LDM setup.
Splitting the request for a feed will insure that poor reception by the
large MCIDAS image products do not adversly impact the reception of the
rest of the products.
Adding the allow will allow us to do run notifyme to your LDM machine
from thelma. This does not work at the moment since the reverse name
lookup (the 'nslookup ip_address_of_thelma' returns thelma.ucar.edu
not thelma.unidata.ucar.edu.
Finally, restart the LDM:
ldmadmin start
>The pqact.conf file is include as an attachment:
As far as your pqact.conf entries for decoding PNG-compressed MCIDAS
images, these look OK as long as you have followed the recommendations
on the LDM-MCIDAS web pages that talk about making sure that you have
put copies of SYSKEY.TAB and ROUTE.SYS in /home/ldm/data/mcidas/area:
#
#### PNG compressed Unidata-Wisconsin channel image (AREA) files
#
# Example of decoding CIMSS products in MCIDAS stream using McIDAS-X routing
# table for product naming and possible execution of ROUTE PostProcess BATCH
# files.
MCIDAS ^pnga2area Q. (..) (.*) (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area -d /home/ldm/data/mcidas/area -r \1,\2
# Examples for storing decoded CIMSS output files in a hierarchical directory
# structure usable by GEMPAK/GARP/NMAP and McIDAS-X V 7.7 or higher
# CIMSS Cloud Top Pressure - McIDAS product code CA
MCIDAS ^pnga2area Q. CA .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/CTP/CTP_\4_\5
# CIMSS Total Column Precipitable Water - McIDAS product code CB
MCIDAS ^pnga2area Q. CB .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/PW/PW_\4_\5
# CIMSS Sea Surface Temperature - McIDAS product code CC
MCIDAS ^pnga2area Q. CC .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/SST/SST_\4_\5
# CIMSS Lifted Index - McIDAS product code CD
MCIDAS ^pnga2area Q. CD .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/LI/LI_\4_\5
# CIMSS CAPE - McIDAS product code CE
MCIDAS ^pnga2area Q. CE .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/CAPE/CAPE_\4_\5
# CIMSS Ozone - McIDAS product code CF
MCIDAS ^pnga2area Q. CF .... (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area /home/ldm/data/gempak/nport/SOUNDER/\3/OZONE/OZONE_\4_\5
The following entry for the old compression technique (LWTOA3 decoder) and
those for products you do not receive (e.g., NLDN lightning, FSL2
wind profiler, and WSI NIDS data), can be commented out or deleted:
#
#### Delta compressed Unidata-Wisconsin channel image (AREA) files
#
MCIDAS ^(LWTOA3 .*)
PIPE -close
lwtoa3 -d /home/ldm/data/mcidas/area -v
#
#### NLDN lightning product
#
NLDN
^([12][0-9][0-9][0-9]|[0-9][0-9])([0-3][0-9][0-9])([0-2][0-9])([0-5][0-9])([0-5][0-9])
PIPE -close
nldn2md -d /home/ldm/data/mcidas/area -v 70 NLDN DIALPROD=LD \1\2
\3\400 DEV=CCN
#
#### IDD FSL2 FSL Wind Profiler netCDF file to MD action
#
# Hourly summary
FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.01hr\.(.*)\..*
PIPE -close
proftomd -d /home/ldm/data/mcidas/area -v U2 WPRO 81
# 6-minute
FSL2 ^FSL\.NetCDF\.NOAAnet\.windprofiler\.06min\.(.*)\..*
PIPE -close
proftomd -d /home/ldm/data/mcidas/area -v U6 WPR6 91
#------------------------------------------------------------------------------
#
# ***** Obsolete IDD decoder actions as of 7/1/99 *****
#
#------------------------------------------------------------------------------
#
#### Unidata-Wisconsin channel FSL Wind Profiler netCDF file to MD action
#
#MCIDAS ^(LWFILE PROFILER.CDF)
# PIPE -close
# lwfile -d /home/ldm/data/mcidas/area -v
#MCIDAS ^(LWFILE) (PROFILER.CDF) .* DIALPROD=(..)
# EXEC
# cdftomd -d /home/ldm/data/mcidas/area -v \3 \2 81
#MCIDAS ^(LWFILE) (PROFILER.CDF) .* DIALPROD=(..)
# PIPE -close
# cdfconvert \3 \2 81
#
#### Unidata-Wisconsin channel gridded/model (GRID) files
#
#MCIDAS ^(GUNRV2 .*)
# PIPE -close
# gunrv2 -d /home/ldm/data/mcidas/area -v
#
#### Unidata-Wisconsin channel "text" files (i.e. NOT of type AREA, GRID or MD)
#
#MCIDAS ^(LWFILE) (ADMIN.MSG|UNIDATAS) .* DIALPROD=(..)
# PIPE -close
# lwfile -d /home/ldm/data/mcidas/area -v
#
#### Unidata-Wisconsin channel point source (MD) files
#
#MCIDAS ^(LWTMD2 .*)
# PIPE -close
# lwtmd2 -d /home/ldm/data/mcidas/area -v
#
#### WSI NIDS radar data products
#
#WSI
^NEX/(...)/(BREF1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8001 \1 DIALPROD=R1 \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(BREF2)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8002 \1 DIALPROD=R2 \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(BREF3)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8003 \1 DIALPROD=R3 \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(BREF4)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8004 \1 DIALPROD=R4 \3%j \6\700
DEV=CCN
# Action that creates Composite Reflectivity AREAs in the
/home/ldm/data/mcidas/area
# directory.
#
#WSI
^NEX/(...)/(CREF)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8005 \1 DIALPROD=R5 \3%j \6\700
DEV=CCN
# Actions that create Layer Reflectivity AREAs for the three layers: sfc-24 Kft,
# 24-33 K ft, and 33-60 Kft in the # /home/ldm/data/mcidas/area directory.
#
#WSI
^NEX/(...)/(LREF1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8006 \1 DIALPROD=R6 \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(LREF2)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8007 \1 DIALPROD=R7 \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(LREF3)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8008 \1 DIALPROD=R8 \3%j \6\700
DEV=CCN
# Action that creates Echo Tops AREAs in the /home/ldm/data/mcidas/area
directory.
#
#WSI
^NEX/(...)/(TOPS)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8009 \1 DIALPROD=R9 \3%j \6\700
DEV=CCN
# Action that creates Vertically Integrated Liquid Water AREAs in the
# /home/ldm/data/mcidas directory.
#
#WSI
^NEX/(...)/(VIL)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8010 \1 DIALPROD=RA \3%j \6\700
DEV=CCN
# Actions that create 1-hour, 3-hour, and Storm Total Rainfall total AREAs in
# the /home/ldm/data/mcidas directory.
#
#WSI
^NEX/(...)/(PRE1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8011 \1 DIALPROD=RB \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(PRE3)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8012 \1 DIALPROD=RC \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(PRET)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8013 \1 DIALPROD=RD \3%j \6\700
DEV=CCN
# Actions that create Radial Velocity AREAs at all 4 radar tilt in the
# /home/ldm/data/mcidas directory.
#
#WSI
^NEX/(...)/(VEL1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8014 \1 DIALPROD=RE \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(VEL2)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8015 \1 DIALPROD=RF \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(VEL3)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8016 \1 DIALPROD=RG \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(VEL4)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8017 \1 DIALPROD=RH \3%j \6\700
DEV=CCN
# Action that creates 248 nm Base Reflectivity AREA tilt in the
# /home/ldm/data/mcidas directory.
#
#WSI
^NEX/(...)/(BRLR1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8018 \1 DIALPROD=RI \3%j \6\700
DEV=CCN
# Action that creates Storm Relative Mean Velocity at two tilts in the
# /home/ldm/data/mcidas directory.
#
#WSI
^NEX/(...)/(SRMV1)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8019 \1 DIALPROD=RJ \3%j \6\700
DEV=CCN
#WSI
^NEX/(...)/(SRMV2)/..([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
# PIPE -close
# nids2area -d /home/ldm/data/mcidas/area 8020 \1 DIALPROD=RK \3%j \6\700
DEV=CCN
After changing the machine name/IP from which you are requesting data,
stopping the LDM, modifying your ldmd.conf and pqact.conf files, and
restarting the LDM, you should see better reception of at least non-MCIDAS
products. The MCIDAS images will be the hardest for you to get since
they are the largest.
Tom Yoksas
- Prev by Date: Re: 20010421: Data feed to UCR
- Next by Date: Re: 20010421: Thoughts on LDM reclasses
- Previous by thread: Re: 20010421: Data feed to UCR
- Next by thread: 20010424: Data feed to UCR (cont.)
- Index(es):