20000622: LDM setup at INM

>From: Pepo Juega <address@hidden>
>Organization: Instituto Nacional de Meteorologia
>Keywords: 200006220829.e5M8TZT11974 LDM

Pepo,

Our workshop is finally over, and I have time to respond to your questions.

>As I said:
>I still haven't managed to have a decent transfer between two 
>machines located here, in the same building. My intentions
>are to exchange GRID and AREA McIDAS files.

OK.  Let's attack this problem one step at a time.

Before delving into the setup of ldmd.conf on alfa.inm.es, let me comment
on the error you are eventually receiving:

>My ldmd.log file says:
>----------------------
>Jun 19 08:06:51 alfa rpc.ldmd[1123]: Starting Up (built: Jun 15 2000 14:34:03)
>Jun 19 08:06:51 alfa meteosat[1125]: run_requester: Starting Up: 
>meteosat.inm.es
>Jun 19 08:06:51 alfa meteosat[1125]: run_requester: 20000619070651.785
>TS_ENDT {
>{ANY,  ".*"}}
>Jun 19 08:06:51 alfa grid[1126]: run_requester: Starting Up: grid.inm.es
>Jun 19 08:06:51 alfa grid[1126]: run_requester: 20000619070651.803
>TS_ENDT {{ANY
>,  ".*"}}
>Jun 19 08:06:51 alfa pqexpire[1124]: Starting Up
>Jun 19 08:06:52 alfa grid[1126]: FEEDME(grid.inm.es): RPC: Program not 
>registered
>Jun 19 08:06:52 alfa meteosat[1125]: FEEDME(meteosat.inm.es): RPC:
>Program not registered


The program not registered is telling you that the LDM on the machine you
are going to make requests from is not running.  The process the LDM goes
through on startup is:

o read ldmd.conf to see
  o what data should be requested from whom
  o what machines will be allowed to request data

o start the processes listed in the 'exec' lines in ldmd.conf

The LDM won't be running if:

o it was never started

o there are errors in the ldmd.conf configuration file


Now, when the LDM on the machine being asked to send data (the 'request' line)
is not up and running, you will get a 'Program not registered' error.

>I use ldmsend somefile and nothing happens.
>I use ldmping meteosat.inm.es
>and I get:
>----------
>Jun 22 08:23:40      State    Elapsed Port   Remote_Host          
>rpc_stat
>Jun 22 08:23:40 SVC_UNAVAIL   0.372554    0   meteosat.inm.es  RPC:
>Program not registered
>
>
>Any light from this information?

All of these messages are telling you the same thing: the LDM on the
other machine is not running (or not running correctly (most likely
caused by a bad configuration).

The simplest test is:

o setup an ldmd.conf file on one machine that 'allow's another machine
  to request data from it

o setup an ldmd.conf file on the other machine that 'request's data
  from the first machine

o start the LDMs on both machines

o insert a product into the first machine's LDM queue (use pqinsert)

o see the product delivered to the second machine (use notifyme to
  see the product in the queue.  Something like:

  notifyme -vxl- -f MCIDAS -o 3600


Let's setup this simple test:

ldmd.conf on alfa.inm.es.  This will be the machine that feeds data to
the second machine, meteosat.inm.es:

####
#
# Programs that share a queue with rpc.ldmd
# are started by it and are in the same process group.
#
exec    "pqexpire -i 300"
#exec    "pqact"

###############################################################################
# ALLOW: Who we are willing to feed
#
# allow <feedset> <hostname pattern>
###############################################################################
#
# send anything to your own machine
allow   ANY
        ^((localhost|loopback)|(127\.0\.0\.1\.?$))

# Allow meteosat to request anything

allow   ANY     meteosat.inm.es


ldmd.conf on meteosat.inm.es.  This will be the machine that asks for
data from the first machine, alfa.inm.es:

####
#
# Programs that share a queue with rpc.ldmd
# are started by it and are in the same process group.
#
exec    "pqexpire -i 300"
#exec    "pqact"

# request McIDAS GRID and AREA files
request MCIDAS  ".*"    alfa.inm.es


Now, start the LDM on alfa.inm.es and verify that it is running:

ps -u ldm
(you should see an rpc.ldmd running if the LDM is installed correctly)

Now, start the LDM on meteosat.inm.es and verify that it is running.


At this point, alfa.inm.es doesn't have any data files in its queue, so
it has nothing to send to meteosat.inm.es.

After you get to the point where both LDMs are running with no errors,
you can inject products into the feed machine's queue (alfa.inm.es)
and see them arrive at the requesting machine (meteosat.inm.es).

<on alfa.inm.es>

pqinsert -f MCIDAS <AREA file name>

Once a product has been inserted into the product queue on alfa.inm.es,
it will be sent to all machines that have requested products of that
type. In our simple test, this will only be meteosat.inm.es.

The request is composed of the feed type and regular expression
describing the product header.  The default product header used by
pqinsert is the name of the file being inserted into the queue.

Before delving into your ldmd.conf and eventually pqact.conf files,
let's get an example setup like the one above working.

To answer an aside further down in your email, the pqact.conf file
defines the actions the LDM will perform upon receipt of a product
from an upstream feeder.  Without a pqact.conf entry for a product,
it simply remains in the queue until it gets scoured.  The simplest
action would be to simply file the product just received into
a disk file of your choice.  More interesting uses would be kicking
off McIDAS processes to analyze/display the data, etc.

>These are my conf files:
>
>ldmd.conf at alfa.inm.es
>------------------------
>ALLOW ANY meteosat.inm.es
>ALLOW ANY grid.inm.es
>ACCEPT ANY
>         "^AREA[0-9][0-9][0-9][0-9]"
>          meteosat.inm.es
>ACCEPT ANY
>         "^GRID[0-9][0-9][0-9][0-9]"
>          grid.inm.es
>REQUEST ANY ".*" meteosat.inm.es
>REQUEST ANY ".*" grid.inm.es
>EXEC "pqexpire -i 300"
>
>ldmadmin.conf at alfa.inm.es
>----------------------------
>ACCESS meteosat.inm.es oper mcsaida1 mcadde
>ACCESS grid.inm.es mcgrib mcadde
>HOSTNAME alfa.inm.es
>QUEUESIZE 25000000
>SURFSIZE 2000000
>NUMLOGS 8
>BINPATH $LDMHOME/bin
>ETCPATH $LDMHOME/etc
>LOGPATH $LDMHOME/logs
>DATAPATH $LDMHOME/data
>PQPATH $DATAPATH/ldm.pq
>SURFPATH $DATAPATH/pqsurf.pq
>PIDFILE $LDMHOME/ldmd.pid
>LOCKFILE /tmp/.ldmadmin.lck
>LOGFILE $LOGPATH/ldmd.log
>LDMDCONF $ETCPATH/ldmd.conf
>SCOURCONF $ETCPATH/scour.conf
>ADMINLOG $LDMHOME/logs/ldmadmin.log
>VERBOSE no
>LOGHOURS 24
>FEEDSET ANY
>
>netcheck.conf at alfa.inm.es
>----------------------------
>upstream:meteosat.inm.es
>upstream:grid.inm.es
>downstream:meteosat.inm.es
>downstream:grid.inm.es
>mailname:address@hidden
>
>scour.conf at alfa.inm.es
>-------------------------
># Directory                     Days-old        
>~ldm/data/GRIB                  2
>~ldm/data/AREA                  2
>
>pqact.conf at alfa.inm.es (I don't know what this is for)
>-------------------------
>#####
># MCIDAS products:
>#
># GRID files:
>#
>MCIDAS  ^(GUNRV2 .*)
>        PIPE
>        -close gunrv2
>        -l /usr/local/ldm/logs/ldm-mcidas.log -v
>        -d /var/data/mcidasd
># AREA files:
>#
>MCIDAS  ^(LWTOA3 .*)
>        PIPE
>        -close lwtoa3
>        -l /usr/local/ldm/logs/ldm-mcidas.log -v
>        -d /var/data/mcidasd
># mctingest(1) test:
>#
>MCIDAS  .*
>        PIPE    ~ldm/ldm-mcidas/solaris/bin/mctingest -v -e off -m .* --
>-
>
>####
>#
># Put everything else (e.g. new products that we don't know about) in
>daily
># "else" file.  Watch these to see when new products get added to HRS.
>#
>HRS     ^_ELSE_$        FILE    data/GRIB/%m%d_else.wmo
>
>Corresponding files in meteosat.inm.es and grid.inm.es are similar.
>
>My ldmd.log file says:
>----------------------
>Jun 19 08:06:51 alfa rpc.ldmd[1123]: Starting Up (built: Jun 15 2000
>14:34:03)
>Jun 19 08:06:51 alfa meteosat[1125]: run_requester: Starting Up:
>meteosat.inm.es
>Jun 19 08:06:51 alfa meteosat[1125]: run_requester: 20000619070651.785
>TS_ENDT {
>{ANY,  ".*"}}
>Jun 19 08:06:51 alfa grid[1126]: run_requester: Starting Up: grid.inm.es
>Jun 19 08:06:51 alfa grid[1126]: run_requester: 20000619070651.803
>TS_ENDT {{ANY
>,  ".*"}}
>Jun 19 08:06:51 alfa pqexpire[1124]: Starting Up
>Jun 19 08:06:52 alfa grid[1126]: FEEDME(grid.inm.es): RPC: Program not
>registere
>d
>Jun 19 08:06:52 alfa meteosat[1125]: FEEDME(meteosat.inm.es): RPC:
>Program not r
>egistered
>
>I use ldmsend somefile and nothing happens.
>I use ldmping meteosat.inm.es
>and I get:
>----------
>Jun 22 08:23:40      State    Elapsed Port   Remote_Host          
>rpc_stat
>Jun 22 08:23:40 SVC_UNAVAIL   0.372554    0   meteosat.inm.es  RPC:
>Program not registered
>
>
>Any light from this information?
>Thanks,
>Pepo
>+*+*+*+*+*+*+*+*+*+*+*+*+*+*+**+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*
>Jose Juega                        Instituto Nacional de Meteorologia.
>Tecnico de Sistemas                         Camino de las Moreras s/n
>Area de Telematica      JJJJJJJ   JJJJJJJ   Ciudad Universitaria     
>I.N.M.-Madrid-SPAIN       JJJ       JJJ     Aptdo. 285   28040 Madrid
>                          JJJ       JJJ     Tel :     +34 91 581-9654
>                      JJJ JJJ   JJJ JJJ     FAX :     +34 91 544-5307
>UAM (McIDAS)           JJJJJ     JJJJJ    e-mail:         address@hidden
>+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*++*++*+*+*+*+*+*+*+*+*+*+*+*+*+

Tom