20030604: problems with new LDM server

>From: David Fitzgerald <address@hidden>
>Organization: Milersville University
>Keywords: 200306041713.h54HDVLd004378 LDM

Hi Dave,

>I have just installed the most recent LDM version on a new Linux
>server.  I am replacing my Sun Ultra-5 as my LDM server and today was
>supposed to be the "big switch".  I had my new server in test mode
>feeding everything from my old machine just fine.  After changing the
>hostname/IP on the new machine and the ldmd.conf file to point to my
>upstream sites, the LDM starts up, then dies.

It seems to me that the fact that the LDM was running fine before you
changed its IP address and name shows that the problem is related to
the "personality change" and not the LDM.  Another user ran into a
similar problem just recently when bringing up a new Linux machine.
Here is the email I received from him that briefly describes what the
problem was (I suggested that he look at the output of 'hostname' on
his system to see if it was what he thought it should be):

  re: return value of 'hostname'

  >That did the trick. There was a conflict between what was in the /etc/hosts
  >file and what was in the /etc/sysconfig/network file. Data is flowing fine.

>I get a ton of
>pmap_unset(LDMPROG 300029, LDMVERS 5) failed messages in ldmd.log.  I
>can connect to all of my upstream sites via notifyme and traceroute
>The hep archives say, I may have a poor connection to my upstream
>sites, but I don't believe it.  My new LDM pukes even pulling down one
>product from one site, plus my old,slow Ultra-5 doesn't have any
>problems.

Please check the /etc/hosts and /etc/sysconfig/network files for
conflicts in the machine's hostname.

Also, did you do a reboot of the machine after doing the "personality
swap"?

>I have attached my ldmd.log file, could you help?

I just wanted to dash off the above ideas for you to look at.  I will
be discussing your ldmd.log file output (and the corresponding output
in the LDM logs on atm.geo.nsf.gov) with Steve Emmerson to see if he
can shed more light on your problem.

Tom

>Thanks!!
>Dave
>*********************************************
>David Fitzgerald
>Distributed System Specialist II
>Millersville University
>Millersville PA 17551
>Phone: 717-871-2394
>Fax:     717-871-4725
>E-mail: address@hidden
><mailto:address@hidden> 
>
>
>Jun 04 12:56:01 twister rpc.ldmd[2401]: Starting Up (version: 6.0.12; built: M
> ay 29 2003 15:50:38)
>Jun 04 12:56:01 twister pqexpire[2403]: Starting Up
>Jun 04 12:56:01 twister rtstats[2405]: Starting Up (2401)
>Jun 04 12:56:01 twister pqbinstats[2406]: Starting Up (2401)
>Jun 04 12:56:01 twister pqact[2407]: Starting Up
>Jun 04 12:56:01 twister atm[2408]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{UNIDATA,  ".*"}}
>Jun 04 12:56:01 twister atm[2410]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{HDS,  ".*"}}
>Jun 04 12:56:01 twister atm[2408]: Desired product class: 20030604135420.085 T
> S_ENDT {{UNIDATA,  ".*"}}
>Jun 04 12:56:01 twister atm[2410]: Desired product class: 20030604135420.085 T
> S_ENDT {{HDS,  ".*"}}
>Jun 04 12:56:01 twister atm[2411]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{FNEXRAD,  ".*"}}
>Jun 04 12:56:01 twister atm[2413]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{NNEXRAD,  "/p(.*)(CCX|LWX|DIX)"}}
>Jun 04 12:56:01 twister atm[2411]: Desired product class: 20030604135719.789 T
> S_ENDT {{FNEXRAD,  ".*"}}
>Jun 04 12:56:01 twister mapmaker[2415]: Starting Up(6.0.12): mapmaker.aos.wisc
> .edu: TS_ZERO TS_ENDT {{DIFAX,  ".*"}}
>Jun 04 12:56:01 twister atm[2416]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{UNIWISC,  ".*"}}
>Jun 04 12:56:01 twister atm[2413]: Desired product class: 20030604140223.696 T
> S_ENDT {{NNEXRAD,  "/p(.*)(CCX|LWX|DIX)"}}
>Jun 04 12:56:01 twister atm[2410]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2408]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2411]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2410]: Upstream LDM is willing to feed
>Jun 04 12:56:01 twister atm[2408]: Upstream LDM is willing to feed
>Jun 04 12:56:01 twister atm[2428]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{NIMAGE, "GOES-12/(VIS)/.*/EAST-CONUS"}}
>Jun 04 12:56:01 twister atm[2429]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{NIMAGE, "GOES-12/(VIS|IR|WV)/.*/SUPER-NATIONAL/8km"}}
>Jun 04 12:56:01 twister atm[2411]: Upstream LDM is willing to feed
>Jun 04 12:56:01 twister atm[2413]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2430]: Starting Up(6.0.12): atm.geo.nsf.gov: TS_ZE
> RO TS_ENDT {{FSL2,  ".*"}}
>Jun 04 12:56:01 twister atm[2429]: Desired product class: 20030604140055.860 T
> S_ENDT {{NIMAGE, "GOES-12/(VIS|IR|WV)/.*/SUPER-NATIONAL/8km"}}
>Jun 04 12:56:01 twister striker[2445]: Starting Up(6.0.12): striker.atmos.alba
> ny.edu: TS_ZERO TS_ENDT {{NLDN,  ".*"}}
>Jun 04 12:56:01 twister atm[2416]: Desired product class: 20030604141514.890 T
> S_ENDT {{UNIWISC,  ".*"}}
>Jun 04 12:56:01 twister atm[2413]: Upstream LDM is willing to feed Jun 04 12:5
> 6:01 twister atm[2430]: Desired product class: 20030604135804.762 TS_ENDT {{F
> SL2,  ".*"}}
>Jun 04 12:56:01 twister striker[2445]: Desired product class: 20030604123035.1
> 67 TS_ENDT {{NLDN,  ".*"}}
>Jun 04 12:56:01 twister MUGPS[2446]: Starting Up(6.0.12): MUGPS.millersville.e
> du: TS_ZERO TS_ENDT {{ANY,  ".*"}}
>Jun 04 12:56:01 twister MUGPS[2446]: Desired product class: 20030604135420.085
>  TS_ENDT {{ANY,  ".*"}}
>Jun 04 12:56:01 twister suomildm1[2449]: Starting Up(6.0.12): suomildm1.cosmic
> .ucar.edu: TS_ZERO TS_ENDT {{GPS|GPSSRC,  ".*"}}
>Jun 04 12:56:01 twister MUGPS[2446]: NOTICE: requester6.c:421; ldm_clnt.c:270:
>  nullproc_6 failure to MUGPS.millersville.edu;
>ldm_clnt.c:141: RPC: Program/version mismatch; low version3D 4, high
>version3D 5
>Jun 04 12:56:01 twister MUGPS[2446]: Connected to upstream LDM-5
>Jun 04 12:56:01 twister 129.15.194.231[2451]: Starting Up(6.0.12):
>129.15.194.231: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister MUGPS[2446]: FEEDME(MUGPS.millersville.edu): OK
>
>Jun 04 12:56:01 twister 129.15.194.231[2451]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.232[2453]: Starting Up(6.0.12):
>129.15.194.232: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister mapmaker[2415]: Desired product class:
>20030604115601.608 TS_ENDT {{DIFAX,  ".*"}}
>Jun 04 12:56:01 twister ldmtest[2450]: Starting Up(6.0.12):
>ldmtest.millersville.edu: TS_ZERO TS_ENDT {{WSI,  ".*"}}
>Jun 04 12:56:01 twister ldmtest[2450]: Desired product class:
>20030604124756.848 TS_ENDT {{WSI,  ".*"}}
>Jun 04 12:56:01 twister 129.15.194.233[2454]: Starting Up(6.0.12):
>129.15.194.233: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.232[2453]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.234[2455]: Starting Up(6.0.12):
>129.15.194.234: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.235[2456]: Starting Up(6.0.12):
>129.15.194.235: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister atm[2429]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister 129.15.194.234[2455]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.233[2454]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.237[2458]: Starting Up(6.0.12):
>129.15.194.237: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.235[2456]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.237[2458]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.236[2457]: Starting Up(6.0.12):
>129.15.194.236: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.238[2459]: Starting Up(6.0.12):
>129.15.194.238: TS_ZERO TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.238[2459]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister 129.15.194.236[2457]: Desired product class:
>20030604154258.460 TS_ENDT {{CRAFT,  ".KTLX"}}
>Jun 04 12:56:01 twister ldmtest[2450]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2428]: Desired product class:
>20030604115601.668 TS_ENDT {{NIMAGE,  "GOES-12/(VIS)/.*/EAST-CONUS"}}
>Jun 04 12:56:01 twister ldmtest[2450]: Upstream LDM is willing to feed
>Jun 04 12:56:01 twister suomildm1[2449]: Desired product class:
>20030604163505.886 TS_ENDT {{GPS|GPSSRC,  ".*"}}
>Jun 04 12:56:01 twister atm[2416]: Connected to upstream LDM-6
>Jun 04 12:56:01 twister atm[2429]: Upstream LDM is willing to feed
>Jun 04 12:56:01 twister atm[2430]: Connected to upstream LDM-6
>Jun 04 12:56:02 twister 129.15.194.231[2451]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister mapmaker[2415]: Connected to upstream LDM-6
>Jun 04 12:56:02 twister atm[2416]: Upstream LDM is willing to feed
>Jun 04 12:56:02 twister 129.15.194.232[2453]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister atm[2430]: Upstream LDM is willing to feed
>Jun 04 12:56:02 twister 129.15.194.234[2455]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister 129.15.194.233[2454]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister 129.15.194.235[2456]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister 129.15.194.237[2458]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister 129.15.194.238[2459]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister 129.15.194.236[2457]: Connected to upstream
>LDM-6
>Jun 04 12:56:02 twister atm[2428]: Connected to upstream LDM-6
>Jun 04 12:56:02 twister suomildm1[2449]: NOTICE: requester6.c:421;
>ldm_clnt.c:270: nullproc_6 failure to suomildm1.cosmic.ucar.edu;
>ldm_clnt.c:141: RPC: Program/version mismatch; low version3D 4, high =
>version3D 5
>Jun 04 12:56:02 twister suomildm1[2449]: Connected to upstream LDM-5
>Jun 04 12:56:02 twister mapmaker[2415]: Upstream LDM is willing to feed
>
>Jun 04 12:56:02 twister 129.15.194.231[2451]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.232[2453]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.234[2455]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.233[2454]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.235[2456]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.237[2458]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister 129.15.194.238[2459]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister atm[2428]: Upstream LDM is willing to feed
>Jun 04 12:56:02 twister 129.15.194.236[2457]: Upstream LDM is willing
>to feed
>Jun 04 12:56:02 twister suomildm1[2449]:
>FEEDME(suomildm1.cosmic.ucar.edu): OK
>Jun 04 12:56:09 twister rpc.ldmd[2401]: child 2405 terminated by signal
>11
>Jun 04 12:56:09 twister rpc.ldmd[2401]: Killing (SIGINT) process group
>Jun 04 12:56:09 twister pqexpire[2403]: Interrupt
>Jun 04 12:56:09 twister pqexpire[2403]: Exiting
>Jun 04 12:56:09 twister pqexpire[2403]: > Up since:     
>20030604125601.596
>Jun 04 12:56:09 twister atm[2408]: SIGINT
>Jun 04 12:56:09 twister rpc.ldmd[2401]: SIGINT
>Jun 04 12:56:09 twister striker[2445]: SIGINT
>Jun 04 12:56:09 twister pqexpire[2403]: > Queue usage (bytes):19886504
>Jun 04 12:56:09 twister pqexpire[2403]: >          (nregions):    1679
>Jun 04 12:56:09 twister pqexpire[2403]: > nprods deleted 0
>Jun 04 12:56:09 twister atm[2416]: SIGINT
>Jun 04 12:56:09 twister mapmaker[2415]: SIGINT
>Jun 04 12:56:09 twister MUGPS[2446]: SIGINT
>Jun 04 12:56:09 twister atm[2428]: SIGINT
>Jun 04 12:56:09 twister atm[2429]: SIGINT
>Jun 04 12:56:09 twister atm[2411]: SIGINT
>Jun 04 12:56:09 twister atm[2413]: SIGINT
>Jun 04 12:56:09 twister atm[2430]: SIGINT
>Jun 04 12:56:09 twister suomildm1[2449]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.234[2455]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.235[2456]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.236[2457]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.238[2459]: SIGINT
>Jun 04 12:56:09 twister atm[2410]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.231[2451]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.232[2453]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.233[2454]: SIGINT
>Jun 04 12:56:09 twister 129.15.194.237[2458]: SIGINT
>Jun 04 12:56:09 twister striker[2445]: pmap_unset(LDMPROG 300029,
>LDMVERS 5) failed
>Jun 04 12:56:09 twister atm[2416]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister nexradII[2489]: Interrupt
>Jun 04 12:56:09 twister ldmtest[2450]: SIGINT
>Jun 04 12:56:09 twister pqbinstats[2406]: Interrupt
>Jun 04 12:56:09 twister rpc.ldmd[2401]: pmap_unset(LDMPROG 300029,
>LDMVERS 5) failed
>Jun 04 12:56:09 twister mapmaker[2415]: pmap_unset(LDMPROG 300029,
>LDMVERS 5) failed
>Jun 04 12:56:09 twister MUGPS[2446]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister atm[2428]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister atm[2429]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister atm[2411]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister atm[2413]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister atm[2430]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister suomildm1[2449]: pmap_unset(LDMPROG 300029,
>LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.234[2455]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.235[2456]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.236[2457]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.238[2459]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister atm[2410]: pmap_unset(LDMPROG 300029, LDMVERS
>5) failed
>Jun 04 12:56:09 twister 129.15.194.231[2451]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.232[2453]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.233[2454]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister 129.15.194.237[2458]: pmap_unset(LDMPROG
>300029, LDMVERS 5) failed
>Jun 04 12:56:09 twister striker[2445]: pmap_unset(LDMPROG 300029,
>LDMVERS 6) failed
>Jun 04 12:56:09 twister atm[2416]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister pqact[2407]: Interrupt
>Jun 04 12:56:09 twister ldmtest[2450]: pmap_unset(LDMPROG 300029,
>LDMVERS 5) failed
>Jun 04 12:56:09 twister pqbinstats[2406]: Exiting
>Jun 04 12:56:09 twister mapmaker[2415]: pmap_unset(LDMPROG 300029,
>LDMVERS 6) failed
>Jun 04 12:56:09 twister MUGPS[2446]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister atm[2428]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister atm[2429]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister atm[2411]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister atm[2413]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister atm[2430]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister suomildm1[2449]: pmap_unset(LDMPROG 300029,
>LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.234[2455]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.235[2456]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.236[2457]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.238[2459]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister atm[2410]: pmap_unset(LDMPROG 300029, LDMVERS
>6) failed
>Jun 04 12:56:09 twister 129.15.194.231[2451]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.232[2453]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.233[2454]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister 129.15.194.237[2458]: pmap_unset(LDMPROG
>300029, LDMVERS 6) failed
>Jun 04 12:56:09 twister pqact[2407]: Exiting
>Jun 04 12:56:09 twister ldmtest[2450]: pmap_unset(LDMPROG 300029,
>LDMVERS 6) failed
>Jun 04 12:56:09 twister rpc.ldmd[2401]: pmap_unset(LDMPROG 300029,
>LDMVERS 6) failed
>Jun 04 12:56:09 twister rpc.ldmd[2401]: Terminating process group


>From address@hidden Wed Jun  4 12:21:56 2003

>Tom,

>Well I'm showing my stupid side today.  I forgot to change the /etc/hosts
>file.  I even had it on my list of changes to make!  D'OH

>Thanks that fixed it!

>Dave