20050310: LDM problem

Angel,

I'm having a hard time comparing the logfiles.  The times don't match or
overlap very well and ls1-afg's logfile has IP address instead of
hostnames.

What is the IP address of Deimos as used by ls1-afg?

Regards,
Steve Emmerson

------- Original Message

Date:    Thu, 10 Mar 2005 18:29:29 +0000
From:    "Angel Corona" <address@hidden>To:      Steve Emmerson <steve@u
          nidata.ucar.edu>
cc:      address@hidden
Subject: Re: 20050310: LDM problem

This is a multi-part message in MIME format.
--------------070509000605020706020802
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Here is the deimos log for the same time. Sorry.


Mar 09 17:56:46 deimos rpc.ldmd[12058]: Starting Up (version: 6.2.1; built: Mar
  9 2005 17:40:59) 
Mar 09 17:56:46 deimos pqbinstats[12059]: Starting Up (12058) 
Mar 09 17:56:46 deimos pqact[12060]: Starting Up 
Mar 09 17:56:46 deimos rtstats[12061]: Starting Up (12058) 
Mar 09 17:56:56 deimos ls1-afg(feed)[12063]: Starting Up(6.2.1/5): 200503091650
10.701 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 17:56:56 deimos ls1-afg(feed)[12063]: topo:  ls1-afg.fai.nwsar.gov EXP 
Mar 09 17:56:58 deimos ls1-afc(feed)[12064]: Starting Up(6.2.1/5): 200503091654
57.372 TS_ENDT {{EXP,  "mesoEta(U|S)_.*(00|12)Z_GribF.*"},{EXP,  "_akwGribF"}} 
Mar 09 17:56:58 deimos ls1-afc(feed)[12064]: topo:  ls1-afc.anc.nwsar.gov EXP 
Mar 09 17:57:03 deimos ls1-ajk(feed)[12066]: Starting Up(6.2.1/5): 200503091655
02.462 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 17:57:03 deimos ls1-ajk(feed)[12066]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 19:22:57 deimos ls1-afg(feed)[12063]: RECLASS: 20050309181812.128 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 19:22:57 deimos ls1-afg(feed)[12063]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 19:22:57 deimos ls1-afg(feed)[12063]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 19:22:57 deimos rpc.ldmd[12058]: child 12063 exited with status 1 
Mar 09 19:23:05 deimos ls1-ajk(feed)[12066]: RECLASS: 20050309182305.112 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:23:05 deimos ls1-ajk(feed)[12066]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 19:23:05 deimos ls1-ajk(feed)[12066]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 19:23:05 deimos rpc.ldmd[12058]: child 12066 exited with status 1 
Mar 09 19:56:48 deimos ls1-ajk(feed)[18253]: Starting Up(6.2.1/5): 200503091856
48.044 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:56:48 deimos ls1-ajk(feed)[18253]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 19:56:49 deimos ls1-ajk(feed)[18253]: RECLASS: 20050309185649.203 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:56:50 deimos ls1-ajk(feed)[18253]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 19:56:50 deimos ls1-ajk(feed)[18253]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 19:56:50 deimos rpc.ldmd[12058]: child 18253 exited with status 1 
Mar 09 19:57:20 deimos ls1-ajk(feed)[18260]: Starting Up(6.2.1/5): 200503091857
20.452 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:57:20 deimos ls1-ajk(feed)[18260]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 19:57:21 deimos ls1-ajk(feed)[18260]: RECLASS: 20050309185721.607 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:57:22 deimos ls1-ajk(feed)[18260]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 19:57:22 deimos ls1-ajk(feed)[18260]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 19:57:22 deimos rpc.ldmd[12058]: child 18260 exited with status 1 
Mar 09 19:57:52 deimos ls1-ajk(feed)[18294]: Starting Up(6.2.1/5): 200503091857
51.983 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:57:52 deimos ls1-ajk(feed)[18294]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 19:57:53 deimos ls1-ajk(feed)[18294]: RECLASS: 20050309185753.139 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 19:57:53 deimos ls1-ajk(feed)[18294]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 19:57:53 deimos ls1-ajk(feed)[18294]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 19:57:53 deimos rpc.ldmd[12058]: child 18294 exited with status 1 
Mar 09 20:24:58 deimos ls1-ajk(feed)[23752]: Starting Up(6.2.1/5): 200503091924
58.251 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:24:58 deimos ls1-ajk(feed)[23752]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 20:24:59 deimos ls1-ajk(feed)[23752]: RECLASS: 20050309192459.553 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:25:01 deimos ls1-ajk(feed)[23752]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 20:25:01 deimos ls1-ajk(feed)[23752]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 20:25:01 deimos rpc.ldmd[12058]: child 23752 exited with status 1 
Mar 09 20:25:31 deimos ls1-ajk(feed)[23755]: Starting Up(6.2.1/5): 200503091925
30.849 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:25:31 deimos ls1-ajk(feed)[23755]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 20:25:32 deimos ls1-ajk(feed)[23755]: RECLASS: 20050309192532.015 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:25:32 deimos ls1-ajk(feed)[23755]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 20:25:32 deimos ls1-ajk(feed)[23755]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 20:25:32 deimos rpc.ldmd[12058]: child 23755 exited with status 1 
Mar 09 20:26:02 deimos ls1-ajk(feed)[24214]: Starting Up(6.2.1/5): 200503091926
02.369 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:26:02 deimos ls1-ajk(feed)[24214]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 20:26:03 deimos ls1-ajk(feed)[24214]: RECLASS: 20050309192603.496 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:26:03 deimos ls1-ajk(feed)[24214]: RECLASS: 20050309192603.546 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:26:04 deimos ls1-ajk(feed)[24214]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 20:26:04 deimos ls1-ajk(feed)[24214]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 20:26:04 deimos rpc.ldmd[12058]: child 24214 exited with status 1 
Mar 09 20:26:34 deimos ls1-ajk(feed)[24215]: Starting Up(6.2.1/5): 200503091926
33.949 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:26:34 deimos ls1-ajk(feed)[24215]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 20:26:35 deimos ls1-ajk(feed)[24215]: RECLASS: 20050309192635.130 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:26:35 deimos ls1-ajk(feed)[24215]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 20:26:35 deimos ls1-ajk(feed)[24215]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 20:26:35 deimos rpc.ldmd[12058]: child 24215 exited with status 1 
Mar 09 20:27:05 deimos ls1-ajk(feed)[24216]: Starting Up(6.2.1/5): 200503091927
05.560 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:27:05 deimos ls1-ajk(feed)[24216]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 20:27:07 deimos ls1-ajk(feed)[24216]: RECLASS: 20050309192706.680 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "mesoEta.*"}} 
Mar 09 20:27:07 deimos ls1-ajk(feed)[24216]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 20:27:07 deimos ls1-ajk(feed)[24216]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 20:27:07 deimos rpc.ldmd[12058]: child 24216 exited with status 1 
Mar 09 21:03:19 deimos ls1-ajk(feed)[27468]: Starting Up(6.2.1/5): 200503092003
18.669 TS_ENDT {{EXP,  "akwGribF.*"}} 
Mar 09 21:03:19 deimos ls1-ajk(feed)[27468]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 21:03:20 deimos ls1-ajk(feed)[27468]: nullproc5(ls1-ajk.jnu.nwsar.gov): 
RPC: Unable to receive 
Mar 09 21:09:43 deimos ls1-ajk(feed)[27473]: Starting Up(6.2.1/5): 200503092009
43.493 TS_ENDT {{EXP,  "akwGribF.*"}} 
Mar 09 21:09:43 deimos ls1-ajk(feed)[27473]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 21:51:19 deimos ls1-afg(feed)[27524]: Starting Up(6.2.1/5): 200503092046
33.848 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 21:51:19 deimos ls1-afg(feed)[27524]: topo:  ls1-afg.fai.nwsar.gov EXP 
Mar 09 21:51:20 deimos ls1-afg(feed)[27524]: RECLASS: 20050309204634.934 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 21:51:20 deimos ls1-afg(feed)[27524]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 21:51:20 deimos ls1-afg(feed)[27524]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 21:51:20 deimos rpc.ldmd[12058]: child 27524 exited with status 1 
Mar 09 21:51:50 deimos ls1-afg(feed)[27527]: Starting Up(6.2.1/5): 200503092047
04.970 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 21:51:50 deimos ls1-afg(feed)[27527]: topo:  ls1-afg.fai.nwsar.gov EXP 
Mar 09 23:01:46 deimos ls1-ajk(feed)[27473]: RECLASS: 20050309220145.837 TS_END
T {{EXP,  "akwGribF.*"}} 
Mar 09 23:01:47 deimos ls1-ajk(feed)[27473]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:01:47 deimos ls1-ajk(feed)[27473]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:01:47 deimos rpc.ldmd[12058]: child 27473 exited with status 1 
Mar 09 23:01:52 deimos ls1-afg(feed)[27527]: RECLASS: 20050309215706.276 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 23:01:52 deimos ls1-afg(feed)[27527]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:01:52 deimos ls1-afg(feed)[27527]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:01:52 deimos rpc.ldmd[12058]: child 27527 exited with status 1 
Mar 09 23:02:01 deimos ls1-afc(feed)[12064]: RECLASS: 20050309220201.208 TS_END
T {{EXP,  "mesoEta(U|S)_.*(00|12)Z_GribF.*"},{EXP,  "_akwGribF"}} 
Mar 09 23:02:02 deimos ls1-afc(feed)[12064]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:02:02 deimos ls1-afc(feed)[12064]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:02:02 deimos rpc.ldmd[12058]: child 12064 exited with status 1 
Mar 09 23:02:17 deimos ls1-ajk(feed)[28288]: Starting Up(6.2.1/5): 200503092202
17.014 TS_ENDT {{EXP,  "akwGribF.*"}} 
Mar 09 23:02:17 deimos ls1-ajk(feed)[28288]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 23:02:18 deimos ls1-ajk(feed)[28288]: RECLASS: 20050309220218.231 TS_END
T {{EXP,  "akwGribF.*"}} 
Mar 09 23:02:18 deimos ls1-ajk(feed)[28288]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:02:18 deimos ls1-ajk(feed)[28288]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:02:18 deimos rpc.ldmd[12058]: child 28288 exited with status 1 
Mar 09 23:02:22 deimos ls1-afg(feed)[28289]: Starting Up(6.2.1/5): 200503092157
36.360 TS_ENDT {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 23:02:22 deimos ls1-afg(feed)[28289]: topo:  ls1-afg.fai.nwsar.gov EXP 
Mar 09 23:02:23 deimos ls1-afg(feed)[28289]: RECLASS: 20050309215737.459 TS_END
T {{EXP,  "akwGribF.*"},{EXP,  "^mesoEta.*GribF.*"}} 
Mar 09 23:02:23 deimos ls1-afg(feed)[28289]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:02:23 deimos ls1-afg(feed)[28289]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:02:23 deimos rpc.ldmd[12058]: child 28289 exited with status 1 
Mar 09 23:02:48 deimos ls1-ajk(feed)[28300]: Starting Up(6.2.1/5): 200503092202
48.555 TS_ENDT {{EXP,  "akwGribF.*"}} 
Mar 09 23:02:48 deimos ls1-ajk(feed)[28300]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 23:02:50 deimos ls1-ajk(feed)[28300]: RECLASS: 20050309220249.697 TS_END
T {{EXP,  "akwGribF.*"}} 
Mar 09 23:02:50 deimos ls1-ajk(feed)[28300]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:02:50 deimos ls1-ajk(feed)[28300]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:02:50 deimos rpc.ldmd[12058]: child 28300 exited with status 1 
Mar 09 23:03:20 deimos ls1-ajk(feed)[28301]: Starting Up(6.2.1/5): 200503092203
20.355 TS_ENDT {{EXP,  "akwGribF.*"}} 
Mar 09 23:03:20 deimos ls1-ajk(feed)[28301]: topo:  ls1-ajk.jnu.nwsar.gov EXP 
Mar 09 23:03:21 deimos ls1-ajk(feed)[28301]: RECLASS: 20050309220321.491 TS_END
T {{EXP,  "akwGribF.*"}} 
Mar 09 23:03:22 deimos ls1-ajk(feed)[28301]: forn5_svc.c:127: blkdata5(0): RPC:
 Remote system error 
Mar 09 23:03:22 deimos ls1-ajk(feed)[28301]: pq_sequence failed: Input/output e
rror (errno = 5) 
Mar 09 23:03:22 deimos rpc.ldmd[12058]: child 28301 exited with status 1 
Mar 09 23:03:27 deimos rpc.ldmd[12058]: SIGTERM 
Mar 09 23:03:27 deimos rpc.ldmd[12058]: Terminating process group 
Mar 09 23:03:27 deimos pqbinstats[12059]: Interrupt 
Mar 09 23:03:27 deimos pqbinstats[12059]: Exiting 
Mar 09 23:03:27 deimos pqact[12060]: Interrupt 
Mar 09 23:03:27 deimos pqact[12060]: Exiting 
Mar 09 23:03:27 deimos rtstats[12061]: Interrupt 
Mar 09 23:03:27 deimos rtstats[12061]: Exiting 
Mar 09 23:03:28 deimos ldmping[28311]: SVC_UNAVAIL   0.001450    0   localhost 
   RPC: Program not registered 
Mar 09 23:05:24 deimos ldmping[28321]: SVC_UNAVAIL   0.001489    0   localhost 
   RPC: Program not registered 
Mar 09 23:05:27 deimos ldmping[28329]: SVC_UNAVAIL   0.001534    0   localhost 
   RPC: Program not registered 
Mar 09 23:11:27 deimos ldmping[1914]: SVC_UNAVAIL   0.001515    0   localhost  
  RPC: Program not registered 
Mar 09 23:11:41 deimos ldmping[1922]: SVC_UNAVAIL   0.001659    0   localhost  
  RPC: Program not registered 
Mar 09 23:11:41 deimos pqcheck[1924]: Starting Up (1917) 
Mar 09 23:11:41 deimos pqcheck[1924]: The writer-counter of the product-queue i
s 0 
Mar 09 23:11:41 deimos pqcheck[1924]: Exiting 
Mar 09 23:12:10 deimos ldmping[1933]: SVC_UNAVAIL   0.001552    0   localhost  
  RPC: Program not registered 
Mar 09 23:12:10 deimos pqcheck[1937]: Starting Up (1928) 
Mar 09 23:12:10 deimos pqcheck[1937]: The writer-counter of the product-queue i
s 0 
Mar 09 23:12:10 deimos pqcheck[1937]: Exiting 





Steve Emmerson wrote:

>Angel,
>
>  
>
>>Date: Thu, 10 Mar 2005 18:05:51 +0000
>>From: "Angel Corona" <address@hidden>
>>Organization: NOAA
>>To: Steve Emmerson <address@hidden>
>>Subject: Re: 20050310: LDM problem
>>    
>>
>
>The above message contained the following:
>
>  
>
>>Here is the log file from the "ls1-afg" that is at the same time. Just
>>for clarification. It would be the ls1-afg (running ldm5) that was so
>>called "crashing" and not deimos (running ldm6) which was the log that
>>I had already sent you.
>>
>>
>>
>>Mar 09 22:59:00 ls1-afg rpc.ldmd[26598]: Starting Up (built: Aug 13 1999 11:2
4:09)
>>    
>>
>[snip]
>
>The timestamp on the above is later than the last timestamp of my
>previous reply (which was "Mar 09 19:22:57").  I need the entries to
>cover the same time interval.  Please retry.
>
>Regards,
>Steve Emmerson
>  
>

--------------070509000605020706020802
Content-Type: text/x-vcard; charset=utf-8;
 name="angel.corona.vcf"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="angel.corona.vcf"

begin:vcard
fn:Angel Corona
n:Corona;Angel
org:National Weather Service - Alaska Region;Systems Integration Branch
adr;dom:;;222 W. 7th Ave. #23 Rm. 517;Anchorage;AK;99513-7575
email;internet:address@hidden
title:Special Applications Program Manager
tel;work:(907)271-4428
tel;fax:(907)271-3711
tel;cell:(907)229-5614
x-mozilla-html:FALSE
url:http://www.arh.noaa.gov
version:2.1
end:vcard


--------------070509000605020706020802--

------- End of Original Message