[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: 19990322: ldm problem
- Subject: Re: 19990322: ldm problem
- Date: Wed, 24 Mar 1999 14:04:43 -0700 (MST)
Clint,
The culprit sending duplicate data is Gilbert Sebenste
<address@hidden>
Thanks for checking,
Robb...
On Wed, 24 Mar 1999, Clint Rowe wrote:
> Robb and Jim,
>
> I've looked data and it appears that we are only getting a single
> set of data. I have not changed anything at this end in many months, so
> I don't see how I could be creating a problem. All our data is from the
> NOAAport feed via IDD (plus lightning from albany). There have been a lot of
> pq_sequence errors from hpccsun in our logs -- I agree that their queue may
> be corrupted, but I don't know why that would result in duplicate data.
>
> Clint
>
> >
> > Clint,
> >
> > Are you getting feeds from two sources, NOAAport, satellite, FOS ? It
> > appears the duplicates are flowing to the downstream sites.
> >
> >
> > Jim,
> >
> > If you give me a login, I'll look at the ldmadmin start problem. I thought
> > it was the HP security mechanism but I don't know what it is. Also, was
> > there are hw/sw changes lately?
> >
> > Robb....
> >
> >
> > On Tue, 23 Mar 1999, Jim Hines (awdnsun) 472-6708 wrote:
> >
> > > Robb
> > >
> > > I changed the check_registered and I am still getting
> > > the same error....
> > >
> > > Mar 22 19:00:17 UTC hpccsun.unl.edu : stop_ldm: Server not started or
> registered after 61 seconds
> > >
> > > My files are larger than they should be. Here is some examples
> > >
> > > The following directory listing shows hourly file size over the past
> > > month. Size is stable, then jumps on March 20th.
> > >
> > > These are SRUS 5* headers:
> > >
> > > -rw-r--r-- 1 ldm ldmgrp 73882 Mar 1 10:19 99030115.sro
> > > -rw-r--r-- 1 ldm ldmgrp 74413 Mar 2 10:19 99030215.sro
> > > -rw-r--r-- 1 ldm ldmgrp 80732 Mar 3 14:16 99030315.sro
> > > -rw-r--r-- 1 ldm ldmgrp 79075 Mar 4 11:17 99030415.sro
> > > -rw-r--r-- 1 ldm ldmgrp 73360 Mar 5 10:22 99030515.sro
> > > -rw-r--r-- 1 ldm ldmgrp 83463 Mar 6 10:21 99030615.sro
> > > -rw-r--r-- 1 ldm ldmgrp 72853 Mar 7 10:23 99030715.sro
> > > -rw-r--r-- 1 ldm ldmgrp 85399 Mar 8 10:23 99030815.sro
> > > -rw-r--r-- 1 ldm ldmgrp 88224 Mar 9 10:22 99030915.sro
> > > -rw-r--r-- 1 ldm ldmgrp 75836 Mar 10 10:24 99031015.sro
> > > -rw-r--r-- 1 ldm ldmgrp 73767 Mar 11 10:16 99031115.sro
> > > -rw-r--r-- 1 ldm ldmgrp 84939 Mar 13 15:21 99031215.sro
> > > -rw-r--r-- 1 ldm ldmgrp 84223 Mar 13 10:26 99031315.sro
> > > -rw-r--r-- 1 ldm ldmgrp 77376 Mar 14 10:25 99031415.sro
> > > -rw-r--r-- 1 ldm ldmgrp 75306 Mar 15 10:25 99031515.sro
> > > -rw-r--r-- 1 ldm ldmgrp 65347 Mar 16 10:28 99031615.sro
> > > -rw-r--r-- 1 ldm ldmgrp 71227 Mar 17 10:19 99031715.sro
> > > -rw-r--r-- 1 ldm ldmgrp 66315 Mar 18 10:25 99031815.sro
> > > -rw-r--r-- 1 ldm ldmgrp 64168 Mar 19 11:43 99031915.sro
> > > -rw-r--r-- 1 ldm ldmgrp 298815 Mar 20 11:10 99032015.sro
> > > -rw-r--r-- 1 ldm ldmgrp 302363 Mar 21 11:11 99032115.sro
> > > -rw-r--r-- 1 ldm ldmgrp 255646 Mar 22 11:14 99032215.sro
> > > -rw-r--r-- 1 ldm ldmgrp 96843 Mar 23 11:14 99032315.sro
> > >
> > > These are SAUS headers:
> > >
> > > /disk4/saus> ls -l 9903*06.sao
> > > -rw-r--r-- 1 ldm ldmgrp 352166 Mar 2 18:30 99030106.sao
> > > -rw-r--r-- 1 ldm ldmgrp 503295 Mar 2 07:47 99030206.sao
> > > -rw-r--r-- 1 ldm ldmgrp 341236 Mar 3 23:59 99030306.sao
> > > -rw-r--r-- 1 ldm ldmgrp 355425 Mar 4 03:18 99030406.sao
> > > -rw-r--r-- 1 ldm ldmgrp 373280 Mar 5 11:18 99030506.sao
> > > -rw-r--r-- 1 ldm ldmgrp 346361 Mar 6 03:53 99030606.sao
> > > -rw-r--r-- 1 ldm ldmgrp 345604 Mar 7 04:11 99030706.sao
> > > -rw-r--r-- 1 ldm ldmgrp 297425 Mar 8 08:58 99030806.sao
> > > -rw-r--r-- 1 ldm ldmgrp 14363 Mar 9 10:47 99030906.sao
> > > -rw-r--r-- 1 ldm ldmgrp 330816 Mar 10 06:09 99031006.sao
> > > -rw-r--r-- 1 ldm ldmgrp 340374 Mar 14 09:40 99031106.sao
> > > -rw-r--r-- 1 ldm ldmgrp 354273 Mar 13 00:00 99031206.sao
> > > -rw-r--r-- 1 ldm ldmgrp 334814 Mar 13 08:38 99031306.sao
> > > -rw-r--r-- 1 ldm ldmgrp 1195 Mar 14 10:35 99031406.sao
> > > -rw-r--r-- 1 ldm ldmgrp 343841 Mar 15 08:24 99031506.sao
> > > -rw-r--r-- 1 ldm ldmgrp 330082 Mar 16 04:13 99031606.sao
> > > -rw-r--r-- 1 ldm ldmgrp 329361 Mar 17 10:00 99031706.sao
> > > -rw-r--r-- 1 ldm ldmgrp 299322 Mar 18 10:14 99031806.sao
> > > -rw-r--r-- 1 ldm ldmgrp 336199 Mar 19 03:24 99031906.sao
> > > -rw-r--r-- 1 ldm ldmgrp 1301814 Mar 21 00:51 99032006.sao
> > > -rw-r--r-- 1 ldm ldmgrp 1283641 Mar 21 05:12 99032106.sao
> > > -rw-r--r-- 1 ldm ldmgrp 1861567 Mar 22 08:11 99032206.sao
> > > -rw-r--r-- 1 ldm ldmgrp 1220523 Mar 23 09:18 99032306.sao
> > > /disk4/saus>
> > >
> > >
> > >
> > > also here is a dup within a file .....
> > >
> > > SRUS53 KLBF 221517
> > > RR3LBF
> > > BIS
> > > .A ECSN1 0322 C DH07/PPP 0.40/SF 2/SD 2/TA 31/TX 55/TN 21/AD 2
> > >
> > > b
> > > SRUS53 KLBF 221517
> > > RR1LBF
> > >
> > > PRECIPITATION/SNOWFALL REPORTS
> > > NATIONAL WEATHER SERVICE NORTH PLATTE NE
> > > 915 AM CST MON MAR 22 1999
> > >
> > > :B LBF 0322 DH15/PP/SF/SD
> > > :STA ID PRECIPITATION/ SNOWFALL/ SNOWDEPTH/ STATION AND REMARKS
> > >
> > > AMEN1 / / 5 / :AMELIA 2 W
> > > ANSN1 0.27 / / 1 / :ANSELMO
> > > BUTN1 0.21 / 1.5 / / :BUTTE
> > > ELON1 0.11 / / / :ELLSWORTH
> > > ENDN1 0.00 / / / :ENDERS LAKE
> > > EUSN1 0.01 / / / :EUSTIS 2 NW
> > > HYNN1 0.18 / 2.0 / 2 / :HYANNIS 6 N
> > > IMPN1 0.00 / 0.0 / 0 / :IMPERIAL
> > > MDDN1 0.00 / / / :MADRID
> > > NPAN1 0.01 / / / :NORTH PLATTE 10 S
> > > ROSN1 / / 5 / :ROSE 10 WNW
> > > STAN1 0.15 / 0.4 / T / :STAPLETON 5 W
> > > SWAN1 0.32 / 4.1 / 4 / :SWAN LAKE
> > > TRYN1 0.10 / / T / :TRYON
> > > .END
> > >
> > >
> > >
> > >
> > > b
> > > SRUS53 KLBF 221517
> > > RR3LBF
> > > BIS
> > > .A ECSN1 0322 C DH07/PPP 0.40/SF 2/SD 2/TA 31/TX 55/TN 21/AD 2
> > >
> > > b
> > > SRUS53 KLBF 221517
> > > RR1LBF
> > >
> > > PRECIPITATION/SNOWFALL REPORTS
> > > NATIONAL WEATHER SERVICE NORTH PLATTE NE
> > > 915 AM CST MON MAR 22 1999
> > >
> > > :B LBF 0322 DH15/PP/SF/SD
> > > :STA ID PRECIPITATION/ SNOWFALL/ SNOWDEPTH/ STATION AND REMARKS
> > >
> > > AMEN1 / / 5 / :AMELIA 2 W
> > > ANSN1 0.27 / / 1 / :ANSELMO
> > > BUTN1 0.21 / 1.5 / / :BUTTE
> > > ELON1 0.11 / / / :ELLSWORTH
> > > ENDN1 0.00 / / / :ENDERS LAKE
> > > EUSN1 0.01 / / / :EUSTIS 2 NW
> > > HYNN1 0.18 / 2.0 / 2 / :HYANNIS 6 N
> > > IMPN1 0.00 / 0.0 / 0 / :IMPERIAL
> > > MDDN1 0.00 / / / :MADRID
> > > NPAN1 0.01 / / / :NORTH PLATTE 10 S
> > > ROSN1 / / 5 / :ROSE 10 WNW
> > > STAN1 0.15 / 0.4 / T / :STAPLETON 5 W
> > > SWAN1 0.32 / 4.1 / 4 / :SWAN LAKE
> > > TRYN1 0.10 / / T / :TRYON
> > > .END
> > >
> > >
> > >
> > >
> > > b
> > > SRUS53 KLBF 221517
> > > RR3LBF
> > > BIS
> > > .A ECSN1 0322 C DH07/PPP 0.40/SF 2/SD 2/TA 31/TX 55/TN 21/AD 2
> > >
> > > b
> > > SRUS53 KLBF 221517
> > > RR1LBF
> > >
> > > PRECIPITATION/SNOWFALL REPORTS
> > > NATIONAL WEATHER SERVICE NORTH PLATTE NE
> > > 915 AM CST MON MAR 22 1999
> > >
> > > :B LBF 0322 DH15/PP/SF/SD
> > > :STA ID PRECIPITATION/ SNOWFALL/ SNOWDEPTH/ STATION AND REMARKS
> > >
> > > AMEN1 / / 5 / :AMELIA 2 W
> > > ANSN1 0.27 / / 1 / :ANSELMO
> > > BUTN1 0.21 / 1.5 / / :BUTTE
> > > ELON1 0.11 / / / :ELLSWORTH
> > > ENDN1 0.00 / / / :ENDERS LAKE
> > > EUSN1 0.01 / / / :EUSTIS 2 NW
> > > HYNN1 0.18 / 2.0 / 2 / :HYANNIS 6 N
> > > IMPN1 0.00 / 0.0 / 0 / :IMPERIAL
> > > MDDN1 0.00 / / / :MADRID
> > > NPAN1 0.01 / / / :NORTH PLATTE 10 S
> > > ROSN1 / / 5 / :ROSE 10 WNW
> > > STAN1 0.15 / 0.4 / T / :STAPLETON 5 W
> > > SWAN1 0.32 / 4.1 / 4 / :SWAN LAKE
> > > TRYN1 0.10 / / T / :TRYON
> > > .END
> > >
> > >
> > >
> > >
> > > b
> > >
> > >
> > > I do not know what is causing the dups.
> > > Do you know? We did not change anything on
> > > our end.
> > >
> > > Thanks
> > > Jim Hines
> > >
> > >
> > >
> > >
> > >
> > > > >From address@hidden Mon Mar 22 16:06 CST 1999
> > > > >X-Authentication-Warning: wcfields.unidata.ucar.edu: rkambic owned
> process doing -bs
> > > > >Date: Mon, 22 Mar 1999 15:06:49 -0700 (MST)
> > > > >From: Robb Kambic <address@hidden>
> > > > >To: "Jim Hines (awdnsun) 472-6708" <address@hidden>
> > > > >Cc: support-ldm <address@hidden>
> > > > >Subject: Re: 19990322: ldm problem
> > > > >Mime-Version: 1.0
> > > > >
> > > > >On Mon, 22 Mar 1999, Jim Hines (awdnsun) 472-6708 wrote:
> > > > >
> > > > >> Robb
> > > > >>
> > > > >> I think I still have a problem...
> > > > >> You were right I ran out of disk space, I store
> > > > >> the complete feed, it usually runs about 80,000,000
> > > > >> but sometimes Friday it started getting better, I don't
> > > > >> think anything was changed on this end.
> > > > >>
> > > > >>
> > > > >> /data/ldm/zephyr/ARCHIVES> ls -l
> > > > >> total 2055408
> > > > >> -rw-r--r-- 1 ldm ldmgrp 78783582 Mar 15 17:59 wxfiles.990315
> > > > >> -rw-r--r-- 1 ldm ldmgrp 75313866 Mar 16 17:59 wxfiles.990316
> > > > >> -rw-r--r-- 1 ldm ldmgrp 79378339 Mar 17 18:00 wxfiles.990317
> > > > >> -rw-r--r-- 1 ldm ldmgrp 82350614 Mar 18 17:59 wxfiles.990318
> > > > >> -rw-r--r-- 1 ldm ldmgrp 129895221 Mar 19 18:50
> > > > >> wxfiles.990319
> > > > >> -rw-r--r-- 1 ldm ldmgrp 302479362 Mar 20 18:50
> > > > >> wxfiles.990320
> > > > >> -rw-r--r-- 1 ldm ldmgrp 217571328 Mar 21 15:49
> > > > >> wxfiles.990321
> > > > >> -rw-r--r-- 1 ldm ldmgrp 85962543 Mar 22 12:47 wxfiles.990322
> > > > >> /data/ldm/zephyr/ARCHIVES>
> > > > >>
> > > > >> You can see how the files got big.....
> > > > >>
> > > > >> also I got this email...
> > > > >> >
> > > > >> > >From ldm Fri Mar 19 12:54 CST 1999
> > > > >> > Date: Fri, 19 Mar 1999 12:54:30 -0600
> > > > >> > From: ldm (Unidata LDM)
> > > > >> > Subject: Local LDM is down - stop/start failed
> > > > >> >
> > > > >> > ldmfail: Mar 19 18:54:30 UTC
> > > > >> >
> > > > >> > LDM status report from the logs for the last 24 hours.
> > > > >> >
> > > > >> > Currently hpccsun is running 43 percent idle
> > > > >> > load average: 1.51, 0.64, 0.34
> > > > >> > Running version number 5.0.
> > > > >> > LDM was restarted 1 time(s)
> > > > >> > Last LDM restart at Mar 19 18:50:09
> > > > >> > Max Queue usage is 25001984 bytes, it occurred at Mar 19 18:50:05
> > > > >> >
> > > > >> > Critical LDM problems that need immediate attention:
> > > > >> >
> > > > >> > Potential LDM Problems:
> > > > >> >
> > > > >> > Decoder LDM Problems:
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >> I don't understand what the Critical LDM problem is????
> > > > >
> > > > >Jim,
> > > > >
> > > > >This script is becoming outdated because the log messages have changed
> > > > >so
> > > > >much, so don't worry about the error messages now.
> > > > >
> > > > >
> > > > >>
> > > > >> My guess is that when I got the Critical LDM problem
> > > > >> my files started growing faster!!!!
> > > > >>
> > > > >>
> > > > >> also now when I stop and start the ldm I get.....
> > > > >>
> > > > >> /usr/local/ldm> ldmadmin stop
> > > > >> stopping the LDM server...
> > > > >> LDM server stopped
> > > > >> /usr/local/ldm> ldmadmin start
> > > > >> starting the LDM server...
> > > > >> Mar 22 19:00:17 UTC hpccsun.unl.edu : stop_ldm: Server not started
> > > > >> or
> registered after 61 seconds
> > > > >> /usr/local/ldm>
> > > > >>
> > > > >> Why am I getting Server not started or registered????
> > > > >> the server is running because my files are growing....
> > > > >
> > > > >
> > > > >
> > > > >This will help the LDM start, it's a HP security problem. Change
> > > > >check_registered in bin/ldmadmin from :
> > > > >
> > > > >sub check_registered {
> > > > >
> > > > > $rpcinfo_cmd = "rpcinfo -t localhost 300029";
> > > > > `$rpcinfo_cmd 5 > /dev/null 2>&1`;
> > > > > if($?) {
> > > > > `$rpcinfo_cmd 4 > /dev/null 2>&1`;
> > > > > if($?) {
> > > > > return 1;
> > > > > }
> > > > > }
> > > > > return 0;
> > > > >}
> > > > >
> > > > >
> > > > >to
> > > > >
> > > > >sub check_registered {
> > > > >
> > > > > $rpcinfo_cmd = "rpcinfo -p | grep 300029";
> > > > > `$rpcinfo_cmd > /dev/null 2>&1`;
> > > > > if($?) {
> > > > > return 1;
> > > > > }
> > > > > return 0;
> > > > >}
> > > > >
> > > > >Also since your disk became full it's possible that you ldm queue is
> > > > >corrupted. I would ldmadmin stop/delqueue/mkqueue/start just to make
> sure
> > > > >it's ok. One can check if data is arriving by ldmadmin watch.
> > > > >
> > > > >
> > > > >Robb...
> > > > >
> > > > >
> > > > >>
> > > > >>
> > > > >> Thanks again
> > > > >> Jim Hines
> > > > >>
> > > > >
> > > >
> >===============================================================================
> > > > >Robb Kambic Unidata Program Center
> > > > >Software Engineer III Univ. Corp for Atmospheric
> Research
> > > > >address@hidden WWW: http://www.unidata.ucar..edu/
> > > >
> >===============================================================================
> > > > >
> > > > >
> > >
> >
> >
> ===============================================================================
> > Robb Kambic Unidata Program Center
> > Software Engineer III Univ. Corp for Atmospheric
> > Research
> > address@hidden WWW: http://www.unidata.ucar.edu/
> >
> ===============================================================================
> >
>
>
> ====================================================================
> Clinton M. Rowe
> Associate Professor
> Meteorology/Climatology Program phone:(402)472-1946
> Department of Geosciences fax:(402)472-4917
> University of Nebraska-Lincoln address@hidden
>
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================