[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed
- Subject: Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed
- Date: Thu, 4 Nov 1999 09:45:38 -0700 (MST)
Gregory,
After looking at the log messages, it doesn't appear your queue is 250
megabytes. Look in the data dir, what's the:
% ls -l ldm.pq
If it's 250 megabytes then up it again. There might be other products
coming in on your afos stream, etc. If it isn't then change it in the
bin/ldmadmin file to 250. Then the standard
% ldmadmin stop
% ldmadmin delqueue
% ldmadmin mkqueue
% ldmadmin start
I just put the commands here for my sanity sake.
I assume you are using the same ldmd.conf file for the ldm-5.0.8 ? Are you
sure that the parity is corect for afos?
Nov 04 16:09:17 leslie dds[20383]: TERMIOS "/dev/ttyC11": 9600 baud, none
parity
Nov 04 16:09:17 leslie afos[20382]: TERMIOS "/dev/ttyC13": 9600 baud, none
parity
Nov 04 16:09:17 leslie pps[20384]: TERMIOS "/dev/ttyC15": 9600 baud, even
parity
Also, if you comment out the dds and pps does it make any difference?
Robb...
On Thu, 4 Nov 1999, Gregory Grosshans wrote:
> I increased the LDM queue to 250 MB with LDM 5.0.8 and LDM continues to not
> see
> any AFOS products, but it does see DDS and PPS. When I switched back to
> 5.0.2 AFOS
> products were recognized right away.
>
> I've attached the ldmd.log.1 file for you to look at. You will notice LDM is
> logging
> products from PPS and DDS but only continues to expand the input buffer
> size for the AFOS feed.
>
> I noticed the 'Expanding input buffer size' is generated from pqing, in
> particular
> fxbuf.c. However, I haven't look enough at fxbuf.c or the rest of the pqing
> code
> to understand whats happening.
>
> Our AFOS feed is actually from our local AFOS. We also have AWIPS, and I'm
> ingesting
> the NOAAPORT data into another LDM server. I'm trying to upgrade my LDM
> servers
> to 5.0.8 to support the /p option and wanted to do the two machines that
> ingest actual data
> feeds before I upgrade those machines that actually do something with the
> data.
>
> Any other suggestions?
>
> Thanks,
> Gregg
>
> Gilbert Sebenste wrote:
>
> > Hey Robb and Greg,
> >
> > > > I should also specify that LDM isn't seeing any AFOS products, the
> > > > buffer is
> > > > increasing. I've seen the buffer
> > > > increase before on other data feeds while LDM was still ingesting data.
> > > > That is, you
> > > > can see the data coming
> > > > in via 'ldmadmin watch'.
> > > > >
> > > > > Gregory,
> > > > >
> > > > > This is strange, the only thing I can think of is that the 5.0.8
> > > > > version
> > > > > is receiving more AFOS data. The irix system sometimes get into
> > > > > confused
> > > > > states when the queue is expanded. I would make the queue 150
> > > > > megabytes
> > > > > and try again. If the message still appear, make the queue large
> > > > > until the
> > > > > messages disappear. The comments in RCS for the afos_message.c state
> > > > > that
> > > > > the routines where made more lenient to allow more products to be
> > > > > accepted.
> > > > >
> > > > > Robb...
> >
> > I assume you are getting the full bandwidth of AFOS via AWIPS.
> >
> > Indeed, I had placed a request in to allow the LDM to be able to handle
> > the products better...and it inadvertantly allowed more products to come
> > through. I, too, had to increase my queue...right now, I am just getting
> > the NOAA Wather Wire Service, as paranoid mode has struck the Chicago NWS
> > and they won't let anyone through their firewall anymore. If you can set
> > me up with a test feed, I can check on it here (my machine is
> > weather3.admin.niu.edu).
> >
> > What I suspect is happening is what Robb mentioned above. Presently, I
> > have my machine configured so that my queue is 250 MB. Here's a sample out
> > of my pqact.conf of how I am saving the data (see below for more
> > comments):
> >
> > #
> > #
> > # A F O S S E C T I O N
> > #
> > #
> > #-----------------------------------------------------------------------------
> > # State Zone Forecasts by Specific State (Illinois).
> > AFOS ^CHIZFPIL(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/state_zone_forecasts/IL.(\2:mmm)\2.tmp
> > AFOS ^CHIZFPIL(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC prepend_file domestic/state_zone_forecasts/IL.(\2:mmm)\2
> > #-----------------------------------------------------------------------------
> > # State Forecasts by Specific State (Illinois).
> > AFOS ^CHISFPIL(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/databases/FPUS01.(\3:yy)(\3:mmm)\3
> > AFOS ^CHISFPIL(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/databases/FPUS01.(\3:yy)(\3:mmm)\3 \2
> > (\3:yy)(\3:mmm)\3\4
> > #-----------------------------------------------------------------------------
> > # NOWCAST for Chicago, Illinois.
> > AFOS ^CHINOWCHI ([0-3][0-9])([0-2][0-9])
> > FILE domestic/databases/FXUS21.(\3:yy)(\3:mmm)\3
> > AFOS ^CHINOWCHI ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/databases/FXUS21.(\3:yy)(\3:mmm)\3 \2
> > (\3:yy)(\3:mmm)\3\4
> > #-----------------------------------------------------------------------------
> > # State Weather Roundup for Colorado.
> > AFOS ^DENSWRCO(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/state_roundups/CO.(\2:mmm)\2\3
> > #-----------------------------------------------------------------------------
> > # Severe storm outlook narratives (day 1, day 2, and mesoscale).
> > AFOS ^MKCSWODY1(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/convective/ACUS01.(\2:mmm)\2
> > AFOS ^MKCSWODY1(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> > (\2:yy)(\2:mmm)\2\3
> > AFOS ^MKCSWODY2(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/convective/ACUS02.(\2:mmm)\2
> > AFOS ^MKCSWODY2(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> > (\2:yy)(\2:mmm)\2\3
> > AFOS ^MKCMCD(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/convective/ACUS03.(\2:mmm)\2
> > AFOS ^MKCMCD(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> > (\2:yy)(\2:mmm)\2\3
> > #-----------------------------------------------------------------------------
> > # Tornado warnings.
> > AFOS ^(.*)TOR(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/databases/WF.(\3:yy)(\3:mmm)\3
> > AFOS ^(.*)TOR(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/databases/WF.(\3:yy)(\3:mmm)\3 \2
> > (\3:yy)(\3:mmm)\3\4
> > #Severe thunderstorm and tornado watches.
> > AFOS ^MKCSEL(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/watches/severe_watch.(\3:mmm)\3
> > AFOS ^MKCSEL(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/watches/severe_watch.(\3:mmm)\3 \2
> > (\3:yy)(\3:mmm)\3\4
> > #Flash flood statements.
> > AFOS ^(.*)FFS(.*) ([0-3][0-9])([0-2][0-9])
> > FILE domestic/watches/flood_stmt.(\3:mmm)\3
> > AFOS ^(.*)FFS(.*) ([0-3][0-9])([0-2][0-9])
> > EXEC make_index domestic/watches/flood_stmt.(\3:mmm)\3 \2
> > (\3:yy)(\3:mmm)\3\4
> >
> > -----------------------------------------------------------------------------
> > As you can see, the additional ([0-3][0-9])([0-2][0-9]) comes from the
> > fact that the second line of the AFOS PIL is put on the first...which lets
> > me be much more flexible in what I can do in naming the file. If, however,
> > you don't have that line in there...put a (.*) and that should take care
> > of it.
> >
> > The problem is that there are also inconsistencies, even among SPC
> > products. The ACUS1 and 2 products have the second line, while the meso
> > discussions do not. As a result, they get saved in a different filename,
> > despite the similar entry of those of the ACUS1 and 2.
> >
> > Try 250 MB as the queue...and then go from there. Like I said, I can try
> > it to see if it works on mine...I am using Linux Redhat 6.0 on a PII 450
> > MHZ machine on a partial T3, so I can handle it here.
> >
> > *******************************************************************************
> > Gilbert Sebenste
> > ********
> > Internet: address@hidden (My opinions only!) ******
> > Staff Meteorologist, Northern Illinois University ****
> > Work phone: 815-753-5492 ***
> > *******************************************************************************
> >
>
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================