[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed
- Subject: Re: 19991103: LDM 5.0.8 and 'expanding input buffer size' problemwithAFOS feed
- Date: Thu, 4 Nov 1999 09:08:47 -0700 (MST)
Thanks for the input Gilbert
Robb...
On Thu, 4 Nov 1999, Gilbert Sebenste wrote:
> Hey Robb and Greg,
>
> > > I should also specify that LDM isn't seeing any AFOS products, the buffer
> > > is
> > > increasing. I've seen the buffer
> > > increase before on other data feeds while LDM was still ingesting data.
> > > That is, you
> > > can see the data coming
> > > in via 'ldmadmin watch'.
> > > >
> > > > Gregory,
> > > >
> > > > This is strange, the only thing I can think of is that the 5.0.8 version
> > > > is receiving more AFOS data. The irix system sometimes get into
> > > > confused
> > > > states when the queue is expanded. I would make the queue 150 megabytes
> > > > and try again. If the message still appear, make the queue large until
> > > > the
> > > > messages disappear. The comments in RCS for the afos_message.c state
> > > > that
> > > > the routines where made more lenient to allow more products to be
> > > > accepted.
> > > >
> > > > Robb...
>
> I assume you are getting the full bandwidth of AFOS via AWIPS.
>
> Indeed, I had placed a request in to allow the LDM to be able to handle
> the products better...and it inadvertantly allowed more products to come
> through. I, too, had to increase my queue...right now, I am just getting
> the NOAA Wather Wire Service, as paranoid mode has struck the Chicago NWS
> and they won't let anyone through their firewall anymore. If you can set
> me up with a test feed, I can check on it here (my machine is
> weather3.admin.niu.edu).
>
> What I suspect is happening is what Robb mentioned above. Presently, I
> have my machine configured so that my queue is 250 MB. Here's a sample out
> of my pqact.conf of how I am saving the data (see below for more
> comments):
>
> #
> #
> # A F O S S E C T I O N
> #
> #
> #-----------------------------------------------------------------------------
> # State Zone Forecasts by Specific State (Illinois).
> AFOS ^CHIZFPIL(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/state_zone_forecasts/IL.(\2:mmm)\2.tmp
> AFOS ^CHIZFPIL(.*) ([0-3][0-9])([0-2][0-9])
> EXEC prepend_file domestic/state_zone_forecasts/IL.(\2:mmm)\2
> #-----------------------------------------------------------------------------
> # State Forecasts by Specific State (Illinois).
> AFOS ^CHISFPIL(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/databases/FPUS01.(\3:yy)(\3:mmm)\3
> AFOS ^CHISFPIL(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/databases/FPUS01.(\3:yy)(\3:mmm)\3 \2
> (\3:yy)(\3:mmm)\3\4
> #-----------------------------------------------------------------------------
> # NOWCAST for Chicago, Illinois.
> AFOS ^CHINOWCHI ([0-3][0-9])([0-2][0-9])
> FILE domestic/databases/FXUS21.(\3:yy)(\3:mmm)\3
> AFOS ^CHINOWCHI ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/databases/FXUS21.(\3:yy)(\3:mmm)\3 \2
> (\3:yy)(\3:mmm)\3\4
> #-----------------------------------------------------------------------------
> # State Weather Roundup for Colorado.
> AFOS ^DENSWRCO(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/state_roundups/CO.(\2:mmm)\2\3
> #-----------------------------------------------------------------------------
> # Severe storm outlook narratives (day 1, day 2, and mesoscale).
> AFOS ^MKCSWODY1(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/convective/ACUS01.(\2:mmm)\2
> AFOS ^MKCSWODY1(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> (\2:yy)(\2:mmm)\2\3
> AFOS ^MKCSWODY2(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/convective/ACUS02.(\2:mmm)\2
> AFOS ^MKCSWODY2(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> (\2:yy)(\2:mmm)\2\3
> AFOS ^MKCMCD(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/convective/ACUS03.(\2:mmm)\2
> AFOS ^MKCMCD(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/convective/\1.(\2:mmm)\2 KMKC
> (\2:yy)(\2:mmm)\2\3
> #-----------------------------------------------------------------------------
> # Tornado warnings.
> AFOS ^(.*)TOR(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/databases/WF.(\3:yy)(\3:mmm)\3
> AFOS ^(.*)TOR(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/databases/WF.(\3:yy)(\3:mmm)\3 \2
> (\3:yy)(\3:mmm)\3\4
> #Severe thunderstorm and tornado watches.
> AFOS ^MKCSEL(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/watches/severe_watch.(\3:mmm)\3
> AFOS ^MKCSEL(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/watches/severe_watch.(\3:mmm)\3 \2
> (\3:yy)(\3:mmm)\3\4
> #Flash flood statements.
> AFOS ^(.*)FFS(.*) ([0-3][0-9])([0-2][0-9])
> FILE domestic/watches/flood_stmt.(\3:mmm)\3
> AFOS ^(.*)FFS(.*) ([0-3][0-9])([0-2][0-9])
> EXEC make_index domestic/watches/flood_stmt.(\3:mmm)\3 \2
> (\3:yy)(\3:mmm)\3\4
>
> -----------------------------------------------------------------------------
> As you can see, the additional ([0-3][0-9])([0-2][0-9]) comes from the
> fact that the second line of the AFOS PIL is put on the first...which lets
> me be much more flexible in what I can do in naming the file. If, however,
> you don't have that line in there...put a (.*) and that should take care
> of it.
>
> The problem is that there are also inconsistencies, even among SPC
> products. The ACUS1 and 2 products have the second line, while the meso
> discussions do not. As a result, they get saved in a different filename,
> despite the similar entry of those of the ACUS1 and 2.
>
> Try 250 MB as the queue...and then go from there. Like I said, I can try
> it to see if it works on mine...I am using Linux Redhat 6.0 on a PII 450
> MHZ machine on a partial T3, so I can handle it here.
>
> *******************************************************************************
> Gilbert Sebenste ********
> Internet: address@hidden (My opinions only!) ******
> Staff Meteorologist, Northern Illinois University ****
> Work phone: 815-753-5492 ***
> *******************************************************************************
>
>
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================