[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20030503: 20030502: kickin my server
- Subject: 20030503: 20030502: kickin my server
- Date: Mon, 05 May 2003 13:07:05 -0600
Ray,
Given that you are requesting a small amount of the total datastream,
you can consolidate many of your request lines. Also, you need to
be careful not to introduce uncecessary pattern checking. For example,
you are only interested in HDS products beginning with the
seqences beginning with "Y", whereas you patterns below will look for
sequences ocurring anywhere within the WMO header. The added searching
will create overhead on your upstream. You can use the ~ldm/bin/regex program
(see LDM manpage) for validating regular expressions. For your ldmd.conf,
you can use the following 2 (note, I wrapped the second request for
this email, but you don't have to!):
request NMC3|NNEXRAD|MCIDAS ".*" ldm.envsci.rutgers.edu
request WMO
"^(W[FU]US|Y(PU.89|[TWEZ]..89|T..85) KWBE|YP..89 (KWBH|KWB.*/mAVN))"
ldm.envsci.rutgers.edu
Note the use of "^" for specifying the line begiing with....
Also, WFUS and WUUS is combined to W[FU]US,
the KWBE entries are combined to the second grouping between the "|" delimiters,
and the MRF and AVN to the third grouping.
Since the HDS data and DDPLUS data streams are subsets of the WMO alias,
you can use the single feedtype in that case.
If you experience slow connections, then the most likely problem would be
in grouping NNEXRAD with the large MCIDAS and FNEXRAD products. If this
occurs, then break out the NNEXRAD to its own request line.
Tom: Dana mentioned once before to me that he had several instances of long
TIME_WAIT connections waiting to close. You can probably tune up your
time outs for closing the socket in those cases since otherwise, your
computer is more vulnerable to denial of service attacks (not related
to the LDM!!!) where someone could load your computer up with lots
of attempted connection failures.
Steve Chiswell
Unidata User Support
>From: "Ray Weber" <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200305021649.h42Gn87U002780
>I show only a few outbound, here is the ldmd.conf:
>Problem appears to be if I reboot, the old connections
>do not time out on Tom's end.
>
>The entries for Toms server;
>
>
>request NMC3 ".*" ldm.envsci.rutgers.edu
>request NNEXRAD ".*" ldm.envsci.rutgers.edu
>request MCIDAS ".*" ldm.envsci.rutgers.edu
>request HDS "YP..89 KWBH (......)" ldm.envsci.rutgers.edu
>request HDS "YP..89 KWB. (......) /mAVN" ldm.envsci.rutgers.edu
>request HDS "YPU.89 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YT..89 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YT..85 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YW..89 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YE..89 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YZ..89 KWBE (......)" ldm.envsci.rutgers.edu
>request HDS "YP....KWB. (......) /mMRF" ldm.envsci.rutgers.edu
>request DDPLUS "WFUS.. .... (......)" ldm.envsci.rutgers.edu
>request DDPLUS "WUUS.. .... (......)" ldm.envsci.rutgers.edu
>
>
>This started with the version 6 update since it opens a new connection for
>each entry, and you don't have a reasonable timeout on old connections.
>I take just a few products from HDS, and thats why there are such selective
>entries. Don't need most of them for the summer, but you should correct
>the issue with connection timeouts. I show only 14 rpc.ldmd's running for
>his machine.
>
>Ray
>
>
>
>
>> -----Original Message-----
>> From: Unidata Support [mailto:address@hidden]
>> Sent: Friday, May 02, 2003 12:49 PM
>> To: Ray Weber; Tom Grzelak
>> Cc: address@hidden; address@hidden
>> Subject: 20030502: kickin my server
>>
>>
>>
>> Ray,
>>
>> If you would send a copy of your ldmd.conf file to address@hidden,
>> we may be able to help you with your configuration. If you have many request
>> entries, we will try to suggest a better consolidation.
>>
>> Tom/Dana: It may help to send us a grep of your ldmd.log file with
>> "grep topo ldmd.conf | grep attbi.com"
>> so that we can see what feed connections are being established.
>>
>> Steve Chiswell
>>
>>
>>
>>
>>
>>
>> >From: Tom Grzelak <address@hidden>
>> >Organization: UCAR/Unidata
>> >Keywords: 200305021427.h42ERs7U019833
>>
>> >Ray,
>> >
>> >I have 35 connections right now from your machine. I am not exactly sure
>> >how you have your LDM set up, but this is getting to be a problem. Each
>> >time your ldm makes a separate connection on my machine, it spawns another
>> >process. I can't have all of these connections on my machine. It is
>> >killin' my load rates. Every other group I feed data to only has 1-3
>> >connections at any given time. I know you make separate connections for
>> >very specific products, but each time you do so, my server must run another
>
>> >LDM process to support you. It may help your bandwidth, but at my expense.
>> >
>> >Tom
>> >
>> >dana@web:/users/dana% netstat -a | grep attbi |grep EST
>> >web.ldm weather.ne.client2.attbi.com.46772 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.48881 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.38058 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.57891 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.57901 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32820 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32834 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32836 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32838 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32839 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.816 5840 0 32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.817 5840 0 32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.822 5840 0 32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.831 5840 0 32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.839 5840 0 32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32946 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32948 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32949 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32956 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32802 62928 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32803 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32795 5840 0
>> >32832
> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32797 63940 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32798 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32799 5840 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.32801 62928 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.37735 62928 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.45275 62928 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.45395 62928 0
>> >32832
>> >0 ESTABLISHED
>> >web.ldm weather.ne.client2.attbi.com.45412 62928 0
>> >32832
>> >0 ESTABLISHED
>> >
>> >
>> >
>> >
>> >===============================================================
>> >Tom Grzelak address@hidden
>> >Senior Systems Administrator
>> > ----> Center for Environmental Prediction
>> > ----> Department of Environmental Sciences
>> >
>> >Rutgers University
>> >14 College Farm Road Voice: 732-932-4923
>> >New Brunswick, NJ 08901-8551 Fax: 732-932-8644
>> > http://www.envsci.rutgers.edu
>> >===============================================================
>> >
>>
>> ****************************************************************************
>> Unidata User Support UCAR Unidata Program
>> (303)497-8643 P.O. Box 3000
>> address@hidden Boulder, CO 80307
>> ----------------------------------------------------------------------------
>> Unidata WWW Service http://my.unidata.ucar.edu/content/support
>> ****************************************************************************
>>
>>
>
>