20030704: ldmd.conf request tuning at SFSU

>From: Dave Dempsey <address@hidden>
>Organization: SFSU
>Keywords: 200307042124.h64LOMLd021565 IDD tuning

Hi Dave,

re: ldmd.conf requests
>What about entries such as this one:
>Would it be better to drop the restriction to specific products in the
>second request above and consolidate the two requests, or is there a way to
>request a restricted product set on each of multiple feeds using a single
>request line?

>#
># Profiler data.
>#
>request         FSL2    ".*"            sunny.atmos.washington.edu
>#
># Textual data and forecasts.
>#
>request         WMO
>         "(^F[^O]*|^[ACRW]*|^S[ETXR]*|^TB*)"     sunny.atmos.washington.edu

I imagine that you are including a pattern in the second request line
to limit your ingest of HDS data, and this may have originally been
done not because you didn't want the data, but, rather, because you
were having problem getting the data.

Even though your latencies look reasonably good:

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?IDS|DDPLUS+enso.sfsu.edu

I would recommend that you experimentally reorganize your data requests.
Below, I list several different approaches to the reorganization.  The
one I would try first is the one I list last.

1) First reorganization

#
# IDS|DDPLUS - Global observational data from NOAAPORT
# FSL2       - Profiler data
#
request IDS|DDPLUS|FSL2 ".*"    sunny.atmos.washington.edu

#
# HDS - Model output from NOAAPORT
#
request HDS     "(^F[^O]*|^[ACRW]*|^S[ETXR]*|^TB*)"     
sunny.atmos.washington.edu

2) Second reorg

If you were limiting your HDS data request because you were having
problems getting that data using LDM-5, I would experiment to see if
you can't get all of the data using LDM-6:

request HDS     ".*"    sunny.atmos.washington.edu

3) Third reorg

What about the UNIWISC data?  Since you are also getting UNIWISC from U
Washington, I would recommend a slightly different set of requests:

#
# IDS|DDPLUS - Global observational data from NOAAPORT
#
request IDS|DDPLUS      ".*"    sunny.atmos.washington.edu

#
# FSL2       - Profiler data
# UNIWISC    - Unidata-Wisconsin GOES satellite images and products
#
request FSL2|UNIWISC    ".*"    sunny.atmos.washington.edu

#
# HDS - Model output from NOAAPORT
#
request HDS     ".*"    sunny.atmos.washington.edu


4) Fourth reorg

Given your current latencies, you may even be able to consolidate the
IDS|DDPLUS, FSL2, and UNIWISC requests:

#
# IDS|DDPLUS - Global observational data from NOAAPORT
# FSL2       - Profiler data
# UNIWISC    - Unidata-Wisconsin GOES satellite images and products
#
request IDS|DDPLUS|FSL2|UNIWISC ".*"    sunny.atmos.washington.edu

#
# HDS - Model output from NOAAPORT
#
request HDS     ".*"    sunny.atmos.washington.edu

I would give this last configuration a try first.  Keep an eye on your
latencies after the reorganization to see if you need to refine your
requests.  If yes, drop back to approach 3).

Cheers,

Tom