[IDD #RPV-768746]: CONDUIT configuragion in pqact.conf

Hi Byoung-Joo,

I apologize for not being able to respond to your first inquiry on this subject!

> I am Ph.D student from Korea.
> I have some questions about pqact.conf configuration
> 
> 
> My CONDUIT configuration is as following:
> 
> CONDUIT ^data/nccf/com/gfs/prod/gfs\.(.*) !(.*)!
> FILE    data/conduit/gfs/\1
> CONDUIT ^data/nccf/com/gfs/prod/gdas\.(.*) !(.*)!
> FILE    data/conduit/gdas/\1

OK.

> Using above configuration, I can currently get following data
> 
> a. NCEP GFS forecast 1x1 and 0.5x0.5 (in Grib2 format)
> b. NCEP Final analysis (in Grib1 format)
> 
> My questions are . . .
> 
> 1. Currently 3 hour interval data is coming. But, I want 6 hour interval.
> How to do that ?

The best thing would be to limit the data you are requesting from your
upstream feed host.  You do this by crafting an extended regular expression
to use in the REQUEST line in the ~ldm/etc/ldmd.conf file.  Use the '-p'
flag in a 'notifyme' invocation to test regular expressions to get the one
that limits your data request to the subset that you want:

<as 'ldm'>
notifyme -vxl- -f CONDUIT -o 3600 -h <upstream_feed_host> -p 'ERE test pattern'

Once you have an ERE pattern that gives just the set of data you want, use
it in the ldmd.conf REQUEST line:

REQUEST CONDUIT "ERE pattern you found that works" upstream_feed_host

You can get more information on how to use 'notifyme from its man page which
is included in the LDM distribution,   

> 2. Sometimes the some file does not come ! (only portion of the filesize)
> WHY ??
> 
> Is this due to the network traffic ??

Most likely, yes.  It is also possible that you are receiving so much data that
the product coming in is not processed out of the LDM queue before it is 
overwritten
by new products.

Questions:

- is CONDUIT the only data you are requesting

- how large is your LDM queue.  A system receiving all CONDUIT data should
  have an LDM queue that is at least 1 GB.  If you are also receiving
  other data and your machine is heavily loaded, you would do well to
  increase the queue size to 2 GB or larger.

> Example of question 2. ~ The gfs.t00z.pgrbf27.grib2 file size is not
> complete !!!
> 
> ----------------------------------------------------------------------------
> -rw-rw-r--   1 ldm ldm 13593076 Apr  5 11:08 gfs.t00z.pgrbanl.grib2
> -rw-rw-r--   1 ldm ldm 14231842 Apr  5 11:08 gfs.t00z.pgrbf00.grib2
> -rw-rw-r--   1 ldm ldm 15987180 Apr  5 11:10 gfs.t00z.pgrbf03.grib2
> -rw-rw-r--   1 ldm ldm 15852094 Apr  5 11:12 gfs.t00z.pgrbf06.grib2
> -rw-rw-r--   1 ldm ldm 15865397 Apr  5 11:14 gfs.t00z.pgrbf09.grib2
> -rw-rw-r--   1 ldm ldm 16099857 Apr  5 11:15 gfs.t00z.pgrbf12.grib2
> -rw-rw-r--   1 ldm ldm 16294154 Apr  5 11:17 gfs.t00z.pgrbf15.grib2
> -rw-rw-r--   1 ldm ldm 16264285 Apr  5 11:23 gfs.t00z.pgrbf18.grib2
> -rw-rw-r--   1 ldm ldm 16216251 Apr  5 11:24 gfs.t00z.pgrbf21.grib2
> -rw-rw-r--   1 ldm ldm 16266201 Apr  5 11:26 gfs.t00z.pgrbf24.grib2
> -rw-rw-r--   1 ldm ldm  9753913 Apr  5 11:30 gfs.t00z.pgrbf27.grib2
> -rw-rw-r--   1 ldm ldm 16369195 Apr  5 11:32 gfs.t00z.pgrbf30.grib2
> -rw-rw-r--   1 ldm ldm 16350248 Apr  5 11:35 gfs.t00z.pgrbf33.grib2
> -rw-rw-r--   1 ldm ldm 16474868 Apr  5 11:37 gfs.t00z.pgrbf36.grib2
> -rw-rw-r--   1 ldm ldm 16506491 Apr  5 11:41 gfs.t00z.pgrbf39.grib2
> -rw-rw-r--   1 ldm ldm 16520590 Apr  5 11:44 gfs.t00z.pgrbf42.grib2
> -rw-rw-r--   1 ldm ldm 16495506 Apr  5 11:48 gfs.t00z.pgrbf45.grib2
> -rw-rw-r--   1 ldm ldm 16403241 Apr  5 11:51 gfs.t00z.pgrbf48.grib2
> ----------------------------------------------------------------------------

I understand.  The missing piece(es) is(are) likely due to network
congestion or overloading of your machine to the point of not being able
to process the data being received out of the queue fast enough.

> (actually, I sent same message to "support-ldm at unidata.ucar.edu", but no
> reply)

Again, I apologize for the tardiness of this reply.

> (so, I send to you again)
> Thank you in advance for your help ~

No worries.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: RPV-768746
Department: Support IDD
Priority: Normal
Status: Closed