[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #YUZ-392149]: Problems with duplicate grib grids

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [LDM #YUZ-392149]: Problems with duplicate grib grids
  • Date: Tue, 31 Aug 2010 13:19:06 -0600
Jerrold,

> We are having a problem with duplicate grib2 entries being filed by the ldm 
> from our CONDUIT feed.   We are collecting GFS 1/2 degree grib2 files.     
> When I break out the duplicate individual grib messages and do a md5sum on 
> them, they are identical.

Please forgive my lack of knowledge about the data, but do the relevant LDM 
data-products contain multiple GRIB messages?  If so, then are those LDM 
data-products duplicated or just some of their GRIB messages?

> Looking back at support emails, I found one from Tom Yoksas telling Kevin 
> Baggot that the product queue may be too small.  see 
> http://www.unidata.ucar.edu/support/help/MailArchives/ldm/msg05035.html
> 
> When I did a pqmon to find the oldest product  I see a time of 16,626 
> seconds.   This seemed like I should not have been a problem, but to be safe, 
> yesterday I increased our queue size from 400 MB to 1 GB on the server where 
> I am seeing the problem.
> 
> Today, we are seeing even more duplicates than before,   so the increased 
> queue size seem to make it worse?
> 
> Here are my ldmd.conf request entries:  (we only wanted certain variables, 
> not all of the GFS that's why there are so many requests.)
> 
> # U and V component of the wind
> request CONDUIT "gfs.*pgrb2.*[UV]REL/"  pepe.ssec.wisc.edu PRIMARY
> 
> #Temperature
> request CONDUIT "gfs.*pgrb2.*TMPK/"  pepe.ssec.wisc.edu PRIMARY
> 
> # Relative Humidity
> request CONDUIT "gfs.*pgrb2.*RELH/"  pepe.ssec.wisc.edu PRIMARY
> 
> #Precipitable Water
> request CONDUIT "gfs.*pgrb2.*PWTR/"  pepe.ssec.wisc.edu PRIMARY
> 
> #Pressure at the Surface
> request CONDUIT "gfs.*pgrb2.*PRES/0 - NONE"  pepe.ssec.wisc.edu PRIMARY
> 
> #Cloud Water
> request CONDUIT "gfs.*pgrb2.*CWTR/"  pepe.ssec.wisc.edu PRIMARY
> 
> #MSL Pressure
> request CONDUIT "gfs.*pgrb2.*PMSL/"  pepe.ssec.wisc.edu PRIMARY
> 
> #Total and Convective Precip
> request CONDUIT "gfs.*pgrb2.*[PC]..M/"  pepe.ssec.wisc.edu PRIMARY
> 
> #HPBL
> request CONDUIT "gfs.*pgrb2.*HPBL/"  pepe.ssec.wisc.edu PRIMARY
> 
> #LAND
> request CONDUIT "gfs.*pgrb2.*LAND/"  pepe.ssec.wisc.edu PRIMARY
> 
> 
> We have several different pqact.conf entries,   but here is a typical example:
> 
> CONDUIT       
> ^data/nccf/com/.*(gfs).*/prod/.*/.*gfs.*.t.*pgrb2.*grib2/ncep/GFS/#000/20([0-9][0-9])([0-9][0-9])([0-9][0-9])(
> .[02])..F(0[01][23689])/.*
> FILE  
> /data4/ancillary/20\2_\3_\4_(\4:ddd)/forecast/\1.p5.20\2\3\4_\5_\6.ldm.grib2
> 
> 
> Thanks for any help you can provide.
> 
> Regards,
> 
> Jerry
> 
> 
> --
> Jerrold Robaidek                              Email:  address@hidden
> SSEC Data Center                              Phone: (608) 262-6025
> University of Wisconsin                       Fax: (608) 263-6738
> Madison, Wisconsin

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: YUZ-392149
Department: Support LDM
Priority: Normal
Status: Closed