[GEMPAK #RKS-826804]: GEMPAK - LDM Problem? KRAP and other Rapid City NWS stations "missing"

Bill,

I have each hour's METAR from CUT and RAP decoded by GEMPAK here:

 SFFILE   = metar
 AREA     = @cut;rap
 DATTIM   = all
 SFPARM   = text
 OUTPUT   = t
 IDNTYP   = stid
 GEMPAK-SFLIST>r

KCUT 012353Z AUTO 28009G14KT 10SM CLR 30/07 A3008 RMK AO2 SLP112 T03000067 
10328 20278 58004 TSNO $ 
KRAP 012352Z 34021G26KT 10SM CLR 35/13 A2989 RMK AO2 PK WND 34027/2330 LTG DSNT 
E AND SE SLP085 T03500128 10389 20328 55003 
  
KCUT 020053Z AUTO 31007KT 10SM CLR 30/02 A3010 RMK AO2 SLP112 T03000022 TSNO $  
KRAP 020052Z 35018G23KT 10SM CLR 34/13 A2991 RMK AO2 PK WND 34028/0013 SLP093 
T03390128 
  
KCUT 020153Z AUTO 35003KT 10SM CLR 28/03 A3011 RMK AO2 SLP111 T02830028 TSNO $  
KRAP 020152Z 36008KT 10SM CLR 32/13 A2993 RMK AO2 SLP098 T03220133  
  
KCUT 020253Z AUTO 03003KT 10SM CLR 25/03 A3013 RMK AO2 SLP114 T02500033 51014 
TSNO $
KRAP 020252Z 36004KT 10SM CLR 27/14 A2995 RMK AO2 SLP108 T02720139 51021
  
KCUT 020353Z AUTO 08007KT 10SM CLR 22/08 A3015 RMK AO2 SLP125 T02220078 TSNO $  
KRAP 020352Z 33007KT 10SM CLR 27/14 A2997 RMK AO2 SLP115 T02670139  
  
KCUT 020453Z AUTO 03003KT 10SM CLR 21/08 A3017 RMK AO2 SLP133 T02060078 TSNO $  
KRAP 020452Z AUTO 34008KT 10SM CLR 24/16 A3000 RMK AO2 SLP125 T02390161 
  
KCUT 020553Z AUTO 08007KT 10SM CLR 21/11 A3018 RMK AO2 SLP138 T02060106 10311 
20194 51015 TSNO $
KRAP 020552Z AUTO 31006KT 10SM CLR 23/17 A3000 RMK AO2 SLP124 T02330172 10356 
20228 51017   
  
KCUT 020653Z AUTO 04004KT 10SM CLR 18/13 A3017 RMK AO2 SLP139 T01830128 
403280117 TSNO $
KRAP 020652Z AUTO 33010KT 10SM CLR 23/17 A3000 RMK AO2 SLP122 T02330172 
403890183   
  
KCUT 020753Z AUTO 32003KT 10SM CLR 17/13 A3017 RMK AO2 SLP136 T01720128 TSNO $  
KRAP 020752Z AUTO 34012KT 10SM CLR 23/17 A3001 RMK AO2 SLP124 T02330172 
  
KCUT 020853Z AUTO 01004KT 10SM CLR 16/13 A3017 RMK AO2 SLP137 T01610133 55003 
TSNO $
KRAP 020852Z AUTO 34013KT 10SM CLR 23/18 A3003 RMK AO2 SLP131 T02280178 53009   
  
KCUT 020953Z AUTO 02006KT 10SM CLR 18/14 A3017 RMK AO2 SLP132 T01780139 TSNO $  
KRAP 020952Z AUTO 34015KT 10SM CLR 23/18 A3004 RMK AO2 SLP134 T02280178 
  
KCUT 021053Z AUTO 06004KT 10SM CLR 18/12 A3017 RMK AO2 SLP133 T01830117 TSNO $  
KRAP 021052Z AUTO 30005KT 10SM CLR 21/18 A3001 RMK AO2 SLP128 T02110178 
  
KCUT 021153Z AUTO 03004KT 10SM CLR 19/11 A3017 RMK AO2 SLP136 T01890106 10206 
20161 52002 TSNO $
KRAP 021152Z AUTO 17004KT 10SM CLR 20/17 A3002 RMK AO2 SLP132 T02000172 10239 
20200 55003   
  
KCUT 021253Z AUTO 04006KT 10SM CLR 22/06 A3018 RMK AO2 SLP133 T02170061 TSNO $  
KRAP 021252Z 32008KT 10SM FEW120 24/16 A3005 RMK AO2 SLP140 T02390161

  
KCUT 021353Z AUTO 03011G20KT 10SM CLR 23/06 A3019 RMK AO2 SLP137 T02280056 TSNO 
$   
KRAP 021352Z 35011KT 10SM CLR 27/12 A3004 RMK AO2 LTG DSNT NE SLP133 T02720122  
  
KCUT 021453Z AUTO 04014G19KT 10SM CLR 26/03 A3019 RMK AO2 SLP139 T02560033 
51005 TSNO $ 
KRAP 021452Z 35013G20KT 10SM CLR 29/09 A3005 RMK AO2 SLP140 T02940094 51008 
  
KCUT 021553Z AUTO A3020 RMK AO2 SLPNO PWINO FZRANO TSNO PNO $   
KRAP 021552Z 06010KT 10SM CLR 32/08 A3004 RMK AO2 SLP134 T03220083  
  
KCUT 021653Z AUTO 10014G20KT 10SM CLR 27/03 A3020 RMK AO2 SLP147 T02670033 TSNO 
KRAP 021652Z 09011G16KT 10SM CLR 33/08 A3005 RMK AO2 SLP138 T03280083   
  
KCUT 021753Z AUTO 06017G25KT 10SM CLR 27/05 A3019 RMK AO2 SLP144 6//// 
T02720050 50001 TSNO 
KRAP 021752Z 13010G19KT 10SM CLR 33/07 A3003 RMK AO2 SLP134 T03330072 10333 
20200 58005 
  
KCUT 021853Z AUTO 10013G21KT 10SM CLR 27/06 A3019 RMK AO2 SLP145 T02720061 TSNO 
KRAP 021852Z 14012G18KT 10SM CLR 34/09 A3002 RMK AO2 SLP129 T03390089   
  
KCUT 021953Z AUTO 10011G18KT 10SM CLR 28/06 A3018 RMK AO2 SLP142 T02830056 TSNO 
KRAP 021952Z 08007G16KT 050V120 10SM CLR 34/08 A3001 RMK AO2 SLP127 T03390083   
  
KCUT 022053Z AUTO 15008G15KT 120V180 10SM CLR 29/06 A3017 RMK AO2 SLP140 
T02890056 57006 TSNO   
KRAP 022052Z 15015G21KT 10SM CLR 36/08 A3001 RMK AO2 SLP124 T03560078 56008 


I looked at your LDM latencies for IDS|DDPLUS and it looks like you are 
receiving the feed 
in a timely fashion, but possibly you aren't getting the data out of your queue 
by pqact
fast enough before it gets scoured out of your queue.

Your LDM version is 6.4.6. The current LDM 6.6.5 version will log a message
if it is processing the oldest product in the queue....eg, when pqact has
fallen behind to the point that incoming data would be scouring out data that 
still
needs to be processed. You will also want to determine if your
queue is sized properly for the amount of data you are receiving. Your volume 
plot:

http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?unidata.sdstate.edu+GRAPH

shows that you are averaging about 1.6GB of data received per hour,and peaking 
around 2.4GB per hour.
The LDM command "pqmon" will show you the age of the oldest product in your 
queue in the righmost column.
If your decoding and writing to dist is slower than that, you will have to tune 
up your pqact processing.

Are you using the pqact actions that I supply with GEMPAK? Are you using the 
current 
distribution of GEMPAK? are you using a single pqact.conf file, or are you
splitting up the pqact processing into multiple pqact processes such as 
gen_pqact.csh will 
provide as an option?

The test machine I have here for real-time data is not very fast, but is 
decoding the data, 
out of a 2GB queue.

Steve Chiswell
Unidata User Support





> Institution: Institute of Atmospheric Sciences
> Package Version: 5-10-2
> Operating System: Linux and Irix
> Hardware Information: SGIs and Linux Fleet
> Inquiry: Hi all.  This may also be an LDM issue but I can't see the problem 
> in that software either.  Is there a problem in the METAR feeds from Rapid 
> City's NWS (and a few others in the midwest). We are only getting a few 
> observations from many of these stations, namely KRAP, KCUT and other 
> stations whose metars are showing up on other sources like RAP's web sites?  
> Are you guys getting the same data big-empty in this region?  Everything else 
> seems to be working...
> 
> Cheers and Thanks
> Bill Capehart
> IAS-SDSMT
> 
> 
> 
> 


Ticket Details
===================
Ticket ID: RKS-826804
Department: Support GEMPAK
Priority: Normal
Status: Closed