[LDM #MIY-170840]: One of our two LDM servers is having problems

Brian,

All I could make of this is that the "-flush" option was rejected. Prthe obably
best to just send us plain text.

Make sure that whitespace between the "FILE" and the "-flush" is a tab.

> The "-flush" option on the "FILE" action is rejected:Checking 
> pqact(1) configuration-file(s)... /home/ldm/etc/pqact.conf: has problems: 
> 20170517T173812.527398Z pqact[7798] NOTE pqact.c:420:main() Starting Up 
> 20170517T173812.527935Z pqact[7798] ERROR palt.c:373:new_palt_fromStr() 
> Unknown action "FILE -flush data/dds/sa/(\1:yyyy)(\1:mm)\1\2.sa" at 
> line 141 20170517T173812.527957Z pqact[7798] ERROR palt.c:460:readPatFile() 
> Error in configuration-file "/home/ldm/etc/pqact.conf" 
> 20170517T173812.527980Z pqact[7798] NOTE pqact.c:111:cleanup() ExitingOn May 
> 17, 2017 at 9:49 AM Unidata LDM Support <address@hidden> wrote:Hi 
> Tom,re: LDM configuration files requested> Here are the files you 
> requested.Thanks for sending the ldmd.conf, pqact.conf and registry.xml 
> filesfrom colaweb.gmu.edu. I'm reviewing them now to see if anythingjumps 
> out at me.re:> I saw there is a netcheck command in etc, so I ran it with 
> out data> 
 providers. All I can tell is that we can reach both hosts. And the> ping 
times seem reasonable.Thanks. Ping times to upstream LDM feed hosts being low 
do notnecessarily indicate that data will flow with low latencies (thecondition 
is necessary, but not sufficient).Brian wrote:> I have a question regarding 
the way the LDM interacts with the OS;> For the DDS data feed, the report 
headers are parsed and then the> reports are put into various files based 
on the header and the> date/time stamp.> > These files -- are they 
kept open ...or are they opened, written> to, and then closed?The answer 
would depend on whether or not you include the '-close'option on the 
LDM FILE action(s) that are filing the products.A quick review of the LDM 
pattern-action file, ~ldm/etc/pqact.conf,that Tom sent along yesterday evening 
shows that the '-close' optionis not included on any of the 
FILE/STDIOFILE actions. In this case,the files will be kept open
  by the OS and only closed when 'pqact'runs out of file descriptors 
and needs to close old ones. Whenthe OS will flush pending data writes to disk, 
is totally dependenton the OS, not the LDM.Here is the current METAR processing 
action from the pattern-actionfile that Tom sent:# All metars in files by day 
and hour.IDS|DDPLUS ^SA.... .... (..)(..).* FILE 
data/dds/sa/(\1:yyyy)(\1:mm)\1\2.saOne could force the products to be flushed 
to the file by includingthe '-flush' option on the FILE action. For 
example:# All metars in files by day and hour.IDS|DDPLUS ^SA.... .... 
(..)(..).* FILE -flush data/dds/sa/(\1:yyyy)(\1:mm)\1\2.saIn a similar way, one 
can force the file to be closed aftereach write (and this would also cause the 
product to bewritten to the file):# All metars in files by day and 
hour.IDS|DDPLUS ^SA.... .... (..)(..).* FILE -close 
data/dds/sa/(\1:yyyy)(\1:mm)\1\2.saNB: the white space between FILE and its 
option (e.g., '-close')must be a tab, no
 t spaces.If you decide to alter your pattern-action file actions, youshould 
always do a gross check to make sure that there are noobvious syntax errors by 
running:ldmadmin pqactcheckYou can cause the newly modified action(s) to become 
active(after running the check!) by running:ldmadmin pqactHUPYou can see all of 
the things that 'ldmadmin' supports by simply running:ldmadminComments 
on files sent by Tom yesterday evening:- I see nothing wrong with the REQUEST 
settings in ldmd.conf:REQUEST IDS|DDPLUS ".*" idd.meteo.psu.eduREQUEST 
IDS|DDPLUS ".*" idd.unidata.ucar.eduREQUEST FNEXRAD 
"^radar_mosaic" idd.meteo.psu.eduREQUEST FNEXRAD 
"^radar_mosaic" idd.unidata.ucar.eduREQUEST NIMAGE " TIG[NE]" 
idd.meteo.psu.eduREQUEST NIMAGE " TIG[NE]" idd.unidata.ucar.edu I would 
expect that on any machine that has any sort of reasonable network connection 
(and by reasonable I mean really low bandwidth) would receive all of the 
products REQ
 UESTed with no problems. As a comparison, I routinely REQUEST LOTS (100x) more 
data in the LDM I have setup on my Odroid U3 single board computer that is 
running Lbuntu 14.04 LTS (it is like a Raspberry Pi) and is connected to the 
Internet by a slow (~5 Mbps) wireless connection.- I see nothing obviously 
wrong with colaweb's LDM pattern-action file, pqact.conf- I see nothing 
wrong with the default LDM queue configuration in colaweb's LDM registry: 
<queue> <path>/home/ldm/var/queues/ldm.pq</path> 
<size>500M</size> <slots>default</slots> 
</queue>Given that nothing seems untoward in colaweb's LDM 
configuration files, mysuspicion returns back to colaweb's network 
connection.Questions:- are there any quality of service (QoS) setups on the 
firewall either/both on colaweb or on the department/college/university 
firewalls?- it is possible (edge case) that there is something wrong with the 
Ethernet adapte
 r that LDM data is flowing into on colaweb I don't think that this is 
likely because you (Jennifer) indicated that colaweb's OS was upgraded from 
CentOS 5 to CentOS 7, and the LDM was upgraded from 6.7.0 to 6.13.6. Although 
not explicitly stated, the inference was that the old LDM running in the old OS 
on the same hardware was working correctly, but now things are not working. If 
the inference is true, it should be the case that the network connection to 
colaweb is OK (meaning sufficient to transfer the very low volume of data that 
is being REQUESTed), and that the Ethernet interface is OK. This leaves 
something in the OS as being the culprit, but, for the life of me, I can't 
imagine what this would/could be!We will get together to discuss your situation 
today and get back to youwith any ideas/requests for tests that we can come up 
with.Cheers,Tom--****************************************************************************Unidata
 User Support UCAR Unidata Progra
 m(303) 497-8642 P.O. Box address@hidden Boulder, CO 
80307----------------------------------------------------------------------------Unidata
 HomePage 
http://www.unidata.ucar.edu****************************************************************************Ticket
 Details===================Ticket ID: MIY-170840Department: Support 
LDMPriority: NormalStatus: Open===================NOTE: All email exchanges 
with Unidata User Support are recorded in the Unidata inquiry tracking system 
and then made publicly available through the web. If you do not want to have 
your interactions made available in this way, you must let us know in each 
email you send to us.>

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: MIY-170840
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.