[Datastream #IZJ-689237]: Additional Datafeeds

Hi Jeff,

re:
> I'm still seeing it.  I just went in and did a quick look at the logs and, in 
> part:

I logged in after seeing your note:

> dcactf.log is ALL "cannot read file" or "cannot write to file"

Hmm... when I got on, I noticed that the dcactf.log file was empty:

[ldm@whistler logs]$ pwd
/var/data/ldm/data/gempak/logs
[ldm@whistler logs]$ ls -alt dcacft.log*
-rw-r--r-- 1 ldm mcdata      0 Oct 16 21:00 dcacft.log
-rw-rw-r-- 1 ldm mcdata 104901 Oct 16 20:59 dcacft.log.1

I then noticed that the new log file had just been created.  You are right the 
old one
has plenty of "[3748] 081016/1836[FL -5]  Cannot write to file ...." messages.  
So, I
did a long listing in the ~ldm/data/gempak/acft directory to see what was 
happening:

[ldm@whistler acft]$ ls -alt
total 69060
-rw-rw-r--  1 ldm mcdata    1730052 Oct 16 21:25 2008101621_acf.gem
-rw-rw-r--  1 ldm mcdata 4965086720 Oct 16 21:20 2008101620_acf.gem
-rw-rw-r--  1 ldm mcdata    1879044 Oct 16 21:10 2008101618_acf.gem
-rw-rw-r--  1 ldm mcdata    1909252 Oct 16 21:10 2008101619_acf.gem
drwxrwxr-x  2 ldm mcdata      12288 Oct 16 20:48 .
-rw-rw-r--  1 ldm mcdata    1880576 Oct 16 20:00 2008101617_acf.gem
-rw-rw-r--  1 ldm mcdata 3868396032 Oct 16 20:00 2008101616_acf.gem
-rw-rw-r--  1 ldm mcdata    1911808 Oct 16 18:00 2008101613_acf.gem
-rw-rw-r--  1 ldm mcdata    1875456 Oct 16 18:00 2008101614_acf.gem
-rw-rw-r--  1 ldm mcdata    1814020 Oct 16 18:00 2008101615_acf.gem
drwxrwxr-x 33 ldm mcdata       4096 Oct 16 15:01 ..

Notice that the 2008101620_acf.gem file is over 4 GB in size!  It is no
wonder that the decoder could not write to this file!!

My conclusion is that a file cleanup may be in order so we can see what happens
without baggage from previous setups:

- stop the LDM
- rename the ~ldm/data/gempak directory to ~ldm/data/gempak.old
- restart the LDM and let the entire directory tree be recreated
  from scratch

*** later:  I decided to stop the LDM and delete all zero-length and HUGE files
    from the various sub-directories of ~ldm/data/gempak.  Let's see if that
    gets rid of the errors about not being able to read or write files

A related topic:

By the way, I just noticed that whistler's system clock is using UTC.  This
means that all of the crontab entries will need to be redone as they are 
currently
all set assuming local time.  Alternatively, and what I recommend, is that the
system be run on local time.  To be clear: the OS really runs using UTC.  What
the users see and what cron uses, however, can be different.

For now, I edited 'ldm's cron file and changed the times to reflect UTC.

re: high product latencies
> I'll talk to the network guys here and see what they say.  I know that they 
> use a
> Packet Shaper here, and that it can have adverse effects on things -
> http://www.unidata.ucar.edu/software/ldm/ldm-6.6.5/troubleshooting/packetshaper/packetshaper.html
> - so I'll see if they have it setup to throttle us or not.

One way to test to see if the cause of large latencies on high volume feeds is 
to have
the request for IDS|DDPLUS be a separate request.  If the latencies for it drop 
to
(near) zero while the latencies for high volume feeds like HDS or CONDUIT stay 
high,
then the indication is that there is packet shaping going on.  If the separate 
IDS|DDPLUS
feed shows high latencies, then the problem is limited bandwidth.
 
> Depending on what I find out with regards to our network throughput, I can 
> broach the
> subject with the chair and faculty here to see if we can cut down on what we 
> request
> within CONDUIT.

Very good.

> Thanks again.

No worries.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IZJ-689237
Department: Support IDD
Priority: Normal
Status: Closed