[IDD #LAB-620548]: ldm not downloading data

Hi Jordi,

re:
> These last days we didn't receive any data, and although we suspect it's
> due to some changes in the network between our machine and the idd server
> we don't know how to detect the problem. Our server is at eady.uib.es, and
> we get the data files from idd.unidata.ucar.edu. When we run ldmping we get
> these outputs:
> 
> ldmping
> Sep 24 21:00:45 INFO:      State    Elapsed Port   Remote_Host
> rpc_stat
> Sep 24 21:00:45 INFO: Resolving localhost to 127.0.0.1 took 0.000829 seconds
> Sep 24 21:00:45 INFO: RESPONDING   0.002696  388   localhost
> 
> 
> ldmping idd.unidata.ucar.edu
> Sep 24 21:01:10 INFO:      State    Elapsed Port   Remote_Host
> rpc_stat
> Sep 24 21:01:10 INFO: Resolving idd.unidata.ucar.edu to 128.117.140.3 took
> 0.002732 seconds
> (... no more output)

I just checked the LDM log files on all of the real server backend machines
in our idd.unidata.ucar.edu cluster, and I see no indication of any feed
REQUEST(s) from eady.uib.es.  This indicates that your feed REQUEST(s)
are not getting to our machines.

Question/request:

- please send us the output of:

  <as 'ldm' on eady.uib.es>
  telnet idd.unidata.ucar.edu 388

- also send the output of:

  traceroute idd.unidata.ucar.edu

- is it possible that someone configured the firewall on eady.uib.es
  or in the network that eady.uib.es is in to block outbound traffic
  to port 388 on remote machines?

- did someone rebuild the LDM on eady.uib.es recently?

  If yes, the problem is likely that the final step of the LDM build
  process was not run.  The final process is the setting of the 'setuid root'
  bit on the LDM applications 'ldmd' and 'hupsyslog'.  You can check
  to see if this is the problem by doing a long list of the LDM executables:

  <as 'ldm'>
  ls -alt ~ldm/bin/

  'ldmd' and 'hupsyslog' permissions should look like:

  -rwsr-xr-x 1 root Unidata 116818 Apr 16  2013 ldmd*
  -rwsr-xr-x 1 root Unidata  11302 Apr 16  2013 hupsyslog*

  (the dates will be different, of course)

  If the 'setuid root' bit is not set (you don't see the 's' in the
  permissions field), 'root' needs to finish the LDM build as follows:

  <as 'root'>
  cd ~ldm/ldm-<version>/src
  make root-actions

re:
> Does ldmping use the same port as ldm uses to transfer the files?

Yes.

re:
> I mean,
> if that doesn't work, the download isn't going to work either, right?

Correct.

re:
> We also tried: ldmadmin watch
> (Type ^D when finished)
> 
> And nothing else comes out, unless it needs more than a few minutes to come
> out.

We do not see any feed REQUEST(s), so you not getting anything would be
expected.

re:
> Also, doing a portscan to the machine, we get this:
> 
> Open TCP Port: 388     unidata-ldm
> 
> Also tried restarting the service:
> 
> ldmadmin restart
> Stopping the LDM server...
> Waiting for the LDM server to terminate...
> The product-queue is OK.
> Checking pqact(1) configuration-file(s)...
> /usr/local/ldm/etc/pqact.conf: syntactically correct
> Checking LDM configuration-file (/usr/local/ldm/etc/ldmd.conf)...
> Starting the LDM server...
> 
> Looks all right, I guess.

Yes, this looks OK.

re:
> Another thing I tried, but I don't understand its meaning, is this:
> 
> ldmadmin printmetrics
> 
> 20140924.210622 0.00 0.01 0.05 0 0 751673 2971 498434000 1 0 98 0 -1 -1 -1
> -1 160
> 
> Does it help in any way?

No.  The metrics gathered are intended to help sites judge how well their
systems are performing, not if their feed REQUEST(s) are being honored.

re:
> I'm also attaching our ldmd.conf, but we didn't
> make any changes since when it worked. Also, as I said before, the
> networking team say that we should be able to receive data, but we didn't
> make any changes in our configuration and it doesn't work.

Again, was the LDM rebuilt lately?  If yes, I would bet that the cause of
the problem is lack of 'setuid root' privilege on 'ldmd' and 'hupsyslog'.

re:
> Sorry for such a long message. Any advice will be very appreciated. Thanks
> for your help,

Please send us the outputs requested above, and let's go from there.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: LAB-620548
Department: Support IDD
Priority: Normal
Status: Closed