[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
20060203: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.
- Subject: 20060203: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- Date: Fri, 03 Feb 2006 09:56:46 -0700
>From: Steve Emmerson <address@hidden> >Organization: UCAR/Unidata >Keywords: 200602031620.k13GKp7s015769 Steve, re: >The permissions are correct for the LDM server programs on both >ftpsvr.er-rsaiia-unc.int and rsaintrf.midds.jsc.nasa.gov: > > ftpsvr: > -rwsrwxr-x 1 root root 188785 Oct 24 17:02 /usr/local/ldm/ > bin/rpc.ldmd > > rsaintrf: > -rwsr-xr-x 1 root users 368408 Dec 13 15:33 /home/ldm/bin > /rpc.ldmd > >so that's not a problem. OK. Did anybody check to see if the LDM had been installed on a file system that was configured to _NOT_ allow execution of setuid programs? This was the problem that CPTEC had. >Host rsaintrf.midds.jsc.nasa.gov is, apparently, also known as >rsaserv.midds.jsc.nasa.gov, so it was possible that reverse lookups >on ftpsvr were getting that name, which didn't have an ALLOW entry. I >had Jackie add one. Brian told me that 'notifyme's worked. Is this possible if there is a reverse lookup problem? >I'm having a hard time envisioning a situation where notifyme(1) would >work but an LDM REQUEST entry wouldn't -- unless some wierd IP-packet >filtering is occurring. I will talk to you about this when I get in... Tom >Regards, >Steve Emmerson > >------- Original Message > >Date: Fri, 03 Feb 2006 08:27:59 -0700 >From: Tom Yoksas <address@hidden>To: "Hoeth, Brian R. \(JSC-W > S > 8\)[LM]" <address@hidden> >cc: "Biggerstaff, Brice A9" <address@hidden>, > "Petit, > Jackie" <address@hidden>, > "Sims, > Vashon J \(MSFC Secondary\)" <address@hidden>, > "Oram, > Timothy D. \(JSC-ZS8\)" <address@hidden>, > "Batson, > Bryan" <address@hidden>, > "Schaffert, > Lowell" <address@hidden>, > "Sautter, > David" <address@hidden>, > "Zickert, > Glenn A" <address@hidden>, > address@hidden, > Steve Emmerson <address@hidden> >Subject: 20060201: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.) > >>From: "Hoeth, Brian R. \(JSC-WS8\)[LM]" <address@hidden> >>Organization: NASA >>Keywords: 200602011404.k11E427s002835 LDM rpc.ldmd notifyme > >Brice et. al., > >re: >>I did run into Tom and his theory is that step 9 of installing the LDM >>(http://www.unidata.ucar.edu/software/ldm/ldm-6.4.4/basics/source-install >>-steps.html) on our (JSC) side was either not performed or that it was >>not successfully completed. This step is as such: > >>9. Install some components with superuser privileges: > >> su root -c 'make install_setuids' > >>Note that you need the superuser's password to accomplish this step. > >>This step is necessary for the LDM server to listen on port 388 (which >>is a restricted port) and for the hupsyslog(1) utility to notify the >>syslogd(8) daemon when a new log-file has been created. > >>He was thinking that this step either wasn't done or it was attempted, >>but some security setting was in place to disallow the setting of user >>ids. I talked briefly with Tim O about this on the phone yesterday. >>Would one or both of you please check on this? > >I just want to clarify what I discussed with Brian in reaction to >his comments that: > >- the downstream was not getting any data from the upstream > >- a 'notifyme' from the downstream to the upstream shows that the > upstream _is_ getting data products put into its LDM queue > >If, as Brian outlined above, the LDM installation on the downstream >did not include the final step: > ><as 'root'> >cd ~ldm-x.x.x/src >make install_setuids > >then the rpc.ldmd process(es) on the downstream will not use port 388 >(since it is a privileged port). Instead, it(they) will attempt to >contact the portmapper on the upstream for the feed connection, and it >is very common for the portmapper to either not be running or the >firewall to be setup to not allow access to the portmapper. The quick >and easy check to see if the final installation step was done is to do >a long listing of the ~ldm/bin directory and look at the permissions on >rpc.ldmd and hupsyslog. If they are not owned by 'root' and/or do not >have the setuid root permission bit set, then the rpc.ldmd will go the >portmapper route. > >The other thing I mentioned is that the file system that the LDM is >installed on could be configured to _NOT_ allow invocation of setuid >root proceses. In this case, the rpc.ldmd process(es) would also run >without 'root' privilege in the initial step of using port 388, and so >should again try to go through the portmapper. The Brazilian >institution CPTEC just went through this exact same scenario, so I know >that it is possible. CPTEC's "solution" was to copy the rpc.ldmd and >hupsyslog process to a file system that was not setup to block >invocation of setuid root processes and create a symbolic link back to >the ~ldm/bin directory. I don't particularly like this "solution" >since it makes doing an upgrade more difficult (by making the person >doing the upgrade remember that they have to copy the executables to >the other file system and redo the symbolic link). > >The other thing I mentioned was checking the request lines in >~ldm/etc/ldmd.conf to make sure that here were no typos in the request >pattern (unprintable characters, etc.). > >Again, my comments to Brian were in reaction to 'notifyme' running >successfully and 'rpc.ldmd' requests not getting any of the data. > >Cheers, > >Tom > >>________________________________ >> >>From: Biggerstaff, Brice A9 [mailto:address@hidden] >>Sent: Tue 1/31/2006 8:57 AM >>To: Petit, Jackie >>Cc: Sims, Vashon J (MSFC Secondary); Oram, Timothy D. (JSC-ZS8); Batson, = >>Bryan; Schaffert, Lowell; Sautter, David; Hoeth, Brian R. (JSC-ZS8)[LM]; = >>Zickert, Glenn A; address@hidden >>Subject: RE: RSA Comm Test (Weather - JSC) LDM Troubleshooting >> >> >> >>Jackie, et al,=20 >> >> Just wanted everyone else to see the text of a note that I sent to = >>Steve Emmerson - without the ldm logs that I sent to him and a few of = >>you. If you got it double, that's why. >> >> Steve,=20 >> >> Thanks for the information about the multi-homed servers. I am = >>enclosing the reciprocal logs for the timeframe that Jackie sent you. = >>They are tar'd up and gzipped. When you unzip it, there won't be a = >>'.tar' extension, but it is a standard tar file. To clarify some = >>information about our situation. The "SA" and "GP" files that Jackie = >>referred to are all transmitted as LDM "EXP" files. Internally they are = >>surface observation and GPsonde balloon data respectively, in ASCII = >>format. On my side I am putting them in using a simple pqinsert = >>manually. Jackie is 'bent-piping' a data feed of similar data files = >>from local data receivers, wind sensors, radar profilers, etc., all in = >>ASCII files transmitted as LDM EXP. She receives the data feed from her = >>local systems and I can see that her server is getting it by using a = >>notifyme command on our server, which you will be able to see when you = >>check the logs. However, none of the data is ever received at either = >>end. Until Friday that is. On Friday Jackie was able to receive two of = >>the surface ob files, but she never received any of the balloon files = >>that I tried to send. The major difference between the surface ob files = >>and the balloon files, as I see it, is size. Surface ob files are ~ 240 = >>bytes and the balloon files are ~18K. =20 >> >> I didn't get a chance to poke Brian about getting to Tom Yoksas at = >>the AMS conference, but I'm sure he will if he gets a chance. As for = >>letting you login in to the systems, the Cape system doesn't have any = >>Internet connection that I'm aware of and ours is a restricted no = >>incoming login, so that won't be doable, I'm afraid. Appreciate your = >>time and assistance, Steve. We are just not seeing what we expect from = >>our experience. >> >> Also, Jackie, I will be on-site close to the LDM server this week = >>working on another piece of our system. So, I won't be seeing my email = >>except briefly in the mornings. If you want to try anything else or if = >>you want to get a conference call set up with Steve or such, try me at = >>281-483-2270 or the operators at 281-483-1045. I'll make sure to check = >>in with them and they should be able to find me most of the time. You = >>can try my pager at 713-764-2601, but in the bowels of the control = >>center, it is sometimes unreliable. >> >>Brice=20 >> >> >>-----Original Message-----=20 >>From: Steve Emmerson [mailto:address@hidden = >><mailto:address@hidden> ]=20 >>Sent: Friday, January 27, 2006 5:24 PM=20 >>To: Petit, Jackie=20 >>Cc: address@hidden; Sims, Vashon J (MSFC Secondary); Oram, = >>Timothy D. (JSC-ZS8); Batson, Bryan; Schaffert, Lowell; Sautter, David; = >>Hoeth, Brian R. (JSC-ZS8)[LM]; Biggerstaff, Brice A9; Zickert, Glenn A; = >>address@hidden >> >>Subject: Re: RSA Comm Test (Weather - JSC) LDM Troubleshooting=20 >> >>Jackie,=20 >> >>>Date: Fri, 27 Jan 2006 17:07:01 -0500=20 >>>From: "Petit, Jackie" <address@hidden>=20 >>>Organization: UCAR/Unidata=20 >>>To: "Steve Emmerson (E-mail)" <address@hidden>=20 >>>Subject: RE: RSA Comm Test (Weather - JSC) LDM Troubleshooting=20 >> >>The above message contained the following:=20 >> >>> Brice, Tim and I got together and did some troubleshooting and I was=20 >>> able to see some files that they sent. For some reason I could only=20 >>> see type SA files (SA04 and SA11).=20 >> >>I'm afraid that I don't know what "SA04" and "SA11" files are.=20 >> >>> When he tried to send GP type files, nothing came through.=20 >> >>I'm afraid I don't know what "GP" files are, either. Sorry.=20 >> >>In general, for a downstream LDM to be able to receive certain = >>data-products from an upstream LDM, the following must be true: >> >> 1. The upstream LDM must receive the data-products. This can be=20 >> verified by executing, on the upstream LDM's host, the command=20 >> >> pqcat -vl- -f <<feedtype>> -p <<pattern>> -o <<offset>>=20 >> >> where=20 >> <<feedtype>> is the feedtype of the data-products = >>(e.g.,=20 >> EXP)=20 >> >> <<pattern>> is the extended regular expression for=20 >> the product-identifier of the = >>data-aproducts.=20 >> >> <<offset>> Is the time-offset in seconds in which=20 >> to go back in the product-queue to find=20 >> matching data-products (e.g., 300 for 5=20 >> minutes).=20 >> >> 2. The downstream LDM must be able to connect to the upstream LDM.=20 >> This can be verified by executing, on the downstream host, the=20 >> command=20 >> >> ldmping -i 0 <<upstream host>>=20 >> >> where <<upstream host>> is the identifier for the upstream host=20 >> (either hostname, fully-qualified hostname, or IP address).=20 >> >> 3. The downstream LDM must be allowed to receive the requested=20 >> class of data-products from the upstream LDM (i.e., the LDM=20 >> configuration-file on the upstream LDM must have appropriate=20 >> entries).=20 >> >>These three items can be combined into executing, on the downstream = >>host, the single command=20 >> >> notifyme -h <<upstream host>> -f <<feedtype>> -p <<pattern>> -o = >><<offset>>=20 >> >>If a downstream LDM process is unable to connect to the upstream LDM = >>server, then the following command can be useful in diagnosing problems: >> >> rpcinfo -n 388 -t <<upstream host>> 300029 6=20 >> >>This command attempts to contact version 6 of program 300029 (the LDM) = >>via a TCP connection to port 388 on host <<upstream host>>. Because = >>this command is non-standard, it might be necessary to adapt it to your = >>system by using different options. >> >>> They were never able to see files from us but did get notified of our=20 >>> files on the notifier. (Brice, please elaborate.)=20 >> >>Notifier?=20 >> >>> They had to get ready for a power outage so Brice asked if I would=20 >>> send you an E-mail to find out if having two ethernet ports could=20 >>> cause a problem with ldm.=20 >> >>By default, the LDM server will listen for incoming connections on all = >>available interfaces. This is, usually, not a problem. We're running = >>the LDM on several multi-homed computers here. >> >>This default can be overridden via the $ip_addr variable in the file = >>"etc/ldmadmin-pl.conf".=20 >> >>> They use the workstation as a bridge/firewall between their LAN and=20 >>> ours. He thinks it may be getting confused since he sees mention of=20 >>> an ldm5 and ldm6. We only see ldm6 referenced in our log (see=20 >>> attached) and only have one ethernet port.=20 >> >>Looking at just one downstream LDM process on host "rsaintrf", I see the = >>following at the beginning of the log file:=20 >> >> Jan 27 21:10:40 ftpsvr rsaintrf[20183] NOTE: Starting Up(6.4.2): = >>rsaintrf.midds.jsc.nasa.gov:388 20060127201040.869 TS_ENDT {{ANY, = >>".*"}}=20 >> >> Jan 27 21:10:40 ftpsvr rsaintrf[20183] NOTE: LDM-6 desired = >>product-class: 20060127210938.347 TS_ENDT {{ANY, ".*"},{NONE, = >>"SIG=3D9b7a056982e167351f69140376671e58"}}=20 >> >> Jan 27 21:10:41 ftpsvr rsaintrf[20183] NOTE: Upstream LDM-6 on = >>rsaintrf.midds.jsc.nasa.gov is willing to be a primary feeder=20 >> >> Jan 27 21:21:01 ftpsvr rsaintrf[20183] ERROR: Terminating due to LDM = >>failure; Connection to upstream LDM closed=20 >> Jan 27 21:21:01 ftpsvr rsaintrf[20183] NOTE: LDM-6 desired = >>product-class: 20060127211936.115 TS_ENDT {{ANY, ".*"},{NONE, = >>"SIG=3D4330b0a1b311f387d2038d03dd7faa67"}}=20 >> >> Jan 27 21:21:01 ftpsvr rsaintrf[20183] ERROR: Terminating due to LDM = >>failure; Couldn't connect to LDM on rsaintrf.midds.jsc.nasa.gov using = >>either port 388 or portmapper; : RPC: Program not registered=20 >> >> ...=20 >> >>The above indicates that, after an initial, successful connection to the = >>upstream LDM on host "ftpsvr" (from 21:10:41 to 21:21:01) the downstream = >>LDM on "rsaintrf" lost the connection and was unable to reconnect = >>because the upstream LDM wasn't available: it was unable to create a TCP = >>connection to port 388 on the upstream host (because nothing was = >>listening on that port) and the LDM wasn't registered with the = >>portmapper on any other port on the upstream host. >> >>The log file also contains the following:=20 >> >> Jan 27 21:11:03 ftpsvr rsaintrf[20348] NOTE: Data-product with = >>signature 60f97e00a793afb4f67c7dd94fe46e41 wasn't found in product-queue = >> >> >> Jan 27 21:11:03 ftpsvr rsaintrf(feed)[20348] NOTE: Starting = >>Up(6.4.2/6): 20060127205131.054 TS_ENDT {{ANY, ".*"}}, Primary=20 >> >> Jan 27 21:11:03 ftpsvr rsaintrf(feed)[20348] NOTE: topo: = >>rsaintrf.midds.jsc.nasa.gov {{ANY, (.*)}}=20 >> Jan 27 21:27:11 ftpsvr rsaintrf(feed)[20348] ERROR: feed or notify = >>failure; HEREIS: RPC: Unable to send; errno =3D Broken pipe=20 >> >> Jan 27 21:27:11 ftpsvr rpc.ldmd[20180] NOTE: child 20348 exited with = >>status 7=20 >> >>The above indicates that an upstream LDM process was started on host = >>"rsaintrf" feeding data-products of feedtype/pattern ANY/.* to a = >>downstream LDM on host "ftpsvr" using primary exchange mode. This = >>process lasted from 21:11:03 to 21:27:11 at which time the upstream LDM = >>was unable to send a data-product to the downstream LDM because the = >>connection was broken for some reason (the reason might be found in the = >>LDM log file on host "ftpsvr"). At this time, the upstream LDM on host = >>"rsaintrf" exited. >> >>I hope this helps. Feel free to contact me with any questions. Also, = >>if I can log onto the systems in question as the LDM user, then I should = >>be able to more easily diagnose any problems. >> >>Incidentally, Tom Yoksas will also be at the AMS meeting in Atlanta, = >>where he will be presenting several papers on the LDM and Internet data = >>distribution. He has considerable experience diagnosing connectivity = >>problems in LDM networks. You might tell Brian Hoeth to look him up. >> >>He'll have a laptop and the two of them might be able to solve all your = >>problems while at the convention.=20 >> >>Regards,=20 >>Steve Emmerson=20 >>LDM Developer=20 >> >> >>------_=_NextPart_001_01C62738.57610B47 >>Content-Type: text/html; >> charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >><META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = >>charset=3Diso-8859-1">=0A= >><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">=0A= >><HTML>=0A= >><HEAD>=0A= >>=0A= >><META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version = >>6.0.6603.0">=0A= >><TITLE>RE: RSA Comm Test (Weather - JSC) LDM Troubleshooting</TITLE>=0A= >></HEAD>=0A= >><BODY>=0A= >><DIV id=3DidOWAReplyText25445 dir=3Dltr>=0A= >><DIV dir=3Dltr><FONT face=3DArial color=3D#000000 = >>size=3D2>Brice,</FONT></DIV>=0A= >><DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV>=0A= >><DIV dir=3Dltr><FONT face=3DArial size=3D2>I did run into Tom and his = >>theory is =0A= >>that step 9 of installing the LDM (<A =0A= >>href=3D"http://www.unidata.ucar.edu/software/ldm/ldm-6.4.4/basics/source-= >>install-steps.html">http://www.unidata.ucar.edu/software/ldm/ldm-6.4.4/ba= >>sics/source-install-steps.html</A>) =0A= >>on our (JSC) side was either not performed or that it was not = >>successfully =0A= >>completed. This step is as such:</FONT></DIV>=0A= >><DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV>=0A= >><DIV dir=3Dltr> </DIV>=0A= >><DIV dir=3Dltr>9. Install some components with superuser = >>privileges: </DIV>=0A= >><DIV dir=3Dltr> </DIV>=0A= >><DIV dir=3Dltr> su root -c 'make install_setuids'</DIV>=0A= >><DIV dir=3Dltr> </DIV>=0A= >><DIV dir=3Dltr>Note that you need the superuser's password to accomplish = >>this =0A= >>step. </DIV>=0A= >><DIV dir=3Dltr> </DIV>=0A= >><DIV dir=3Dltr>This step is necessary for the <A = >>href=3D"glindex.html#LDM">LDM</A> =0A= >>server to listen on port 388 (which is a restricted port) and for the =0A= >><B><CODE>hupsyslog(1)</CODE></B> utility to notify the =0A= >><B><CODE>syslogd(8)</CODE></B> daemon when a new log-file has been =0A= >>created.</DIV>=0A= >><DIV dir=3Dltr> </DIV></DIV>=0A= >><P dir=3Dltr>He was thinking that this step either wasn't done or = >>it was =0A= >>attempted, but some security setting was in place to disallow the = >>setting of =0A= >>user ids. I talked briefly with Tim O about this on the phone =0A= >>yesterday. Would one or both of you please check on this? </P>=0A= >><P dir=3Dltr>Thanks!</P>=0A= >><P dir=3Dltr>Brian</P>=0A= >><DIV dir=3Dltr>=0A= >><DIV dir=3Dltr><FONT face=3DArial size=3D2></FONT> </DIV></DIV>=0A= >><DIV dir=3Dltr><BR>=0A= >><HR tabIndex=3D-1>=0A= >><FONT face=3DTahoma size=3D2><B>From:</B> Biggerstaff, Brice A9 =0A= >>[mailto:address@hidden]<BR><B>Sent:</B> Tue 1/31/2006 = >>8:57 =0A= >>AM<BR><B>To:</B> Petit, Jackie<BR><B>Cc:</B> Sims, Vashon J (MSFC = >>Secondary); =0A= >>Oram, Timothy D. (JSC-ZS8); Batson, Bryan; Schaffert, Lowell; Sautter, = >>David; =0A= >>Hoeth, Brian R. (JSC-ZS8)[LM]; Zickert, Glenn A; =0A= >>address@hidden<BR><B>Subject:</B> RE: RSA Comm Test (Weather - = >>JSC) LDM =0A= >>Troubleshooting<BR></FONT><BR></DIV>=0A= >><DIV>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Jackie, et = >>al,</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Just</FONT> = >><FONT face=3DArial =0A= >>size=3D2>wanted everyone else to see the text of a note that I sent to = >>Steve =0A= >>Emmerson - without the ldm logs that I sent to him and a few of = >>you. If =0A= >>you got it double, that's why.</FONT></SPAN></P>=0A= >><UL>=0A= >> <P><SPAN lang=3Den-us><FONT face=3DArial color=3D#0000ff = >>size=3D2>Steve</FONT><FONT =0A= >> face=3DArial color=3D#0000ff size=3D2>,</FONT></SPAN> </P>=0A= >> <P><SPAN lang=3Den-us><FONT face=3DArial color=3D#0000ff = >>size=3D2> Thanks for =0A= >> the information about the multi-homed servers. I am enclosing = >>the =0A= >> reciprocal logs for the timeframe that Jackie sent you. They are = >>tar'd =0A= >> up and gzipped. When you unzip it, there won't be a '.tar' = >>extension, =0A= >> but it is a standard tar file. To clarify some information about = >>our =0A= >> situation. The "SA" and "GP" files that Jackie referred to are = >>all =0A= >> transmitted as LDM "EXP" files. Internally they are surface = >>observation =0A= >> and GPsonde balloon data respectively, in ASCII format. On my = >>side I am =0A= >> putting them in using a simple pqinsert manually. Jackie is =0A= >> 'bent-piping' a data feed of similar data files from local data = >>receivers, =0A= >> wind sensors, radar profilers, etc., all in ASCII files transmitted as = >>LDM =0A= >> EXP. She receives the data feed from her local systems and I can = >>see =0A= >> that her server is getting it by using a notifyme command on our = >>server, which =0A= >> you will be able to see when you check the logs. However, none = >>of the =0A= >> data is ever received at either end. Until Friday that is. = >>On =0A= >> Friday Jackie was able to receive two of the surface ob files, but she = >>never =0A= >> received any of the balloon files that I tried to send. The = >>major =0A= >> difference between the surface ob files and the balloon files, as I = >>see it, is =0A= >> size. Surface ob files are ~ 240 bytes and the balloon files are =0A= >> ~18K. </FONT></SPAN></P>=0A= >> <P><SPAN lang=3Den-us><FONT face=3DArial color=3D#0000ff = >>size=3D2> I didn't get =0A= >> a chance to poke Brian about getting to Tom Yoksas at the AMS = >>conference, but =0A= >> I'm sure he will if he gets a chance. As for letting you login = >>in to the =0A= >> systems, the Cape system doesn't have any Internet connection that I'm = >>aware =0A= >> of and ours is a restricted no incoming login, so that won't be = >>doable, I'm =0A= >> afraid. Appreciate your time and assistance, Steve. We are just = >>not =0A= >> seeing what we expect from our experience.</FONT></SPAN></P></UL>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Also, = >>Jackie, I will be =0A= >>on-site close to the LDM server this week working on another piece of = >>our =0A= >>system. So, I won't be seeing my email except briefly in the =0A= >>mornings. If you want to try anything else or if you want to get a =0A= >>conference call set up with Steve or such, try me at 281-483-2270 or the =0A= >>operators at 281-483-1045. I'll make sure to check in with them = >>and they =0A= >>should be able to find me most of the time. You can try my pager = >>at =0A= >>713-764-2601, but in the bowels of the control center, it is sometimes =0A= >>unreliable.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Brice</FONT></SPAN> = >></P><BR>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>-----Original =0A= >>Message-----</FONT></SPAN> <BR><SPAN lang=3Den-us><FONT face=3DArial = >>size=3D2>From: =0A= >>Steve Emmerson [</FONT></SPAN><A = >>href=3D"mailto:address@hidden";><SPAN =0A= >>lang=3Den-us><U><FONT face=3DArial color=3D#0000ff =0A= >>size=3D2>mailto:address@hidden</FONT></U></SPAN></A><SPAN = >>lang=3Den-us><FONT =0A= >>face=3DArial size=3D2>] </FONT></SPAN><BR><SPAN lang=3Den-us><FONT = >>face=3DArial =0A= >>size=3D2>Sent: Friday, January 27, 2006 5:24 PM</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>To: Petit, Jackie</FONT></SPAN> = >><BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>Cc: = >>address@hidden; Sims, =0A= >>Vashon J (MSFC Secondary); Oram, Timothy D. (JSC-ZS8); Batson, Bryan; = >>Schaffert, =0A= >>Lowell; Sautter, David; Hoeth, Brian R. (JSC-ZS8)[LM]; Biggerstaff, = >>Brice A9; =0A= >>Zickert, Glenn A; address@hidden</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Subject: Re: RSA Comm = >>Test (Weather =0A= >>- JSC) LDM Troubleshooting</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Jackie,</FONT></SPAN> = >></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>>Date: Fri, 27 Jan = >>2006 17:07:01 =0A= >>-0500</FONT></SPAN> <BR><SPAN lang=3Den-us><FONT face=3DArial = >>size=3D2>>From: =0A= >>"Petit, Jackie" <address@hidden></FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>>Organization: = >>UCAR/Unidata</FONT></SPAN> =0A= >><BR><SPAN lang=3Den-us><FONT face=3DArial size=3D2>>To: "Steve = >>Emmerson (E-mail)" =0A= >><address@hidden></FONT></SPAN> <BR><SPAN = >>lang=3Den-us><FONT =0A= >>face=3DArial size=3D2>>Subject: RE: RSA Comm Test (Weather - JSC) LDM =0A= >>Troubleshooting</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>The above message = >>contained the =0A= >>following:</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> Brice, Tim and I = >>got together =0A= >>and did some troubleshooting and I was </FONT></SPAN><BR><SPAN = >>lang=3Den-us><FONT =0A= >>face=3DArial size=3D2>> able to see some files that they sent. = >>For some =0A= >>reason I could only </FONT></SPAN><BR><SPAN lang=3Den-us><FONT = >>face=3DArial =0A= >>size=3D2>> see type SA files (SA04 and SA11).</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>I'm afraid that I = >>don't know what =0A= >>"SA04" and "SA11" files are.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> When he tried to = >>send GP type =0A= >>files, nothing came through.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>I'm afraid I don't = >>know what "GP" =0A= >>files are, either. Sorry.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>In general, for a = >>downstream LDM to =0A= >>be able to receive certain data-products from an upstream LDM, the = >>following =0A= >>must be true:</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>1. The =0A= >>upstream LDM must receive the data-products. This can = >>be</FONT></SPAN> =0A= >><BR><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>verified by executing, on the upstream LDM's host, the =0A= >>command</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2> pqcat -vl- -f <<feedtype>> -p =0A= >><<pattern>> -o <<offset>></FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>where</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2> =0A= >><<feedtype>> is = >>the =0A= >>feedtype of the data-products (e.g.,</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> =0A= >> =0A= >> =0A= >> <FONT face=3DArial =0A= >>size=3D2>EXP)</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2> <<pattern>> =0A= >> is the extended regular = >>expression =0A= >>for</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> =0A= >> =0A= >> =0A= >> <FONT face=3DArial = >>size=3D2>the =0A= >>product-identifier of the data-aproducts.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2> <<offset>> =0A= >> Is the time-offset in seconds = >>in =0A= >>which</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> =0A= >> =0A= >> =0A= >> <FONT face=3DArial = >>size=3D2>to go back in =0A= >>the product-queue to find</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> =0A= >> =0A= >> =0A= >> <FONT face=3DArial = >>size=3D2>matching =0A= >>data-products (e.g., 300 for 5</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> =0A= >> =0A= >> =0A= >> <FONT face=3DArial =0A= >>size=3D2>minutes).</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>2. The =0A= >>downstream LDM must be able to connect to the upstream = >>LDM.</FONT></SPAN> =0A= >><BR><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>This can be verified by executing, on the downstream host, =0A= >>the</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>command</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2> ldmping -i 0 <<upstream =0A= >>host>></FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>where <<upstream host>> is the identifier for the = >>upstream =0A= >>host</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>(either hostname, fully-qualified hostname, or IP = >>address).</FONT></SPAN> =0A= >></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>3. The =0A= >>downstream LDM must be allowed to receive the requested</FONT></SPAN> = >><BR><SPAN =0A= >>lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>class of data-products from the upstream LDM (i.e., the = >>LDM</FONT></SPAN> =0A= >><BR><SPAN lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>configuration-file on the upstream LDM must have =0A= >>appropriate</FONT></SPAN> <BR><SPAN =0A= >>lang=3Den-us> <FONT = >>face=3DArial =0A= >>size=3D2>entries).</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>These three items can = >>be combined =0A= >>into executing, on the downstream host, the single command</FONT></SPAN> = >></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>notifyme -h =0A= >><<upstream host>> -f <<feedtype>> -p =0A= >><<pattern>> -o <<offset>></FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>If a downstream LDM = >>process is =0A= >>unable to connect to the upstream LDM server, then the following command = >>can be =0A= >>useful in diagnosing problems:</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>rpcinfo -n 388 -t =0A= >><<upstream host>> 300029 6</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>This command attempts = >>to contact =0A= >>version 6 of program 300029 (the LDM) via a TCP connection to port 388 = >>on host =0A= >><<upstream host>>. Because this command is = >>non-standard, it =0A= >>might be necessary to adapt it to your system by using different =0A= >>options.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> They were never = >>able to see =0A= >>files from us but did get notified of our </FONT></SPAN><BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>> files on the = >>notifier. (Brice, =0A= >>please elaborate.)</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial = >>size=3D2>Notifier?</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> They had to get = >>ready for a =0A= >>power outage so Brice asked if I would </FONT></SPAN><BR><SPAN = >>lang=3Den-us><FONT =0A= >>face=3DArial size=3D2>> send you an E-mail to find out if having two = >>ethernet =0A= >>ports could </FONT></SPAN><BR><SPAN lang=3Den-us><FONT face=3DArial = >>size=3D2>> =0A= >>cause a problem with ldm.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>By default, the LDM = >>server will =0A= >>listen for incoming connections on all available interfaces. This = >>is, =0A= >>usually, not a problem. We're running the LDM on several = >>multi-homed =0A= >>computers here.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>This default can be = >>overridden via =0A= >>the $ip_addr variable in the file "etc/ldmadmin-pl.conf".</FONT></SPAN> = >></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> They use the = >>workstation as a =0A= >>bridge/firewall between their LAN and </FONT></SPAN><BR><SPAN = >>lang=3Den-us><FONT =0A= >>face=3DArial size=3D2>> ours. He thinks it may be getting = >>confused since he =0A= >>sees mention of </FONT></SPAN><BR><SPAN lang=3Den-us><FONT face=3DArial = >>size=3D2>> =0A= >>an ldm5 and ldm6. We only see ldm6 referenced in our log (see =0A= >></FONT></SPAN><BR><SPAN lang=3Den-us><FONT face=3DArial size=3D2>> = >>attached) and =0A= >>only have one ethernet port.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Looking at just one = >>downstream LDM =0A= >>process on host "rsaintrf", I see the following at the beginning of the = >>log =0A= >>file:</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:10:40 =0A= >>ftpsvr rsaintrf[20183] NOTE: Starting Up(6.4.2): = >>rsaintrf.midds.jsc.nasa.gov:388 =0A= >>20060127201040.869 TS_ENDT {{ANY, ".*"}} </FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:10:40 =0A= >>ftpsvr rsaintrf[20183] NOTE: LDM-6 desired product-class: = >>20060127210938.347 =0A= >>TS_ENDT {{ANY, ".*"},{NONE, = >>"SIG=3D9b7a056982e167351f69140376671e58"}} =0A= >></FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:10:41 =0A= >>ftpsvr rsaintrf[20183] NOTE: Upstream LDM-6 on = >>rsaintrf.midds.jsc.nasa.gov is =0A= >>willing to be a primary feeder </FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:21:01 =0A= >>ftpsvr rsaintrf[20183] ERROR: Terminating due to LDM failure; Connection = >>to =0A= >>upstream LDM closed </FONT></SPAN><BR><SPAN lang=3Den-us><FONT = >>face=3DArial =0A= >>size=3D2> Jan 27 21:21:01 ftpsvr rsaintrf[20183] NOTE: = >>LDM-6 =0A= >>desired product-class: 20060127211936.115 TS_ENDT {{ANY, =0A= >>".*"},{NONE, "SIG=3D4330b0a1b311f387d2038d03dd7faa67"}} = >></FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:21:01 =0A= >>ftpsvr rsaintrf[20183] ERROR: Terminating due to LDM failure; Couldn't = >>connect =0A= >>to LDM on rsaintrf.midds.jsc.nasa.gov using either port 388 or = >>portmapper; : =0A= >>RPC: Program not registered </FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> = >>...</FONT></SPAN> =0A= >></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>The above indicates = >>that, after an =0A= >>initial, successful connection to the upstream LDM on host "ftpsvr" = >>(from =0A= >>21:10:41 to 21:21:01) the downstream LDM on "rsaintrf" lost the = >>connection and =0A= >>was unable to reconnect because the upstream LDM wasn't available: it = >>was unable =0A= >>to create a TCP connection to port 388 on the upstream host (because = >>nothing was =0A= >>listening on that port) and the LDM wasn't registered with the = >>portmapper on any =0A= >>other port on the upstream host.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>The log file also = >>contains the =0A= >>following:</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:11:03 =0A= >>ftpsvr rsaintrf[20348] NOTE: Data-product with signature =0A= >>60f97e00a793afb4f67c7dd94fe46e41 wasn't found in product-queue =0A= >></FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:11:03 =0A= >>ftpsvr rsaintrf(feed)[20348] NOTE: Starting Up(6.4.2/6): = >>20060127205131.054 =0A= >>TS_ENDT {{ANY, ".*"}}, Primary </FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:11:03 =0A= >>ftpsvr rsaintrf(feed)[20348] NOTE: topo: = >>rsaintrf.midds.jsc.nasa.gov =0A= >>{{ANY, (.*)}} </FONT></SPAN><BR><SPAN lang=3Den-us><FONT face=3DArial =0A= >>size=3D2> Jan 27 21:27:11 ftpsvr rsaintrf(feed)[20348] = >>ERROR: =0A= >>feed or notify failure; HEREIS: RPC: Unable to send; errno =3D Broken = >>pipe =0A= >></FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2> Jan = >>27 21:27:11 =0A= >>ftpsvr rpc.ldmd[20180] NOTE: child 20348 exited with status 7 = >></FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>The above indicates = >>that an upstream =0A= >>LDM process was started on host "rsaintrf" feeding data-products of =0A= >>feedtype/pattern ANY/.* to a downstream LDM on host "ftpsvr" using = >>primary =0A= >>exchange mode. This process lasted from 21:11:03 to 21:27:11 at = >>which time =0A= >>the upstream LDM was unable to send a data-product to the downstream LDM = >>because =0A= >>the connection was broken for some reason (the reason might be found in = >>the LDM =0A= >>log file on host "ftpsvr"). At this time, the upstream LDM on host =0A= >>"rsaintrf" exited.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>I hope this = >>helps. Feel free =0A= >>to contact me with any questions. Also, if I can log onto the = >>systems in =0A= >>question as the LDM user, then I should be able to more easily diagnose = >>any =0A= >>problems.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Incidentally, Tom = >>Yoksas will also =0A= >>be at the AMS meeting in Atlanta, where he will be presenting several = >>papers on =0A= >>the LDM and Internet data distribution. He has considerable = >>experience =0A= >>diagnosing connectivity problems in LDM networks. You might tell = >>Brian =0A= >>Hoeth to look him up.</FONT></SPAN></P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>He'll have a laptop = >>and the two of =0A= >>them might be able to solve all your problems while at the =0A= >>convention.</FONT></SPAN> </P>=0A= >><P><SPAN lang=3Den-us><FONT face=3DArial size=3D2>Regards,</FONT></SPAN> = >><BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>Steve Emmerson</FONT></SPAN> = >><BR><SPAN =0A= >>lang=3Den-us><FONT face=3DArial size=3D2>LDM Developer</FONT></SPAN> =0A= >></P></DIV>=0A= >>=0A= >></BODY>=0A= >></HTML> >>------_=_NextPart_001_01C62738.57610B47-- >Cheers, > >Tom >-- >+----------------------------------------------------------------------------- > + >* Tom Yoksas UCAR Unidata Program > * >* (303) 497-8642 (last resort) P.O. Box 3000 > * >* address@hidden Boulder, CO 80307 > * >* Unidata WWW Service http://www.unidata.ucar.edu/ > * >+----------------------------------------------------------------------------- > + > >------- End of Original Message Cheers, Tom -- +-----------------------------------------------------------------------------+ * Tom Yoksas UCAR Unidata Program * * (303) 497-8642 (last resort) P.O. Box 3000 * * address@hidden Boulder, CO 80307 * * Unidata WWW Service http://www.unidata.ucar.edu/* +-----------------------------------------------------------------------------+
- Follow-Ups:
- Re: 20060203: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- From: Steve Emmerson
- Re: 20060203: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- Prev by Date: Re: 20060201: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- Next by Date: Re: RSA Comm Test (Weather - JSC) LDM Troubleshooting
- Previous by thread: 20060201: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- Next by thread: Re: 20060203: RSA Comm Test (Weather - JSC) LDM Troubleshooting (cont.)
- Index(es):