[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[THREDDS #IXX-362335]: Urgent: UMASS Production Tomcat/THREDDS server shut down due to flood of DNS requests

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [THREDDS #IXX-362335]: Urgent: UMASS Production Tomcat/THREDDS server shut down due to flood of DNS requests
  • Date: Wed, 07 May 2014 08:24:09 -0600
On 4/21/2014 2:37 PM, Signell, Richard wrote:
> New Ticket: Urgent: UMASS Production Tomcat/THREDDS server shut down due to 
> flood of DNS requests
>
> Thredds guys,
>
> UMASS shutdown their production tomcat/thredds and disabled the tomcat
> user on Saturday, which of course is causing an interuption in ocean
> forecast products in New England used by the US Coast Guard, US IOOS
> and the local Weather Service Offices.
>
> Here is there message about why they shut it down.
>
> Any ideas about what was happening and how to get this back up and running?
>
> >From Kent Gardner at UMASSD:
>
> It appears that the SMAST host system that is running Thredds was
> generating a storm of DNS requests to our campus name server. When
> Mike shut Thredds down and disabled the tomcat account the storm
> stopped.
>
> I can think of no legitimate reason why Thredds would be doing this.
> The only thing that remotely comes to mind would be someone trying to
> look up IP numbers in a log file to get the host name for
> informational purposes. Has anyone come across this behavior before in
> Thredds/Tomcat?
>
> Also looking in /tmp we see the following:
>
>   ls -al /tmp|grep tomcat
>
> drwxr-xr-x   2 tomcat       tomcat           4096 Apr 15 19:42 adiandian
>
> -rwxr-xr-x   1 tomcat       tomcat              5 Apr 18 13:11 bill.lock
>
> drwxr-xr-x   3 tomcat       tomcat           4096 Apr 11 14:32 dEDVea
>
> drwxr-xr-x   3 tomcat       tomcat           4096 Apr 14 10:30 dvcdNo
>
> drwxr-xr-x   3 tomcat       tomcat           4096 Apr  8 09:35 fkuQAx
>
> -rwxr-xr-x   1 tomcat       tomcat              5 Apr 18 13:11 gates.lock
>
> drwxr-xr-x   2 tomcat       tomcat           4096 Apr 18 21:52 
> hsperfdata_tomcat
>
> drwxr-xr-x   2 tomcat       tomcat           4096 Mar 28 23:59 httpdlog
>
> --wx--Sr--   1 tomcat       tomcat             51 Apr 16 11:46 notify.file
>
>
> I do not know of any files that Thredds/Tomcat would put in /tmp. Does
> anyone know if any of these files are legitimate?
>
> As far a game plan goes I will need to confer with Mike. At the very
> least we need to scan for and delete all suspicious files, and change
> the password on the tomcat account. After that we start things up and
> monitor the network traffic. "
>
>
>
> Thanks,
> Rich
>
Hi Rich:

One could (mis)configure tomcat to do ns lookup domain name for each 
request. Check or send me *${tomcat_home}/conf/server.xml*

just got back from vacation so catching up.

John



Ticket Details
===================
Ticket ID: IXX-362335
Department: Support THREDDS
Priority: Normal
Status: Open