[McIDAS #IKW-702363]: Re: [DATA:] I need some reasoning for servers for weather (fwd)

Hi Paul,

I started this reply to your request some time ago... I just found that
I never sent my comments -- too many other things came up in the
interim (like the COD NOAAPort ingest move to Ubuntu); sorry!

re:
> You know much about what we are doing. We are also planning on running a
> plethora of dual-pol images and generate them for web use. Our IT
> department wants to run VM servers. I do not know how to answer the
> questions Dave asks below. Any help is greatly appreciated.

Hmm... It is my experience that generating web content from McIDAS-X
processing is not that big of CPU user.  Having said that, however, I
do not routinely generate LOTS of images for web use.

Are you thinking about generating a GIF/JPEG for each dual-pol product
for each station producing dual-pol products for every reporting period?
If yes, this certainly is a LOT of images.

Some questions:

- have you captured any metrics that would help quantify the problem at
  hand:

  - how many images need to be created every hour

  - the length of time it takes to create an image for each NEXRAD
    site for a single product

My gut feeling is that McIDAS running in a sufficiently ample virtual machine
could produce all of the plots that you need/want.  The environment I run in
almost 100% of the time anymore is a CentOS 6.2 x86_64 VMware virtual machine
running on my Dell Studio 14 (I7, 8 GB ram) laptop that is running Windows 7.
I dedicate 2 GB and 2 cores to the virtual machine, and I never experience
any hiccups performance-wise.  In fact, this is my primary McIDAS development
environment anymore.  The fact that I can build the full Unidata 
McIDAS-X/-XCD/-XRD
distribution in about 11 minutes in this environment is one pretty good measure
of how fast it is.  I have generated a series of GIF images from satellite 
displays,
but the overhead was so small that I don't remember much about it (if it had
been slow, I _would_ have remarked to myself that the environment was not
really ready for "prime time").

I don't know if the above helps at all.  Please keep me informed about what
directions COD is heading in compute-wise.

> To: address@hidden
> Subject: Re: [DATA:] I need some reasoning for servers for weather
> 
> Paul, what I need from what you can give me now is some justification of
> the processing power needed.  Because upper management wants to move
> everything to VM to consolidate costs and harddrive usage, I need to show
> both why we are using this much otherwise they keep saying its a webserver
> basically and we have our websites running off of vm.  Gil gave me the data
> numbers of data/hour/day type of stats, but what I need to know is the
> justification for such powerful processors.  Why can we not limit our data
> processing and run it off a dual processor.  I don't know all what you guys
> are doing nor what you have planned in the future.  Too many things on my
> plate now as it is with other things like wireless, firewall issues, and
> the CIS department complaining that their service here is not up to par for
> their way of doing things.
> 
> ============================
> Dave Bukowski -- NREMT-B
> BLS Instructor (CPR & AED)
> N9KPD
> http://davebb.deviantart.com/
> address@hidden
> address@hidden
> 
> 
> address@hidden> wrote:
> 
> > On Fri, 30 Mar 2012, Gilbert Sebenste wrote:
> >
> >   Also Gil, maybe you can answer this one or Tyler can.  How much data do
> >>> we
> >>>  ingest in a single day and how much is scourted (estimate on the scour).
> >>>
> >>
> >> Climate:
> >>
> >> http://www.unidata.ucar.edu/**cgi-bin/rtstats/rtstats_**
> >> summary_volume?climate.cod.edu<http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?climate.cod.edu>
> >>
> >> Weather:
> >>
> >> http://www.unidata.ucar.edu/**cgi-bin/rtstats/rtstats_**
> >> summary_volume?weather.cod.edu<http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?weather.cod.edu>
> >>
> >> CDstats:
> >>
> >> http://www.unidata.ucar.edu/**cgi-bin/rtstats/rtstats_**
> >> summary_volume?cdstats.cod.edu<http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?cdstats.cod.edu>
> >>
> >
> > NOAAport (incomplete/low due to less than great reception of data):
> >
> > http://www.unidata.ucar.edu/**cgi-bin/rtstats/rtstats_**
> > summary_volume?noaaport.cod.**edu<http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?noaaport.cod.edu>
> >
> >
> > ****************************************************************
> > *******************
> > Gilbert Sebenste
> > ********
> > (My opinions only!)                                                  ******
> > Staff Meteorologist, Northern Illinois University                      ****
> > E-mail: address@hidden
> >  ***
> > web: http://weather.admin.niu.edu                                      **
> > Twitter: 
> > http://www.twitter.com/NIU_**Weather<http://www.twitter.com/NIU_Weather>    
> >                        **
> > Facebook: 
> > http://www.facebook.com/niu.**weather<http://www.facebook.com/niu.weather>  
> >                         *
> > ****************************************************************
> > *******************

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IKW-702363
Department: Support McIDAS
Priority: Normal
Status: Closed