20030604: New ingestion machine (cont.)

>From: Lodovica Illari <address@hidden>
>Organization: MIT
>Keywords: 200206201727.g5KHRxa15668 PC LDM GEMPAK

Hi Lodovica,

>I am forwarding to you an e-mail we exchanged last year, regarding
>switching to a PC to ingest IDD data. At that time, after consultation
>with you, we decided to keep running the ingestion on an Alpha.

I recall the exchange we had (and see it below).  It is hard to believe
that the exchange we had was almost an entire year ago!

>Now, we
>have reached the point that a final decision has to be made. Our Alphas
>are going to be phased out and we are buying more PCs.

This matches a general trend in the Unidata community.

>Can we switch to a
>PC running Linux or Solaris is still a better option?

Yes, you can definitely switch to a PC running Linux.  Solaris x86 may
still be technically better in a number of ways, but Linux is more
popular and so is getting much more attention from developers (and so
may have a longer life than Solaris x86).

As a follow-up to a comment I made in my last email, I can add that we
are now actively using PCs running FreeBSD, and my previous comment
about FreeBSD performance versus Linux still stands.

FreeBSD is:

- much faster on the same hardware than Linux
- more secure than Linux

>Please advise us on
>what machine we should get to run the ingestion.

As far as what class of machine to purchase, I can advise you to get as
fast a machine with as much memory and disk as possible (the easy
answer :-).  As far as the operating system goes, I would advise you to
use an OS that someone there has experience using.  If you are starting
fresh with PCs, I would suggest using FreeBSD.

An example:

We have very recently begun using a dual 2.0 Ghz Athlon based PC with 3
GB of RAM with a 120 GB hard disk and 100 Mbps Ethernet (total system
cost was under $1500) running FreeBSD, and it is an incredible
performer.  As an example, when I set up the machine, I created a 2 GB
LDM queue and requested all data feeds available over the IDD from an
upstream server.  1 GB of data (that in the upstream machine's queue
for the past hour) was transferred down to the PC _and_ processed
(decoded into McIDAS compatible data files) within something like 5-7
minutes.  What we don't know yet about this system is how well it will
perform over the long haul.

Depending on how hard you are planning on working your ingest machine
(relay of data, decoding, file serving, etc.), you may want to consider
including a hardware RAID system for your disk storage.  This is what
we are going to be doing with the system mentioned above.

>Thanks

No worries.

>Lodovica Illari

Tom Yoksas

>---------- Forwarded message ----------
>Date: Thu, 20 Jun 2002 14:00:28 -0600
>From: Unidata Support <address@hidden>
>To: Lodovica Illari <address@hidden>
>Cc: address@hidden, address@hidden
>Subject: 20020620: IDD ingestion machine (cont.)
>
>>From: Lodovica Illari <address@hidden>
>>Organization: MIT
>>Keywords: 200206201727.g5KHRxa15668 PC LDM GEMPAK
>
>Lodovica,
>
>>thank you very much for your comments, I have passed them to our system
>>manager. My only questions at this point are related to Solaris vs Linux
>>for the ingestion:
>>
>>.Do you envisage any problem with GEMPAK?
>
>I do not forsee any problem with GEMPAK.
>
>>Do we need two version of
>>GEMPAK(Solaris and Linux) running on different platforms and how
>>compatible they are?
>
>You will, of course, need two different sets of executables since
>Linux executables will not run on Solaris x86 and visa versa.  I
>don't see any problems with compatibility of the data files produced
>by either.
>
>>. Do you think that Linux is going to get better?
>
>Ah... now we are into the realm of peering into the crystal ball :-)
>To answer your question, I would hope that Linux will get better.
>
>>I could keep the
>>ingestion running on the Alpha until they disappear (1 or 2 years max)
>>and then review the situation or is it worth shifting to Solaris now? 
>
>Well, if you have something that works now and is giving you no problems,
>why change?
>
>For reference, some of us here at Unidata are not huge Linux fans.  We
>have exposed lots of warts in Linux over the years, and see that Linux
>is a hackers paradise.  In the not too distant future, we will be
>experimenting with use of FreeBSD as an IDD platform.  My experience
>with FreeBSD (garnered by working on one of our sites machines) is
>that it is:
>
>o much faster on the same hardware than Linux
>o more secure than Linux
>
>Given the backing that Linux appears to be getting from a variety of
>different sectors, things may change.  My responses were directed
>specifically to your original inquiry assuming that you wanted/needed
>to make a change now.  Changing the timeframe for your query may well
>change my answer (kind of like the Heisenberg uncertainty principle) ;-)
>
>Cheers,
>
>Tom