[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Egrants #YKH-515631]: Role of SSDs in our new distributed EDEX setup

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [Egrants #YKH-515631]: Role of SSDs in our new distributed EDEX setup
  • Date: Tue, 07 Jul 2020 16:15:42 -0600
Hi Gyorgyi,

Congrats on receiving the Equipment Grants Award!  I'm sorry my response has 
been so delayed, I've 
been out of the office for the long weekend until today.
For the SSD I think it would be best to mount /awips2/data_store/ on it.  It's 
possible you might
also want to partition the disk and mount a portion of /awips2/edex/data/hdf5, 
specifically the 
/radar/ directory (/awips2/edex/edex/data/hdf5/radar). I know the SSD can help 
dramatically reduce
write times, and the data_store directory is a good place to start, and won't 
get too large 
(less than 100GB, it fluctuates throughout the day).  The radar directory will 
end up having a lot
of files, that aren't too large, so it might be worth it as well.  But as you 
mentioned, SSD will
have a life limit for the number of writes.  So there is a balance between 
performance and 
longevity.  I suggest to start with you mount the /awips2/data_store/ on the 
SSD and leave the 
rest on the spinning disk.
/awips2/edex/data/hdf5/ will most likely be where the majority of your disk 
space will be used, 
especially since you want to keep a longer archive (10 days or so if I remember 
correctly).
The ldm queue seems to do best if it's running in memory, so that should be 
helpful. 
Also, we are currently working on some things for model data to help reduce 
latencies and improve
overall EDEX performance, so those may be things we should walk you through if 
we aren't able to 
update our EDEX installer by the time you're setting up your servers.

Again, I apologize in the lengthy delay for response.  I am in the office now 
(well from my house)
and well be responding much quicker.
Let me know if this helps.

--Shay Carter

Software Engineer II
UCAR - Unidata

Ticket Details
===================
Ticket ID: YKH-515631
Department: Support AWIPS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.