[IDD #UOZ-684486]: Fwd: IRADS data for academic use

Hi Arun,

re:
> I would like to pull the data to the host “date.cs.umass.edu” that can be 
> looked up
> with DNS both ways. I would like NEXRAD level II, level III, and CONDUIT data.

OK.  An LDM on your machine should be able to REQUEST all of this data
from our and other IDD top level relays.  The datasets that contain the
data you want are:

NEXRAD2 - NEXRAD Level II data
NEXRAD3 - NEXRAD level III products
CONDUIT - high resolution model data

re:
> Can you give me a sense of roughly how many bytes of data per day that would 
> be?

The easiest thing to do is to examine the real time stats from any of
the real server backends of our idd.unidata.ucar.edu relay:

Unidata HomePage
http://www.unidata.ucar.edu

  Data -> IDD Operational Status
  http://rtstats.unidata.ucar.edu/rtstats/

    Real Time IDD Statistics -> Statistics by Host
    http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex

uni14.unidata.ucar.edu [ LDM 6.13.3 ]
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?uni14.unidata.ucar.edu

  Cumulative volume summary
  
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?uni14.unidata.ucar.edu

Data Volume Summary for uni14.unidata.ucar.edu

Maximum hourly volume  53650.330 M bytes/hour
Average hourly volume  28979.457 M bytes/hour

Average products per hour     389025 prods/hour

Feed                           Average             Maximum     Products
                     (M byte/hour)            (M byte/hour)   number/hour
CONDUIT                7931.155    [ 27.368%]    23796.376    90695.585
NEXRAD2                7201.278    [ 24.850%]     9730.076    73282.415
NGRID                  5936.483    [ 20.485%]    11252.577    37812.366
NOTHER                 3273.709    [ 11.297%]     6238.838    10110.122
NEXRAD3                2353.193    [  8.120%]     2973.654   101062.634
FNMOC                  1093.682    [  3.774%]     4039.717     2991.244
HDS                     541.072    [  1.867%]      917.453    26419.390
NIMAGE                  222.181    [  0.767%]      406.132      255.439
GEM                     135.235    [  0.467%]      596.902      825.732
FNEXRAD                 132.279    [  0.456%]      166.668      103.902
UNIWISC                  68.293    [  0.236%]      120.454       47.512
IDS|DDPLUS               64.207    [  0.222%]       74.548    44808.195
EXP                      22.734    [  0.078%]       46.660      260.366
LIGHTNING                 3.824    [  0.013%]        7.095      348.683
GPS                       0.133    [  0.000%]        1.165        1.073

I hesitate to give you a number, because the volumes in the various
data feeds change all of the time.  I can say, however that there is
a LOT of data in the three feeds you are interested in!
 
re:
> That
> will help me make archival plans accordingly. Or does IRADS itself have an 
> archival
> service that allows us to pull old data to do back testing experiments?

I can't say what IRADS does or does not have as this is not operated by
us.  I can say that we do _not_ maintain a long term archive (Unidata
does not archive any data) of any of the data you have expressed interest
in.  I can say, however, that the data that is distributed in all three
of the datasets that you have named are archived at:

NOAA's National Centers for Environmental Information (NCEI)
http://www.ncdc.noaa.gov/

Also, as part of the NOAA Big Data Project, all of the real time and historic
NEXRAD Level II data is being freely (for now) hosted in Amazon's AWS.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: UOZ-684486
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.