[LDM #CSW-815764]: back-up cluster for tomorrow

Hello Bongchul, 

Apologies for the delay, I was sick over the weekend and then we got hit by 
a big snow :)

After much discussion, Ryan May (whom I believe you have also been 
communicating with)
and I feel that the best solution for Amazon S3 bucket access for NEXRAD level 
II data, 
would be to use a Unidata THREDDS Data Server to negotiate the requests. It is 
not straightforward
to do directly from the S3 bucket. It can be done, but would require additional 
development, while using a THREDDS
Data Server we would be ready to go. Perhaps fine tuning some Python method to 
do the same thing as THREDDS ~could be 
part of the proposal, altho the THREDDS solution already exists.

Our THREDDS server on the NEXRAD collection lives at:

http://thredds-aws.unidata.ucar.edu/thredds/catalog.html

..or .xml for machine access

Looking forward to your evaluation! :)


> Thank you so much!
> 
> Bongchul
> 
> -----Original Message-----
> From: Unidata IDD Support [mailto:address@hidden]
> Sent: Thursday, January 28, 2016 4:30 PM
> To: Seo, Bong Chul <address@hidden>
> Cc: address@hidden; Seo, Bong Chul <address@hidden>; Keem, Munsung 
> <address@hidden>
> Subject: [LDM #CSW-815764]: back-up cluster for tomorrow
> 
> Hi Bongchul,
> 
> Yes, I think we can help out with this. I will need to explore a few things, 
> let me get back to you tomorrow...or early next week with a solution....and 
> yes, it will likely be in Python.
> 
> 
> 
> > Thank you Jeff for the link and information.
> > What we would like to explore is that we get the data through AWS and 
> > process and generate Hydro-NEXRAD-like products within the same Cloud (this 
> > is going to be preliminary investigation for the proposal that you 
> > discussed with Witek).
> > In that sense, I am wondering how we can specify required information 
> > (e.g., radar site ID and period of interest) in the script (e.g., phtyon) 
> > to get large chunks of data for the past data processing.
> > The python script provided by Amazon only works for getting a single data 
> > file by directly putting  the Level II file path and name in the script.
> > I hope that this helps you understand what we want to do.
> >
> > Best regards,
> > Bongchul
> >
> > -----Original Message-----
> > From: Unidata IDD Support [mailto:address@hidden]
> > Sent: Thursday, January 28, 2016 12:02 PM
> > To: Seo, Bong Chul <address@hidden>
> > Cc: address@hidden; Seo, Bong Chul <address@hidden>; Keem, Munsung 
> > <address@hidden>
> > Subject: [LDM #CSW-815764]: back-up cluster for tomorrow
> >
> > > Hi Jeff and Tom,
> >
> > Hi Bongchul,
> >
> > Yes it is warm in Boulder this weekend, but a possible big snow is on the 
> > way for monday :)
> >
> > I can be a point of contact from Unidata, and depending upon the nature of 
> > your questions there are a variety of people at Amazon to talk with.
> >
> > I saw Witold at the AMS and we discussed a possible proposal to place 
> > Hydro-NEXRAD on top of the collection. I think this is a great idea and has 
> > a good chance for funding.
> >
> > We have a THREDDS Data Server (TDS) on the collection currently for .edu 
> > domains to access. It is located at:
> >
> > http://thredds-aws.unidata.ucar.edu/thredds/catalog.html
> > (To connect using the IDV, substitute catalog.xml for catalog.html when 
> > entering the URL in the Data Chooser.)
> >
> > Hope this helps to get things rolling :)
> >
> > >
> > > Hope that this email find you well and the weather in Boulder is milder 
> > > than other years.
> > > I was recently informed that Amazon started a service providing NEXRAD 
> > > Level II data and Unidata has been working on this with Amazon.
> > > Since our group is trying to figure out how to retrieve data from the 
> > > Amazon Cloud, we'd like to know a contact information (either in Unidata 
> > > or Amazon) to ask some specific questions.
> > > Thank you for your help!
> > >
> > > Bongchul
> > > ---------------------------------------------------------------------
> > > Bongchul Seo
> > > IIHR-Hydroscience & Engineering
> > > The University of Iowa
> > >
> > > -----Original Message-----
> > > From: Unidata IDD Support [mailto:address@hidden]
> > > Sent: Tuesday, June 09, 2015 3:19 PM
> > > To: Seo, Bong Chul <address@hidden>
> > > Cc: address@hidden
> > > Subject: [LDM #CSW-815764]: back-up cluster for tomorrow
> > >
> > > > Hello Jeff,
> > >
> > > Hello Bongchul,
> > >
> > > Great, I am glad you are all set.... :)
> > >
> > > >
> > > > Thank you for your help and I've added lddb in our ldmd.conf.
> > > > Your response was not late and the outage will not affect our system.
> > > >
> > > > Best,
> > > > Bongchul
> > > >
> > >
> > > Jeff
> > > Jeff Weber
> > > Unidata User Support
> > > http://www.unidata.ucar.edu
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: CSW-815764
> > > Department: Support IDD
> > > Priority: Normal
> > > Status: Open
> > >
> > >
> >
> > Jeff Weber
> > Unidata User Support
> > http://www.unidata.ucar.edu
> >
> > Ticket Details
> > ===================
> > Ticket ID: CSW-815764
> > Department: Support IDD
> > Priority: Normal
> > Status: Open
> >
> >
> 
> Jeff Weber
> Unidata User Support
> http://www.unidata.ucar.edu
> 
> Ticket Details
> ===================
> Ticket ID: CSW-815764
> Department: Support IDD
> Priority: Normal
> Status: Open
> 
> 

Jeff Weber
Unidata User Support
http://www.unidata.ucar.edu

Ticket Details
===================
Ticket ID: CSW-815764
Department: Support IDD
Priority: Normal
Status: Open