[Datastream #IMR-782553]: THREDDS and GOES-R

Hi Steve,

First, I drove your email to Mohan into our inquiry tracking system so that
more than one Unidata staff member can see the question(s) and provide
information when possible.

re:
> Hope this message finds you well.  I'm contacting you to help with a
> data request/question about UNIDATA's plans with GOES-R data.
>
> As you may know, I am leading a team of researchers into the field
> late next year to lead RELAMPAGO, a multi-agency field campaign led by
> NSF to study severe weather in Central Argentina.  We'll have S-PolKa,
> the DOWs, lots of soundings, and DOE and NASA/NOAA instrumentation in
> the field.  You can read more about the project here:
> https://publish.illinois.edu/relampago/. We are working with the WMO
> WWRP on nowcasting and forecasting in this region also.

This sounds very interesting!

re:
> The topic of GOES-R data and its dissemination keeps coming up.  We do
> not have a direct receive station here at UIUC, and with our budget
> situation in Illinois, I don't think the department can support this.

This is the situation that the great majority of Unidata  university
users find themselves in.

re:
> So we are reliant on what we will get from the LDM or technologies
> like THREDDS to receive real time data.

We're glad to be of service!
 
re:
> We have been working with Steve Goodman at NOAA to collect some ABI
> mesoscale sectors over the region at 500 m spatial resolution, 30
> second time resolution data - and the results are amazing.  See
> examples here: https://vimeo.com/user63620950. Combined with the GLM,
> this data will greatly aid countries in Latin America in regions where
> radar coverage is poor.

We agree.

re:
> Clearly this data has use for nowcasting - the problem is that from my
> standpoint, it is still not clear what real-time dissemination plans
> are for the data.  Typically outside the US, UNIDATA has provided
> limited offerings (which was understood since the imager many times
> was only scanning S. America once an hour or less).

Yes, our ability to provide coverage on higher temporal scales (e.g.,
every 15 minutes, etc.) over South America was limited by the scanning
schedule for GOES-East.  The situation was a little better during the
periods when first GOES-10 and then GOES-12 were re-purposed into South
American surveillance after they had reached the end of their usable
life as operational U.S. East and West surveillance platforms.

re:
> Now, we will be
> getting at least 15 minute resolution, and the native resolution data
> will be extremely useful for research into process understanding as
> well as developing nowcasting tools with our South American
> colleagues.  It is my understanding that the National Met Service in
> Argentina will have a GOES-R receiving station, but it is not clear
> whether the universities will have access to the data (same problem we
> have).

It seems that most GOES-R/S ingest sites are reticent about providing
general access to (near) real-time imagery.  This is likely due to the
sheer volume of the imagery and the _great_ numbers of sites that
will want full, real-time access.

re:
> I've become aware via social media that there is a THREDDS test server
> available that is hosting the data now.  I perused this data (and able
> to plot a nice example using UNIDATA's tools:
> https://gist.github.com/swnesbitt/53dbd8a2953dc2eec63c18c7d4fdb37c),
> and while it is great to finally have the ability to access to the
> data (and is the only place we can get data outside of SSEC, who is
> just providing images via WMS), I'm noticing that the resolution of
> the full disk data is significantly degraded (2 km for VIS, 6 km for IR).

The TDS (THREDDS Data Server) that you are referring to,
thredds-test.unidata.ucar.edu, is serving the GOES-16 sectors that are
being disseminated by the NWS in the NOAAPort Satellite Broadcast Network
(SBN) for use by their forecast offices in AWIPS.  Most, if not all,
of these sectors have been remapped from their original satellite
projection into a conical projection, and, in the case of the full
disk coverage ones, resolution reduced to lessen their size.

re:
> For us in the university community, accessing the data at native
> resolution in real time will be critical for research in many
> disciplines, and for preparations for our field campaign.  Is there a
> roadmap towards this goal of real-time dissemination of the native
> resolution data?

Yes.  Our goal is to provide free-and-open access to GOES-16's 
real-time ABI level 1b images and level 2 products via two data
services: TDS and McIDAS ADDE.  Purchasing the equipment needed
to host these two services is scheduled for the very near future,
and we hope to have everything in-place by the time that NOAA
has declared the GOES-16 imagery to be of operational quality.
The last we heard, this milestone should be in the June time frame.

re:
> I realize that this is a lot of data, and there are challenges to be
> faced in dealing with it.

The biggest challenge that we are anticipating is the sheer number
of sites that will be wanting to access the GOES-16 products in near
real-time (we are getting inquiries from all over Latin America).
Our goal is to keep our data servers open to all, but we recognize
that this may prove to be impossible if the demand exceeds our ability
to serve the data.  If overuse becomes an issue, we will be forced to
implement access controls so that our primary constituency, the U.S.
atmospheric science educational community, continues to be able to
get the data they need.

re:
> Thanks for your time in responding.

No worries.

NB: all support email interactions are made public so we can leverage
answers to questions asked.  If you want our exchanges to remain
private, you need to let us know ASAP (today!) and in every email
exchange that we have.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: IMR-782553
Department: Support Datastream
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.