Re: River Forecasting Center Aggregations

ok i think this is working on 8081 now. let me know if you see any problems.

i dont see the problem with "Size" in files catalog.

JOhn

On 8/10/2012 1:15 PM, David Blodgett wrote:
> Sorry for not being more specific, data is only available through 8/4 and the "Size" 
> doesn't show in the "files" catalogs.
>
> Dave
>
>
> On Aug 10, 2012, at 1:31 PM, John Caron wrote:
>
> > Hi David:
> >
> > can you describe what you saw that seems wrong?
> >
> > Im not seeing that extra domain for KMSR on 8081, but i havent looked harder.
> >
> > John
> >
> > On 8/10/2012 10:25 AM, David Blodgett wrote:
> > > Hi John,
> > >
> > > It turns out that the folks I'm working with here that said things were
> > > back working have been hitting motherlode on 9080. I went to look and it
> > > appears that 8081 isn't doing the right thing with this data.
> > >
> > > Any ideas?
> > >
> > > Dave Blodgett
> > > Center for Integrated Data Analytics (CIDA)
> > > USGS WI Water Science Center
> > > 8505 Research Way Middleton WI 53562
> > > 608-821-3899 | 608-628-5855 (cell)
> > > http://cida.usgs.gov <http://cida.usgs.gov/>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Aug 4, 2012, at 3:34 PM, John Caron wrote:
> > >
> > > > motherlode:8081 should have these merged back, using original domain.
> > > > let me know if you see any problems.
> > > >
> > > > On 8/4/2012 1:34 PM, David Blodgett wrote:
> > > > > This is interesting. I'll need to look at the grid more closely to
> > > > > know what we want to do. Thanks for getting back to us on this.
> > > > >
> > > > > Until we have a different solution, can you ignore the shift just to
> > > > > get the main service back up and running.
> > > > >
> > > > > We've been trying to find a way that we can take over hosting this
> > > > > data by signing up for the IDD stream, but have run into a number of
> > > > > road blocks. The latest is the DOI web application firewall which
> > > > > only allows incoming connections over HTTP and setting up a DMZ is
> > > > > pretty much out of the question. I'm working with some folks from the
> > > > > weather service on this and we will find a solution eventually. In
> > > > > the short run, do you mind continuing to host the real time stream on
> > > > > Motherlode?
> > > > >
> > > > > Last question, sorry to pester. This IDD stream was being hosted on
> > > > > an FTP that went down in early June. We made a bad assumption, that
> > > > > motherlode had a 45 day window of the data. Just realized this
> > > > > yesterday. So our archive is now missing 7 days. Guessing there's
> > > > > nothing to be done at this point? There's no chance of a few extra
> > > > > days still sitting around in a location hidden to the motherlode
> > > > > server is there?
> > > > >
> > > > > Thanks for all your help John, we really appreciate it.
> > > > >
> > > > > Dave
> > > > >
> > > > > On Aug 4, 2012, at 1:20 PM, John Caron <address@hidden
> > > > > <mailto:address@hidden>> wrote:
> > > > >
> > > > > > On 8/1/2012 8:01 PM, David Blodgett wrote:
> > > > > > > Hi John,
> > > > > > >
> > > > > > > We are trying to figure out what's up with the RFC aggregations.
> > > > > > > Jordan already sent you one note.
> > > > > > >
> > > > > > > I followed a link to display this:
> > > > > > > http://motherlode.ucar.edu:9080/thredds/catalog/grib/NPVU/RFC/KMSR-North-Central-RFC/files/catalog.html?dataset=grib/NPVU/RFC/files/KMSR/NPVU_RFC_KMSR_NWS_152_20120730.grib1
> > > > > > > in the toolsUI (the link at the bottom of the dataset page.
> > > > > > >
> > > > > > > That brought me into a thredds catalog chooser tab showing four
> > > > > > > grib1 datasets that are not listed in the root catalog from 7/30,
> > > > > > > 7/31, 8/01, and 8/02.
> > > > > > >
> > > > > > > I dug in and found that
> > > > > > > http://motherlode.ucar.edu:9080/thredds/dodsC/grib/NPVU/RFC/files/KMSR/NPVU_RFC_KMSR_NWS_152_20120802.grib1
> > > > > > > exists, but is not listed in the catalog and is not included in the
> > > > > > > FMRC aggregation.
> > > > > > >
> > > > > > > We've got some beach water quality monitoring folks who REALLY want
> > > > > > > this data to work reliably to drive models that predict beach
> > > > > > > closures so… we're getting some pressure to get things working again.
> > > > > > >
> > > > > > > Any ideas?
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Dave Blodgett
> > > > > > Ok, so what happened is that on 7/30 the GRIB files started showing
> > > > > > up with a slightly shifted grid. All the records that use that new
> > > > > > grid are now in a group called
> > > > > > RFC/PolarStereographic_350X450 (Center 43.70N 93.40W)
> > > > > > which you can see from this catalog:
> > > > > >
> > > > > > http://motherlode.ucar.edu:8081/thredds/catalog/grib/NPVU/RFC/catalog.html
> > > > > > My diagnostics show:
> > > > > >
> > > > > > compare gds1 - gds22
> > > > > > Start x diff : 2.355558
> > > > > > Start y diff : 2.421718
> > > > > > End x diff : 2.355558
> > > > > > End y diff : 2.421718
> > > > > > Center lon diff : 0.031485
> > > > > > Center lat diff : 0.015480
> > > > > >
> > > > > > so apparently it shifted 2.3 km in both x and y direction.
> > > > > > I can "fix" this by telling the server to ignore this shift. Is that
> > > > > > what you want? If so, which grid (old or new) do you want to use? I
> > > > > > assume that going forward it will be on the new grid, but perhaps
> > > > > > you can find out if the shift is real or a bug.
> > > > > > PS:
> > > > > > We are running released versions of TDS 4.3 on
> > > > > > motherlode.ucar.edu:8081, and development testing on 9080. So i
> > > > > > would recommend that you move to 8081  server for now. We will
> > > > > > eventually mover 4.3 onto the "production" server on port 8080, and
> > > > > > you will want to move again when that happens.