[IDV #MHJ-258620]: Pydap file server fails

on URL:

http://portal.nersc.gov/pydap/cpmmjo/data4bmapes/wrfidv_d01_2011-11-01_01.nc

netcdf-java is failing when it tries to read a string-valued variable. this is 
happening in the java-opendap code, my guess is that there is a bug in pydap or 
java-opendap for deserializing strings.

it doesnt happen on the other dataset because that one doesnt have a string.

dennis, im going to toss this one to you to see if pydap or java-opendap is not 
following the spec.



> He moved them to a new folder
> http://portal.nersc.gov/pydap/cpmmjo/data4bmapes
> 
> 
> On May 28, 2013, at 12:56 PM, Unidata netCDF Java Support wrote:
> 
> > Im getting a 404 from the browser on
> >
> > http://portal.nersc.gov/pydap/cpmmjo/wrfidv_d01_2011-11-01_01.nc.dds
> >
> > but 200 on
> >
> > http://portal.nersc.gov/pydap/20C_Reanalysis_ensemble/analysis.derived/pwat/pwat_1928.daily.nc.dds
> >
> > that would imply a server problem.
> >
> > did someone seesomething different?
> >
> > John
> >
> >
> >> Thanks for passing it along Yuan.
> >>
> >> The 500MB file opens too quick in idv to have been downloaded I think.
> >>
> >>
> >>
> >> Sent from my iPad
> >>
> >> On May 23, 2013, at 12:10 PM, "Unidata IDV Support" <address@hidden> wrote:
> >>
> >>>> Comments from the PyDAP hosting institution:
> >>>>
> >>>> "Let us know if there is anything we can do on our end to help.
> >>>> Would it be possible to get the underlying HTTP query URL that is 
> >>>> breaking? That would allow us to determing whether this is a client 
> >>>> issue or a server side problem.
> >>>> In other words, I'm looking for the URL that the client sends across, 
> >>>> including the parameters sent in to the file (stuff after the ? symbol).
> >>>> eg. 
> >>>> http://portal.nersc.gov/pydap/cpmmjo/wrfidv_d01_2011-11-01_01.nc?DN[0:1:5][0:1:26]&;
> >>>> "
> >>>>
> >>>>
> >>>> Is there something else they should be running (like Hyrax instead of 
> >>>> pydap?)
> >>>> Are there best practices aspects to this, or do java netCDF library 
> >>>> authors just
> >>>> pick their priorities one by one, and this one is a low priority?
> >>>>
> >>>>
> >>>> Thanks
> >>>> Brian
> >>>
> >>> Brian,
> >>>
> >>>  I moved this ticket to netCDF-Java group to see if they have any good 
> >>> solution. My guessing is that when you try to access the small file in 
> >>> the PyDAP, it is actually downloaded to the local tmp dir and opened in 
> >>> the IDV, and this doesn't work for the very big one and doesn't matter 
> >>> you select one field or the whole file.
> >>>
> >>>
> >>> Yuan
> >>>>
> >>>> On May 20, 2013, at 1:05 PM, Unidata IDV Support wrote:
> >>>>
> >>>>> Brian,
> >>>>>
> >>>>> The problem appears to be a function of the file size. The first file 
> >>>>> you sent us is 3G, the second is 500M. Moreover, IDV support for pydap 
> >>>>> may be limited.
> >>>>>
> >>>>> Best Wishes,
> >>>>>
> >>>>> Unidata IDV Support
> >>>>>
> >>>>>> But if pydap is the problem, why does this one work OK?
> >>>>>> It is on the same pydap server, just a different directory path to the 
> >>>>>> .nc file
> >>>>>> http://portal.nersc.gov/pydap/20C_Reanalysis_ensemble/analysis.derived/pwat/pwat_1928.daily.nc
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On May 20, 2013, at 11:56 AM, Unidata IDV Support wrote:
> >>>>>>
> >>>>>>>> Hello idv folks.
> >>>>>>>>
> >>>>>>>> Any bells ring about why this remote data file fails me?
> >>>>>>>>
> >>>>>>>> Thanks for any ideas.
> >>>>>>>> Brian
> >>>>>>>
> >>>>>>> Brian,
> >>>>>>>   The pydap is quite different comparing to the regular opendap 
> >>>>>>> dataset. The url is basically a http link and the netCDF java library 
> >>>>>>> is not ready to access this kind of dataset.
> >>>>>>>
> >>>>>>>
> >>>>>>> Yuan
> >>>>>>>>> java.io.EOFException.
> >>>>>>>>> In grads, the reading error appears as
> >>>>>>>>> gadsdf: SDF file has no discernable X coordinate.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ---------------
> >>>>>>>>> Example of unopenable URL:
> >>>>>>>>> http://portal.nersc.gov/pydap/cpmmjo/wrfidv_d01_2011-11-01_01.nc
> >>>>>>>>> But when I download the file
> >>>>>>>>> Downloads/wrfidv_d01_2011-11-01_01.nc
> >>>>>>>>> it works OK.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> So it is a PyDAP serving issue.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ------------
> >>>>>>>>> This file on the same server works:
> >>>>>>>>> http://portal.nersc.gov/pydap/20C_Reanalysis_ensemble/analysis.derived/pwat/pwat_1928.daily.nc
> >>>>>>>>> so it seems the PyDAP server itself is OK. Just the files or tree 
> >>>>>>>>> are not set up right?
> >>>>>>>
> >>>>>>>
> >>>>>>> Ticket Details
> >>>>>>> ===================
> >>>>>>> Ticket ID: MHJ-258620
> >>>>>>> Department: Support IDV
> >>>>>>> Priority: Normal
> >>>>>>> Status: Closed
> >>>>>>
> >>>>>> Brian Mapes
> >>>>>> address@hidden or address@hidden
> >>>>>
> >>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: MHJ-258620
> >>>>> Department: Support IDV
> >>>>> Priority: Normal
> >>>>> Status: Closed
> >>>>
> >>>> Brian Mapes
> >>>> address@hidden or address@hidden
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: MHJ-258620
> >>> Department: Support netCDF Java
> >>> Priority: Normal
> >>> Status: Open
> >>>
> >>
> >>
> >
> > Ticket Details
> > ===================
> > Ticket ID: MHJ-258620
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> 
> Brian Mapes
> address@hidden or address@hidden
> 
> 
> 
> 

Ticket Details
===================
Ticket ID: MHJ-258620
Department: Support netCDF Java
Priority: Normal
Status: Open