Re: Using HttpClient/HttpClientManager in NJ 4.2

Hi Robert:

1. ok, we see the multiple challenge problem and are working on a solution.

2. slow download is mostly due to latency, because of the way 
HTTPRandomAccessFile works. Each "seek" causes a round-trip (!). Theres 
no fix for this, better is to use an opendap server for performance.

3. We can probably cut round-trips in half by doing pre-emptive 
authorization. Im not sure if that used to work in 4.2.18 or not. We 
will try to get that working also.

4. We will fix the "user agent" problem in the next release also.

Thanks for reporting the problem, we will get back to you when we have a 
fix.

John


On 3/11/2011 12:49 PM, Robert B. Schmunk wrote:
> John,
>
> The test files are at
>
> http://www.giss.nasa.gov/staff/rschmunk/test/file1.nc
> http://www.giss.nasa.gov/staff/rschmunk/test/file2.nc
>
> Try userid 'jcaron' and password 'boulder'.
>
> Another problem that I encounter with authentication
> in 4.2.21 is that after I have successfully loaded a
> file from a remote directory (albeit having to enter
> the password twice), when I request a second file from
> the same directory, it requests authentication again.
> This is not the case in 4.2.18.
>
> The problem that I reported regarding the slow download
> is apparently not caused by the authentication issues,
> although it's possible that it is exacerbated by it.
> The csae in which I am primarily running into occurs
> when I create a lat-vert plot from an array with lon,
> lat and vert dimensions. I'm averaging over all the
> points in the lon dimension, but am getting each slice
> along that dimension individually, which means a lot of
> individual requests being fired off to the remote server.
>
> rbs
>
>
>
>
>
>
> On Mar 11, 2011, at 09:12, John Caron wrote:
>
> > Hi Robert:
> >
> > We have been refactoring that code, so im guessing we havent got it right yet.
> >
> > Can I get access to the remote netcdf file so I can recreate?
> >
> > John
> >
> > On 3/8/2011 9:58 PM, Robert B. Schmunk wrote:
> > > John,
> > >
> > > I'm trying to diagnose some problems using the NJ libraries to
> > > access a remote netCDF dataset located in a website directory
> > > that requires basic HTTP authentication. It's getting a bit
> > > frustrating, so I'm writing for any advice you might have.
> > >
> > > I've created a CredentialsProvider dialog to handle the necessary
> > > user input, and in general, it seems to work okay except that...
> > >
> > > The first problem is that using NJ 4.2.21, when one enters an ID
> > > and password and the result, the first attempt to enter these
> > > credentials is _always_ rejected, and one is asked to enter them
> > > again. The second time, it always works (assuming the ID and
> > > password are correct, of course).
> > >
> > > Once access is gained, I found that the data downloads were
> > > taking much too long. A check of the webserver logs revealed
> > > that the incoming requests were alternating between no
> > > credentials received and a 401 response code, and receiving
> > > the credentials but only passing back 20 kB of data.
> > >
> > > I reverted to using NJ 4.2.18, which I had been using for a
> > > couple months. The first problem above went away. The second
> > > remained.
> > >
> > > As a side issue, I found that with 4.2.21, the user agent info
> > > received by the server was "Jakarta Commons-HttpClient/3.1",
> > > but with 4.2.18, it included the user agent info that I
> > > passed when creating the HttpClient object.
> > >
> > > My code when using NJ 4.2.21 looks like
> > >
> > >      public static void initHttpClient (
> > >          CredentialsProvider cprov)
> > >      {
> > >          HttpClientManager.init (cprov,
> > >              "Panoply/3.0 (http://www.giss.nasa.gov/tools/panoply/); ");
> > >      }
> > >
> > > My code when using 4.2.18 looks like
> > >
> > >      public static void initHttpClient (
> > >          CredentialsProvider cprov)
> > >      {
> > >          HttpClient client = HttpClientManager.init (cprov,
> > >              "Panoply/3.0 (http://www.giss.nasa.gov/tools/panoply/); ");
> > >          HTTPRandomAccessFile.setHttpClient (client);
> > >          CdmRemote.setHttpClient (client);
> > >          NetcdfDataset.setHttpClient (client);
> > >          DConnect2.setHttpClient (client);
> > >      }
> > >
> > > Thanks,
> > >
> > > rbs
> > >
> > >
> > > --
> > > Robert B. Schmunk, address@hidden
> > > NASA Goddard Institute for Space Studies, 2880 Broadway, New York, NY 10025
> --
> Robert B. Schmunk, address@hidden
> NASA Goddard Institute for Space Studies, 2880 Broadway, New York, NY 10025