Enhance.ScaleMissingDefer

  Hi Jon:

below is my unit test for ScaleMissingDefer, below that is a suggested 
way of doing this that should always work (I hope). The Array object 
doesnt know if its been enhanced, so its helpful to include the read(), 
but obviously you can bend that rule if you are careful.

let me know what you think

John

 public void testEnhanceDefer() throws IOException {
    NetcdfDataset ncd = NetcdfDataset.openDataset(filename, 
EnumSet.of(NetcdfDataset.Enhance.ScaleMissing), -1, null, null);
    VariableDS enhancedVar = (VariableDS) ncd.findVariable("t1");

    NetcdfDataset ncdefer = NetcdfDataset.openDataset(filename, 
EnumSet.of(NetcdfDataset.Enhance.ScaleMissingDefer), -1, null, null);
    VariableDS deferVar = (VariableDS) ncdefer.findVariable("t1");

    Array data = enhancedVar.read();
    Array dataDefer =  deferVar.read();

    System.out.printf("Enhanced=");
    NCdumpW.printArray(data);
    System.out.printf("%nDeferred=");
    NCdumpW.printArray(dataDefer);
    System.out.printf("%nProcessed=");

    CompareNetcdf2 nc = new CompareNetcdf2(new Formatter(System.out), 
false, false, true);
    assert !nc.compareData(enhancedVar.getShortName(), data, dataDefer, 
false);

    IndexIterator ii = dataDefer.getIndexIterator();
    while (ii.hasNext()) {
      double val = deferVar.convertScaleOffsetMissing(ii.getDoubleNext());
      ii.setDoubleCurrent(val);
    }
    NCdumpW.printArray(dataDefer);

    assert nc.compareData(enhancedVar.getShortName(), data, dataDefer, 
false);

    ncd.close();
    ncdefer.close();
  }

  private Array getEnhancedArray(VariableDS vds) throws IOException {
    Array data = vds.read();
    EnumSet<NetcdfDataset.Enhance> mode = vds.getEnhanceMode();
    if (mode.contains(NetcdfDataset.Enhance.ScaleMissing))
      return data;
    if (!mode.contains(NetcdfDataset.Enhance.ScaleMissingDefer))
      throw new IllegalStateException("Must include 
"+NetcdfDataset.Enhance.ScaleMissingDefer);

    IndexIterator ii = data.getIndexIterator();
    while (ii.hasNext()) {
      double val = vds.convertScaleOffsetMissing(ii.getDoubleNext());
      ii.setDoubleCurrent(val);
    }
    return data;
  }




On 9/23/2010 8:10 AM, Jon Blower wrote:
> Hi Ethan,
>
> Sorry to pester, but I'd like to tie a few ends up before I go away for
> a while.  Do you think the code below will work as I intend?
>
> Cheers, Jon
>
> -----Original Message-----
> From: Jon Blower
> Sent: 17 September 2010 20:35
> To: 'Ethan Davis'
> Cc: John Caron
> Subject: RE: VariableDS.getOriginalVariable() in TDS
>
> Hi Ethan,
>
> OK, that's good to know, thanks.  Do you think the following code will
> work as a means to guarantee getting data values that are enhanced
> exactly once?
>
> NetcdfDataset nc = ...
> boolean enhanced =
> nc.getEnhanceMode().contains(Enhance.ScaleMissingDefer); // * See below
> GridDatatype grid = ...
> VariableDS enhancedVar = grid.getVariable();
> Variable origVar = var.getOriginalVariable();
> float val;
> if (origVar == null) {
>      val = enhancedVar.read(...);
>      if (!enhanced) {
>          val = enhancedVar.convertScaleOffsetMissing(val);
>      }
> } else {
>      val = origVar.read(...);
>      val = enhancedVar.convertScaleOffsetMissing(val);
> }
>
> * Is it safer to do contains(ScaleMissingDefer) or
> !contains(ScaleMissing)?
>
> Cheers, Jon
>
> -----Original Message-----
> From: Ethan Davis [mailto:address@hidden]
> Sent: 17 September 2010 18:30
> To: Jon Blower
> Cc: John Caron
> Subject: Re: VariableDS.getOriginalVariable() in TDS
>
> Hi Jon,
>
> > Is there a way to get an "unenhanced", plain Variable
> > from a GridDatatype in THREDDS?
> There is not a way to do this in a general way. In particular, variables
> from FMRC datasets and other aggregations will not always (ever?) have a
> single unenhanced variable behind them.
>
> Getting the handling of enhancements "right" is going to take some
> careful thinking and probably a hefty refactoring of the enhance mode
> code as well as the whole stack of Variable/Dataset wrapping code.
>
> We figure this will all come together for TDS 4.10 which we have planned
> for sometime after the Singularity or the Mayan calendar ends, whichever
> comes last. ;-)
>
> All kidding aside, we will review the enhance mode code for TDS 4.3 but
> it may be more than we can chew off for 4.3. And, frankly, access to
> unenhanced data for aggregations and such may not be feasible.
>
> Ethan
>
> On 9/10/2010 6:39 AM, Jon Blower wrote:
> > Hi John and Ethan,
> >
> > I'm porting my latest ncWMS enhancements (mainly simplifications) to
> > THREDDS-WMS.  In order to avoid ambiguities and unintended behaviour
> > concerning dataset enhancements, I've started to use a formulation
> like
> > this:
> >
> > // Doesn't care about enhance mode used when opening the NetcdfDataset
> > float readData(GridDatatype grid)
> > {
> >      VariableDS var = grid.getVariable();
> >      Variable origVar = var.getOriginalVariable();
> >      float val = readDataFromOriginalVariable(origVar);
> >      val = var.convertScaleOffsetMissing(val);
> >      return val;
> > }
> >
> > In ncWMS this works fine and guarantees that we read "raw" values from
> > disk, and that the enhancements are applied exactly once.  But in TDS,
> > origVar turns out to be null (I should have been prepared from this
> from
> > the javadoc).  Is there a way to get an "unenhanced", plain Variable
> > from a GridDatatype in THREDDS?
> >
> > Cheers, Jon
> >
> > --
> > Dr Jon Blower
> > Technical Director, Reading e-Science Centre
> > Environmental Systems Science Centre
> > University of Reading
> > Harry Pitt Building, 3 Earley Gate
> > Reading RG6 6AL. UK
> > Tel: +44 (0)118 378 5213
> > Fax: +44 (0)118 378 6413
> > address@hidden
> > http://www.nerc-essc.ac.uk/People/Staff/Blower_J.htm