[IDV #MQM-696970]: visad error propagation facility

Hi Tyn-

> Seems Doug has run into the same issue (error values for each point in the 
> image not being supported at the moment):
> 
> http://mailman.unidata.ucar.edu/mailing_lists/archives/visad/2009/msg00000.html

To have a per pixel error estimate, one would need to create a FieldImpl of 
Reals instead of a FlatField which is what the IDV does now.

> [..] As I've mentioned before, I'd like to see FlatField keep an error 
> estimate for each range value and propagate them through the math and 
> resampling operations. I figured if Real-s can do it, why not FlatFields. Is 
> anyone using visad ErrorEstimate-s or have any thoughts on doing this? [..]
> 
> 
> You, Bill, and Tom talk about the issue in the attached paper:
> 
> [..] Image metadata may include ..., and error estimates for pixel radiances 
> and locations. Developers also have the option to ignore most of these types 
> of metadata, and implement images as simple arrays without units, [..]

It can be supported, but not using FlatFields which is what the IDV uses.  
> Willem has almost finished the ArcInfo ascii GRID dataSource, now adding time 
> from the filename. Can he help add the error-estimate for each pixel in the 
> image as Doug discusses.

Sounds good.  If you would like to work with Doug to implement something and 
contribute that back to the VisAD and IDV communities, that would be great.  We 
do not have the resources to work on this at Unidata at the present time.

Don

> -----Original Message-----
> From: Valentijn Venus
> Sent: Tuesday, February 23, 2010 10:37 PM
> To: address@hidden
> Subject: Re: [IDV #MQM-696970]: visad error propagation facility
> 
> Hi Don-
> 
> > So, do you have an error value for each point in the image?  Or is
> > it an error per image?
> 
> Depends on the field, but for Land-surface Temperature i have it for
> each point in the singleBand of the imageSeq. For other fields i have
> an error per image.
> 
> Cheers, Tyn
> 
> +31 6 290 72 718
> 
> Op Feb 23, 2010 om 22:30 heeft "Unidata IDV Support" <address@hidden
> > het volgende geschreven:\
> 
> > Hi Tyn-
> >
> >> Thanks, sounds wonderful.
> >
> > No problems.  It's more complete this way.
> >
> >> According to this example Tom provides the variable should already
> >> have an error attached before an operation is preformed (see first
> >> two lines):
> >>
> >> a = new Real(10.,1.);
> >> b = new Real(255.,10.);
> >> c = (Real) a.add(b, Data.NEAREST_NEIGHBOR, Data.INDEPENDENT);
> >> System.out.println("sum = "+c);
> >> System.out.println("error of sum is="+c.getError().getErrorValue());
> >> When you run this example, you get:
> >> sum = 265.0
> >> error of sum is=10.04987562112089
> >>
> >> How do I provide the error FieldImpl along with the FieldImpl,
> >> should this be done before I perform sampleToReal?
> >
> > When you call setSamples on the FieldImpl, you would pass in the
> > ErrorEstimate.  I'll have to get more info on what that will look
> > like.
> >
> >> As i wrote earlier: I have a seperate error-field along with the
> >> FieldImpl i do the GridUtil.sampleToReal on. The two FieldImpl
> >> share the same time-dimension as they're from the same ADDE imageSeq.
> >
> > So, do you have an error value for each point in the image?  Or is
> > it an error per image?
> >
> > Don
> >
> >> -----Original Message-----
> >> From: Unidata IDV Support [mailto:address@hidden]
> >> Sent: Tuesday, February 23, 2010 5:24 PM
> >> To: Valentijn Venus
> >> Cc: address@hidden
> >> Subject: [IDV #MQM-696970]: visad error propagation facility
> >>
> >> Hi Tyn-
> >>
> >>> Further to my previous message, I already found some answers to
> >>> some of my questions. However, can you comment on the ??? below.
> >>>
> >>>> What are the numerical equivalents for the other propagations
> >>>> methods to set int errorMode correctly?
> >>>
> >>> Data.NO_ERROR    = 202
> >>> Data.INDEPENDENT = ???
> >>> Data.DEPENDENT   = ???
> >>
> >> Data.INDEPENDENT = 200
> >> Data.DEPENDENT   = 201
> >>
> >> I've added the following to GridUtil so you can use them there:
> >>
> >> public static final int NO_ERRORS = Data.NO_ERRORS;
> >> public static final int DEPENDENT_ERRORS = Data.DEPENDENT;
> >> public static final int INDEPENDENT_ERRORS = Data.INDEPENDENT;
> >>
> >> I've also added an error mode to the sampleToReal. sample already
> >> had the error mode signature.
> >>
> >> These will be in tomorrow's build.
> >>
> >>> http://www.unidata.ucar.edu/software/idv/docs/javadoc/constant-values.html#ucar.unidata.data.grid.GridUtil.DEFAULT_ERROR_MODE
> >>>
> >>>> Is there a javadoc showing all propagations methods?
> >>>
> >>> NO, except this snippet in 
> >>> http://www.unidata.ucar.edu/mailing_lists/archives/visad/2002/msg00170.html:
> >>>
> >>>
> >>>> Also, I can see that Data.INDEPENDENT uses a linear 'error
> >>>> propagation' algorithm, but what does actually Data.DEPENDENT do
> >>>> (doesn't
> >>>> one need to know the functional form of the dependency?). Is
> >>>> there any
> >>>> auxiliary documentation somewhere, or references to the
> >>>> algorithms used?
> >>>
> >>> For Data.DEPENDENT the system uses the worst case error
> >>> propagation, where they are simply added. This is essentially
> >>> interval arithmetic. For Data.INDEPENDENT the system uses
> >>> root mean square error propagation.
> >>
> >> I don't know any more than what was in that message.  As Bill
> >> pointed out, this has not been tested very well. I have no
> >> experience using the error propagation, but if you run into
> >> problems, we can muddle along together. ;-)
> >>
> >> Don
> >>
> >>> -----Original Message-----
> >>> From: Unidata IDV Support [mailto:address@hidden]
> >>> Sent: Wednesday, May 13, 2009 12:25 AM
> >>> To: Valentijn Venus
> >>> Cc: address@hidden
> >>> Subject: [IDV #MQM-696970]: visad error propagation facility
> >>>
> >>> Hi Tyn-
> >>>
> >>>> thanks!
> >>>>
> >>>> Unfortunately, not everything is done with visad.
> >>>
> >>> It will be difficult to do the error propagation fully then.
> >>>
> >>>> On the IDV side, would it take a lot of work to for all methods
> >>>> to pass an error propagation type other than Data.NO_ERROR? Do
> >>>> you think Willem could handle such a task?
> >>>
> >>> You'd need a way for the user to set the error mode and then
> >>> modify all the
> >>> methods in GridUtil that take a sampling method to also take an
> >>> error mode.
> >>> I took a stab at the latter, but not sure I got all the methods
> >>> yet.  Then
> >>> any of the controls/methods that do sampling would need to
> >>> implement this
> >>> as well.
> >>>
> >>> I'm sure there are other gotcha's out there.  We're in the middle of
> >>> finalizing the 2.7 release, so don't have time to work on this.  If
> >>> Willem wants to look into it I can try to answer some of his
> >>> questions.
> >>>
> >>> Don
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: MQM-696970
> >>> Department: Support IDV
> >>> Priority: Normal
> >>> Status: Open
> >>>
> >>>
> >>> Internal Virus Database is out of date.
> >>> Checked by AVG - http://www.avg.com
> >>> Version: 8.0.176 / Virus Database: 270.9.17/1845 - Release Date:
> >>> 12/12/2008 9:02 AM
> >>> Faculty of Geo-Information Science and Earth Observation (ITC)
> >>> University of Twente
> >>>
> >>> E-mail disclaimer
> >>> The information in this e-mail, including any attachments, is
> >>> intended for the addressee only. If you are not the intended
> >>> recipient, you are hereby notified that any disclosure, copying,
> >>> distribution or action in relation to the content of this
> >>> information is strictly prohibited. If you have received this e-
> >>> mail by mistake, please delete the message and any attachment and
> >>> inform the sender by return e-mail. ITC accepts no liability for
> >>> any error or omission in the message content or for damage of any
> >>> kind that may arise as a result of e-mail transmission.
> >>>
> >>>
> >>
> >>
> >> Ticket Details
> >> ===================
> >> Ticket ID: MQM-696970
> >> Department: Support IDV
> >> Priority: Normal
> >> Status: Open
> >>
> >> Faculty of Geo-Information Science and Earth Observation (ITC)
> >> University of Twente
> >>
> >> E-mail disclaimer
> >> The information in this e-mail, including any attachments, is
> >> intended for the addressee only. If you are not the intended
> >> recipient, you are hereby notified that any disclosure, copying,
> >> distribution or action in relation to the content of this
> >> information is strictly prohibited. If you have received this e-
> >> mail by mistake, please delete the message and any attachment and
> >> inform the sender by return e-mail. ITC accepts no liability for
> >> any error or omission in the message content or for damage of any
> >> kind that may arise as a result of e-mail transmission.
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: MQM-696970
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> >
> 
> Faculty of Geo-Information Science and Earth Observation (ITC)
> University of Twente
> 
> E-mail disclaimer
> The information in this e-mail, including any attachments, is intended for 
> the addressee only. If you are not the intended recipient, you are hereby 
> notified that any disclosure, copying, distribution or action in relation to 
> the content of this information is strictly prohibited. If you have received 
> this e-mail by mistake, please delete the message and any attachment and 
> inform the sender by return e-mail. ITC accepts no liability for any error or 
> omission in the message content or for damage of any kind that may arise as a 
> result of e-mail transmission.
> 
> 


Ticket Details
===================
Ticket ID: MQM-696970
Department: Support IDV
Priority: Normal
Status: Open