[McIDAS #KXE-710062]: Jython Scripting for GOES Fog Product

Hi Daphne,

Sorry for the slow reply.  It is training workshop time here at Unidata,
and a number of us have been spending lots of time with setup...

re:
> It's been a while since we have corresponded so I just wanted to update
> you on our current status.  We are able to run the co-registration and
> resampling algorithm on our data. The co-registration error seems to be a
> function of time and day of the year. Consequently, the error is different
> for each day and can range anywhere from 0.5-1.4 pixels.

This matches the very limited set of comparisons that I made where the
offset correction was not a simple 1 pixel.

re:
> We resampled
> CH2 for a couple of the days in our dataset and saw an improvement in
> the fog product.

Given the overall resolution of the data (GOES IR channels have a nominal
4 km resolution at the sub-satellite point), I am curious to learn how
big a difference was achieved by the resampling you performed?

re:
> We are currently calculating the errors for the entire
> dataset and applying the error to resample the CH2 images.

Very good.

re:
> Once we have the images corrected, we will want to create a fog product
> for each day using Jython scripting to automate the process. Our
> dataset consists of one time scan for each day between 1-1-2012 through
> 12-31-2012 (the time varies throughout the year).

OK.  I must tell you that I am a long-term McIDAS-X user/developer, so
I can readily help out with how to do things in McIDAS-X.  When it comes
to doing the same in McV/IDV using Jython, I am a novice like you.  Given
this, it might be best (quickest with the possibility of getting the
fastest reply/help) for you to post your questions to the McIDAS-V forums:

http://mcidas.ssec.wisc.edu/forums/

re:
> I started to write
> a script for a few images using the Scripting Tutorial on the McIdas-V
> website and ran into a couple of issues. While the script runs, it doesn't
> do what I want it to do. I have attached the script I wrote along with
> a sample of the images that the script is processing for reference. The
> files with *.area extension are the original scans and the *.area_cal
> is the resampled CH2 image (which has a later timestamp). We will want
> to subtract:
> 
> CH4 Temp(*.area file) - CH2 Temp(*.area_cal file)

OK.  Modulo the lack of a scale and offset, this is a fog product.

re:
> The sample.py is the script I ran and the fogClassification.txt has the
> procedure that I copied and pasted into the 'Local Jython User's Library'.
> 
> 1.) One thing I had trouble with was the listADDEImage request commands.
> How would I go about creating an image list request in order to get
> the CH4 Temp data from the earlier time and the CH2 Temp data from
> the later time for each day?

This I can not tell you since I am not intimately familiar with how
the ADDE server that can hand back temperature data is called within
Jython in McV.  This question is probably best asked/answered in
the McIDAS-V forums.

re:
> 2.) Next thing I came across was that the tutorial uses the sub() function
> to subtract two bands. However, I created a formula in McIDAS-V for the
> fog product which is:
> 
> fogprod(T4,T2)=3D 10*(T4-T2)+128
> 
> Is there a way that I can script this in Jython instead of just using
> the sub() function?

It would seem like this would be straight forward if fogprod was added
to the Jython library.  Exactly how you would do this should be covered
in one of the scripting tutorials.

re:
> 3.) The next question I had was how to save the output of the fog product
> to a new AREA file. The folks at Wisconsin sent us this processing
> command for McIDAS-X:
> 
> imgoper LOC/AREA.2000 LOC/AREA.2001 LOC/AREA.2002 FORM=ADD COEFF=1 -1 
> ACON=128 MCON=10 UNIT=TEMP BAND=4 2 SIZE=ALL

The McIDAS-X example provided is what I am familiar with... the dataset
LOC/AREA is undoubtedly defined to be the set of all images in AREA file
format.  References like LOC/AREA.2000 specify the 2000th member of the
LOC/AREA set, etc.  The IMGOPER command (imgoper.k) will write its output
to the last image referenced in the invocation command line, LOC/AREA.2002,
as long as the LOC dataset is local.  This is virtually the same command
that I was using to create fog products from the small set of examples
that were provided to me at the beginning of our exchanges.  It would
be the same thing I would script in McIDAS-X to work through the entire
collection of images that you want to use in fog product creation.

re:
> In the above command the output is stored in 'LOC/AREA.2002', what
> would be a similar command in McIDAS-V? The fogClassification procedure
> uses the 'destinationDataset=T4Data.clone() and destinationArray[0]'
> commands to store output values, but I am not sure if I am using them
> correctly. We would like to have these fog images stored some where for
> further analysis.

Again, a great question to ask in the McIDAS-V forums.  And, I agree that
you want to keep the fog product generated so that you can reexamine them
with different classification criteria (below).

re:
> I may be getting ahead of myself with the fogClassification procedure.
> This procedure is supposed to loop through each pixel of each fog image
> and classify it as fog, no fog or cloud. But to do this we need to
> look at the fog images to determine what our fog threshold values will
> be. So our first goal is to create a loop that would get the ADDEImage
> data for CH 2 and 4 for each day, apply the fog prod(T4,T2) formula,
> and store the output as a separate Area file on my system.

Yes, I understand.

re:
> Sorry for the long e-mail, but we appreciate your help! We are very
> eager to FINALLY get to the analysis portion of our study.

I feel bad that I am unable to help you immediately with creation of
your fog products using Jython.  If the process was to be done in McIDAS-X,
on the other hand, I could write a simple shell script that would create
and save all of the fog products in AREA files.

Please give the McIDAS-V forums a look.  If you do not find the help
you need (which I think you will), then I will get together with the
IDV developers here in Unidata to see if we can get you going.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KXE-710062
Department: Support McIDAS
Priority: Normal
Status: Closed