[McIDAS #JGN-249914]: IMGMAKE

Hi Mike,

re:
> COD seems to be having some problems with Unidata support emails and
> your last few didn't get to me reliably.

This is very weird, but now that you say it, I am reminded that we have
"talked" about this before.

re:
> I haven't missed anything as
> I've been watching the email archive page, but for some reason it's
> becoming a consistent problem.  Would you mind CCing
> xxxxx@xxxxx.xxxxx on future messages for this thread?

Any time that you include a CC in one of your replies, it will get added
as a CC to all future replies from Unidata User Support.  It would be
better if you could add a CC on your side than on ours as I can pretty
much guarantee that I will forget about the need to add a CC (sorry!).

re:
> Would it
> also be possible to blank out that email address before this hits the
> archive page?

Just so you know: the way that an exchange that is conducted through
our inquiry tracking system gets made public is when the Unidata staff
member that is responding to an inquiry includes a CC to:

support-<topic>@unidata.ucar.edu

Once this CC is done, it is included in the ticket and will be made
automatically each time a staffer sends a reply.  The way that we can
stop the follow-on exchanges being made public is by removing the CC
to:

support-<topic>@unidata.ucar.edu

Also, the staffer's reply will be sent to the owner of the ticket (Paul in
this case), and CCed to all users in the ticket's CC list.  The CC should
not be included in the email header of the copy CCed to support-<topic>,
so anonymity should be maintained.  Inline references to email addresses,
on the other hand, are not blanked out by the script that we use to
HTMLize the exchange in preparation for making it public, so it is up to
the staff member to remove/obfuscate any email address in the reply body.
I have done just this above by munging the email address you want CCed.

re:
> I'm responding to your last email in-line below.  Thanks!
re: test of NOAAPort-delivery of unremapped GOES-16 imagery

> I saw a TIN message about this a week or so ago, but I thought it was
> supposed to be today.

I believe that the test starts on September 6 which is tomorrow, but since
times used are UTC, it is only one hour+ away as I write this.  This does
not mean that the test will start at 0 UTC, however.

re:
> I wanted to watch that, save some data and do
> some tests, but became unexpectedly busy around here today.  If it's
> actually tomorrow I might have a chance to catch it.

The test is scheduled for September 6.

As I understand it, the images that will be sent as part of the test will
be in the FixedGrid projection.  I am not sure if the test images will
replace the existing ones, or be in addition to the existing ones.

re: At least one of the reasons for the test is reports by forecast offices that
> the remapped imagery can miss features (e.g., fire) signatures, so they are
> not as useful as they could be.

> Interesting, I didn't know that was happening but I could understand why.

Yup.  In fact, the NOAAPort delivered images (except for the full disk ones)
are remapped twice: the first remap is to put the image into the FixedGrid
projection (or, this is what I understand it to be), and the second is
to put the appropriate sector into MERCator or Lambert Conformal projections.

re: I am hopeful that this test will result in a replacement of the remapped
> sectors in the NOAAPort SBN-delivered and LDM/IDD relayed imagery with
> ones that are created directly from ones being distributed in the GOES
> ReBroadcast (GRB).  Part of my hope is that the internals of the un-remapped
> images will be the same as those in the GRB.  If they are, McIDAS end users
> will be able to serve the data via ADDE using the SSEC-developed core
> ABIN servers (abinadir.cp and abinaget.cp).  This will be a big win for
> McIDAS users, and users of ADDE enabled applications like the IDV, McIDAS-V
> and McIDAS-X.

> Also interesting, I wonder if that will be the case.  I'm also
> wondering if this could potentially solve the black pixel problem with
> visible bands (I can dream can't I).

It should definitely solve the "black pixel" problem seen in the VIS bands.

re:
> I hope I can capture some of this data when they send it.  One of my
> fears is if they do switch to that it might not be compatible with the
> Python code we have making GOES-16 imagery now.

Is your Python code doing anything with the values needed to do calibration
of the images, or are you just working with the raw (e.g., 12-bit) values
in the image?  Just curious...

re:
> I'd like a glimpse to
> see how much work might lay ahead of me if I need to re-engineer a few
> things.

It may sound glib of me to say, but them switching to internal information
that matches what is in the GRB would be a BIG win for you in the long
run.  It may cause a bit of pain in the short run (meaning tweaking what
you have in place), but that should be relatively straightforward.  Also,
NOAA has been abundantly clear that the GOES-16 imagery is not to be
considered to be operational yet.  Them changing things substantially
to make the data more scientifically useful is in the spirit of their
continued warnings to not treat the imagery as being operational.

One other comment:

If one names the L2 products being delivered in NOAAPort and restitched
together using Ryan May's ldm-alchemy using the name included in the
'dataset_name' global attribute of the reconstituted scenes, then one can
use the SSEC core ABIN servers for the L2 products!  I figure this out
as I was reviewing the internals of the L2 products in preparation for
writing an ADDE server for them.  What I saw was the internals looked
much more like the ABI images being delivered in the GRB, and all
products are in the FixedGrid projection.  As soon as I saw that the
L2 files all have a 'dataset_name" global attribute, and its value
looks like the naming convention that CSPP GEO uses for the ABI
imagery, I decided to test to see if the ABIN servers worked with
one of the L2 products, TPW.  To my delight, it did, and the calibrated
values returned from IMGPROBEs matched the values reported by the IDV
when working with the file directly.  This was an Ah Ha moment for me!

re:
> Thanks for letting me know, Tom.  One more thing.  I saw another
> support message on the email archive page mentioning v2017 may have
> some python integration.  I just wanted to mention I would be THRILLED
> for that addition!  

What SSEC has done in their v2017.1 is make some XRD (X Research and 
Development)
code available.  Fair warning: XRD code is not fully supported by SSEC;
in fact, some XRD routines are included that only pass the weak test of
being able to be compiled!

That being said, I just included the SSEC v2017.1 XRD Python "stuff" in
my v2017 pre-release, so it will be available.

re:
> If that's part of the pre-release or if there's a way I can test that
> package now, let me know.

I have been biding my time about doing more development of the ADDE servers
that I wrote for the NOAAPort-delivered GOES-16 imagery in anticipation
of positive results of the testing that should start tomorrow.  If the
test shows NOAA that they should dump the remapped sectors currently
being sent in NOAAPort (as tiles, of course) in favor of unremapped
ones, AND if they make the _what I consider smart_ move of making the
internals of the netCDF4 files that are being delivered match the
internals of the netCDF4 files that are in the GRB, then McIDAS users
and users of images served by ADDE will benefit greatly.  McIDAS users
will be able to use the core SSEC ABIN ADDE servers directly, and this
means that they can use McIDAS to IMGCOPY into AREA files; have correct
calibration; etc.  In this case, I will gladly pull my code from my
distribution chalking it up to work wasted but a huge benefit to end users.

re:
> Thanks again,

No worries.

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: JGN-249914
Department: Support McIDAS
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.