[IDV #ILT-566886]: vertical profiles with depth, not altitude

> Hi, Yuan
> 
> Thank for your suggestion on the GTSPP netCDF conventions. We will
> explore the framework to develop a new template as suggested by Rich
> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#idp8372832
> 
> One question for you, I have installed IDV on various platforms, such
> as Windows XP, Linux, and Mac. It seems OK. for all platforms, but I
> don't know where or how to use gtspp.jar. Please help.

Go to Tools > Plugin Manager, and then File > Install from file, and select 
this jar file. You need to restart the IDV.

Then, the next time you run the IDV, there is a Data Source Type named GTSPP 
data ... added.

Yuan
> 
> Thanks.
> 
> Cheers,
> 
> Charles
> 
> On Mon, Mar 19, 2012 at 12:15 PM, Unidata IDV Support
> <address@hidden> wrote:
> >> Hi Murray,
> >>
> >> I agree with you 100% that creating TGZ files on the fly is the best way
> >> to go and will bring it to the NODC leadership's attention.
> >>
> >> Charles
> >> On 03/18/2012 10:43 AM, Brown, Murray wrote:
> >> > Gang (sounds right),
> >> >
> >> > Charles' other product line is covered by a different exercise at
> >> > http://marinedataliteracy.org/odv/gtspp_bestnc.htm.  These are TGZ
> >> > compressed nc profiles all loaded at once into the display program
> >> > OceanDataView.  The TGZ files are available per-ocean basin, and
> >> > per-month.
> >> >
> >> > But with that approach and (if we had it) with multi-profile NC files,
> >> > the more important question is how do we reduce the file load to the
> >> > spatial and temporal envelope of interest?  I know beans about the
> >> > technology, but it seems like creating TGZ compilations or NC
> >> > compilations on the fly would be a good bet.  I say this mainly
> >> > because I'm thinking of our IOC students in Pago Pago who have just
> >> > slightly better technology than those old phones we used to stick into
> >> > rubber cradles (yeah, I know you remember Omnet).  The global
> >> > operational game isn't won until we can offer everybody a reasonable
> >> > shot at any kind of data.  I'm very very grateful about just this past
> >> > week's revelations.
> >> >
> >> > Murray
> >> >
> >> > ----- Original Message ----- From: "Rich Signell" <address@hidden>
> >> > To: "Charles Sun" <address@hidden>
> >> > Cc: "Brown, Murray" <address@hidden>;
> >> > <address@hidden>; <address@hidden>; "Reed, Greg"
> >> > <address@hidden>
> >> > Sent: Sunday, March 18, 2012 10:02 AM
> >> > Subject: Re: PC issues
> >> >
> >> >
> >> > Gang,
> >> >
> >> > With the new CF Discrete Sampling Geometry featureTypes implemented in
> >> > NetCDF-Java Common Data Model, you can save many profiles in a single
> >> > file, rather than separate files.   For example:
> >> >
> >> > http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#idp8372832
> >> >
> >> >
> >> > But I don't know if IDV can read these files and do anything useful
> >> > with them yet.
> >> >
> >> > -Rich
> >
> > This is the best solution for this kind of dataset, and the IDV can 
> > certainly handle it better in this approach.
> >
> >
> > Yuan
> >> >
> >> >
> >> > On Sun, Mar 18, 2012 at 9:29 AM, Charles Sun <address@hidden>
> >> > wrote:
> >> >> Hi Murray,
> >> >>
> >> >> It is a typical issue for using IDV or IDV-like applications for reading
> >> >> large number of small size files, either from local hard drive or over
> >> >> Internet.
> >> >>
> >> >> One other maybe unrelated issue that I would like to bring to your
> >> >> attention
> >> >> is a jar file, called gtspp.jar (attached) suggested by Yuen for
> >> >> using IDV
> >> >> with the GTSPP data, has two files, datasource.xml and gstpp.ncml. I
> >> >> don't
> >> >> know too much about IDV. Should the filename, gstpp.ncml, be
> >> >> gtspp.ncml or
> >> >> it does not matter?
> >> >>
> >> >> Regards,
> >> >>
> >> >> Charles
> >> >>
> >> >> On 03/18/2012 07:38 AM, Brown, Murray wrote:
> >> >>>
> >> >>> Dear Folks,
> >> >>>
> >> >>> Saturday morning I began to experience some very strange PC behavior
> >> >>> (no,
> >> >>> I didn't suddenly become a Republican). My laptop, admittedly an
> >> >>> ancient
> >> >>> beast (5 yrs, a Compaq) started running extremely slowly and just
> >> >>> died on
> >> >>> big programs. With no online shopping, wild browsing or other usual
> >> >>> suspects for infection. Checking my memory usage I found that one
> >> >>> instance
> >> >>> of the notorious svchost.exe was steadily growing by the minute, and
> >> >>> had
> >> >>> reached 1.2 Gb. Yep. Rebooting just started the growth cycle again,
> >> >>> and I
> >> >>> could see the file grow about 1 Mb per second with nothing running.
> >> >>> If I
> >> >>> stop that particular svchost.exe (there are usually several) then
> >> >>> some other
> >> >>> stuff can run, but not IDV. It seems, and the Microsoft technical
> >> >>> confirmed, that the runaway instance of svchost.exe is the one that
> >> >>> usually
> >> >>> gets associated with IDV (a little Google searching turned up that
> >> >>> svchost.exe is the controller for dll's called by programs, among other
> >> >>> functions). [Other instances of svchost.exe remain unaffected and
> >> >>> seemingly
> >> >>> benign.] But it starts the weird behavior always, even when IDV isn't
> >> >>> started at all. I'm nuts, my PC is toast, or something weird was
> >> >>> alive and
> >> >>> well in the resources.
> >> >>>
> >> >>> The MS technician said they could clean svchost.exe, but they didn't
> >> >>> know
> >> >>> what was wrong with it and no virus checker (they used 3) could
> >> >>> identify
> >> >>> what it was. "It can be your old PC, or it can be the files you've
> >> >>> worked
> >> >>> with, but we just don't know." I've been working only with GTSPP
> >> >>> files and
> >> >>> IDV for the past 4 days or so (including the new jar), as you've all
> >> >>> seen.
> >> >>> But I'm up and running now, and the main svchost.exe is remaining
> >> >>> stable at
> >> >>> 28Kb. Bullet dodged.
> >> >>>
> >> >>> CHARLES: Sorry I sent that note about your servers, because now I
> >> >>> realize
> >> >>> it could have been the very beginning of my troubles. IDV loads
> >> >>> http://data.nodc.noaa.gov/thredds/catalog/gtspp/realtime/2012/catalog.xml
> >> >>> in
> >> >>> about 2 mins. now and all seems fine.
> >> >>>
> >> >>> Murray
> >> >>>
> >> >>>
> >> >>> ----- Original Message ----- From: "Unidata IDV Support"
> >> >>> <address@hidden>
> >> >>> To: <address@hidden>
> >> >>> Cc: <address@hidden>; <address@hidden>;
> >> >>> <address@hidden>; <address@hidden>;
> >> >>> <address@hidden>
> >> >>> Sent: Friday, March 16, 2012 4:12 PM
> >> >>> Subject: [IDV #ILT-566886]: vertical profiles with depth, not altitude
> >> >>>
> >> >>>
> >> >>>>> Wow. I am so grateful. But could you change the code/name so it is
> >> >>>>> GTSPP?
> >> >>>>>
> >> >>>>
> >> >>>> See the attached.
> >> >>>>
> >> >>>> Y
> >> >>>>>
> >> >>>>> ----- Original Message -----
> >> >>>>> From: "Unidata IDV Support" <address@hidden>
> >> >>>>> To: <address@hidden>
> >> >>>>> Cc: <address@hidden>; <address@hidden>;
> >> >>>>> <address@hidden>;
> >> >>>>> <address@hidden>; <address@hidden>;
> >> >>>>> <address@hidden>
> >> >>>>> Sent: Friday, March 16, 2012 1:11 PM
> >> >>>>> Subject: [IDV #ILT-566886]: vertical profiles with depth, not
> >> >>>>> altitude
> >> >>>>>
> >> >>>>>
> >> >>>>> >> Yuan,
> >> >>>>> >>
> >> >>>>> >> I am aware of it. The typo was in my format conversion program
> >> >>>>> to >>
> >> >>>>> >> create
> >> >>>>> >> the physical files.
> >> >>>>> >>
> >> >>>>> >> By the way, all files in the
> >> >>>>> >> http://data.nodc.noaa.gov/gtspp/realtime/2012/03 are the
> >> >>>>> corrected >> >>
> >> >>>>> >> ones.
> >> >>>>> >>
> >> >>>>> >> Thanks.
> >> >>>>> >>
> >> >>>>> >> Charles
> >> >>>>> >> On 3/16/2012 12:50 PM, Unidata IDV Support wrote:
> >> >>>>> >> >> Yuan,
> >> >>>>> >> >>
> >> >>>>> >> >> This is Charles Sun from US National Oceanographic Data Center.
> >> >>>>> >> >>
> >> >>>>> >> >> We are replacing the old GTSPP netcdf conventions, currently
> >> >>>>> >> >> >> >>
> >> >>>>> >> >> located
> >> >>>>> >> >> in
> >> >>>>> >> >> http://data.nodc.noaa.gov/thredds/dodsC/gtspp/, by moving all
> >> >>>>> >> >> files
> >> >>>>> >> >> below the folder, netcdf4.0, one level up. This will not
> >> >>>>> happen >> >> >>
> >> >>>>> >> >> >> till
> >> >>>>> >> >> late next week.
> >> >>>>> >> >>
> >> >>>>> >> >> I will suggest use
> >> >>>>> >> >>
> >> >>>>> >> >>
> >> >>>>> http://data.nodc.noaa.gov/thredds/dodsC/gtspp/netcdf4.0/indian/2012/01/gtspp_13237135_te_111.nc
> >> >>>>> >> >> for testing purpose. In addition, please be advised that a
> >> >>>>> typo >> >> in
> >> >>>>> >> >> >> >> the
> >> >>>>> >> >> z-variable attribute was found by Rich, which causes the
> >> >>>>> vertical
> >> >>>>> >> >> profile displayed up-side down.
> >> >>>>> >> > Charles,
> >> >>>>> >> > I checked the physical file, and the typo is in the file. Not
> >> >>>>> >> > sure how you plan to fix it.
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > Yuan
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > Murray,
> >> >>>>> >> > See the attached jar file, this is a plugin created for this
> >> >>>>> >> > kind of typo files located locally or remotely. Please
> >> >>>>> install it
> >> >>>>> >> > >> > and
> >> >>>>> >> > restart the IDV, when you load the dataset, you need to
> >> >>>>> select the
> >> >>>>> >> > >> > data
> >> >>>>> >> > source type as "GSTPP data file". You should get what you
> >> >>>>> wanted.
> >> >>>>> >> >
> >> >>>>> >
> >> >>>>> > Sorry I forgot to add the attached file in my last email.
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > Yuan
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > Yuan
> >> >>>>> >> >> However, the typo has been corrected, again all files will
> >> >>>>> be >>
> >> >>>>> >> >> >> updated
> >> >>>>> >> >> by
> >> >>>>> >> >> late next week.
> >> >>>>> >> >>
> >> >>>>> >> >> Sorry for any inconvenience.
> >> >>>>> >> >>
> >> >>>>> >> >> Regards,
> >> >>>>> >> >>
> >> >>>>> >> >> Charles
> >> >>>>> >> >> On 3/16/2012 12:17 PM, Unidata IDV Support wrote:
> >> >>>>> >> >>>> Yuan,
> >> >>>>> >> >>>>
> >> >>>>> >> >>>> Please take a look at the attached figure (#1), which
> >> >>>>> doesn't >> >>>> >>
> >> >>>>> >> >>>> >>>> seem
> >> >>>>> >> >>>> like
> >> >>>>> >> >>>> much, but I'm so proud of it and so thankful to you and
> >> >>>>> Rich >>
> >> >>>>> >> >>>> >>>> Signell
> >> >>>>> >> >>>> for
> >> >>>>> >> >>>> helping me with all the tricks involved. It is based on the
> >> >>>>> >> >>>> "corrected"
> >> >>>>> >> >>>> NetCDF profile that Charles Sun provided (consisting of
> >> >>>>> only a
> >> >>>>> >> >>>> >> >>>> few
> >> >>>>> >> >>>> points).
> >> >>>>> >> >>>> But eventually, we could make these color-coded profiles
> >> >>>>> for >>
> >> >>>>> >> >>>> >>>> groups
> >> >>>>> >> >>>> of GTSPP
> >> >>>>> >> >>>> stations, grabbed online through THREDDS catalogs.
> >> >>>>> >> >>> Murray,
> >> >>>>> >> >>> The physical file (13237135.nc) is corrected from the
> >> >>>>> >> >>> beginning. The problem is the THREDDS server need to be
> >> >>>>> updated
> >> >>>>> >> >>> (I
> >> >>>>> >> >>> will check with TDS developer to make sure). There are two
> >> >>>>> URLs
> >> >>>>> >> >>> >> >>> here
> >> >>>>> >> >>> and confused me:
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> http://data.nodc.noaa.gov/thredds/dodsC/gtspp/indian/2012/01/13237135.nc
> >> >>>>>
> >> >>>>> >> >>>
> >> >>>>> >> >>> and
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> http://data.nodc.noaa.gov/thredds/dodsC/gtspp/netcdf4.0/indian/2012/01/gtspp_13237135_te_111.nc
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> >> >>> Which one are you planning to use?
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> >> >>> Yuan
> >> >>>>> >> >>>> This will be, for us in the water world, a huge step, in
> >> >>>>> terms
> >> >>>>> >> >>>> of
> >> >>>>> >> >>>> efficiency, potential for synthesis, and compatibility
> >> >>>>> with >> >>>> some
> >> >>>>> >> >>>> training
> >> >>>>> >> >>>> we're already doing. The hope (watch out Charles, here
> >> >>>>> comes my
> >> >>>>> >> >>>> soap box)
> >> >>>>> >> >>>> is that we'll have user-defined delivery of spatially and >>
> >> >>>>> >> >>>> >>>> temporally
> >> >>>>> >> >>>> selected individual NetCDF profiles. When the files have the
> >> >>>>> >> >>>> "positive"
> >> >>>>> >> >>>> syntax correction, then the use of 0 to -160 depths gives
> >> >>>>> >> >>>> really
> >> >>>>> >> >>>> FINE
> >> >>>>> >> >>>> figures. [I'm using Ocean Data View for mega-collections
> >> >>>>> based
> >> >>>>> >> >>>> >> >>>> on
> >> >>>>> >> >>>> Charles'
> >> >>>>> >> >>>> TGZ compressions...that's a different matter.]
> >> >>>>> >> >>>>
> >> >>>>> >> >>>> The second figure is one of the existing NC profiles on the
> >> >>>>> >> >>>> GTSPP
> >> >>>>> >> >>>> server, to
> >> >>>>> >> >>>> show you how much better they begin to look with more data >>
> >> >>>>> >> >>>> >>>> points.
> >> >>>>> >> >>>> That
> >> >>>>> >> >>>> figure comes directly from IDV/THREDDS, one click, without
> >> >>>>> any
> >> >>>>> >> >>>> >> >>>> ncml
> >> >>>>> >> >>>> fix. So
> >> >>>>> >> >>>> it's still upside down. I'll need some IDV help on applying a
> >> >>>>> >> >>>> layout that
> >> >>>>> >> >>>> makes the dots larger (perhaps scaled to temp); they ARE
> >> >>>>> >> >> >>>> >>>>
> >> >>>>> >> >>>> currently
> >> >>>>> >> >>>> colored
> >> >>>>> >> >>>> by temp, but it's not easy to see on these small figs.
> >> >>>>> >> >>>>
> >> >>>>> >> >>>> I really hope this dialogue, made necessary by my
> >> >>>>> blundering >>
> >> >>>>> >> >>>> >>>> around,
> >> >>>>> >> >>>> might
> >> >>>>> >> >>>> continue and lead to good IDV/UNIDATA-GTSPP/NODC discussions.
> >> >>>>> >> >>>> It
> >> >>>>> >> >>>> seems to
> >> >>>>> >> >>>> me that a lot could be accomplished.
> >> >>>>> >> >>>>
> >> >>>>> >> >>>> Murray
> >> >>>>> >> >>>>
> >> >>>>> >> >>>>
> >> >>>>> >> >>>
> >> >>>>> >> >>>
> >> >>>>> >> >>> Ticket Details
> >> >>>>> >> >>> ===================
> >> >>>>> >> >>> Ticket ID: ILT-566886
> >> >>>>> >> >>> Department: Support IDV
> >> >>>>> >> >>> Priority: Normal
> >> >>>>> >> >>> Status: Open
> >> >>>>> >> >>>
> >> >>>>> >> >>
> >> >>>>> >> >
> >> >>>>> >> > Ticket Details
> >> >>>>> >> > ===================
> >> >>>>> >> > Ticket ID: ILT-566886
> >> >>>>> >> > Department: Support IDV
> >> >>>>> >> > Priority: Normal
> >> >>>>> >> > Status: Closed
> >> >>>>> >> >
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > Ticket Details
> >> >>>>> > ===================
> >> >>>>> > Ticket ID: ILT-566886
> >> >>>>> > Department: Support IDV
> >> >>>>> > Priority: Normal
> >> >>>>> > Status: Closed
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>>
> >> >>>> Ticket Details
> >> >>>> ===================
> >> >>>> Ticket ID: ILT-566886
> >> >>>> Department: Support IDV
> >> >>>> Priority: Normal
> >> >>>> Status: Closed
> >> >>>
> >> >>>
> >> >>
> >> >
> >> >
> >> >
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: ILT-566886
> > Department: Support IDV
> > Priority: Normal
> > Status: Closed
> >
> 
> 


Ticket Details
===================
Ticket ID: ILT-566886
Department: Support IDV
Priority: Normal
Status: Closed