[IDV #SRS-784269]: PermGen error and memory issues

Thanks for fixing this Don!

Sean

> There is already an item up there on permgen.  I've working on a branch
> that I hope to check in today.
> 
> Don
> 
> On 4/4/12 8:22 AM, Unidata IDV Support wrote:
> > I think that sounds reasonable. Would you like me to post this conversation 
> > to readmine?
> >
> > Sean
> >
> >> Okay, I just checked and I'm running with 128m even on the session where
> >> I run with 10G of max heap on the mac.  We could have the default as 128
> >> and let the user set it to as low as 64 if they want.
> >>
> >> Don
> >>
> >> On 4/3/12 11:41 AM, Unidata IDV Support wrote:
> >>> I run 128 MB on my mac and haven't ran into any issues yet, but I'm sure 
> >>> you abuse your IDV a bit more than me :-)
> >>>
> >>> Sean
> >>>
> >>>> I'd have to run some tests to make sure 128 works on the mac.  It might
> >>>> need to be 256.
> >>>>
> >>>> Don
> >>>>
> >>>> On 4/3/12 11:01 AM, Unidata IDV Support wrote:
> >>>>> I wouldn't have a problem with 128 MB - I don't think it would hurt 
> >>>>> anything, although we should do some checks to make sure that the 
> >>>>> automagic memory management does not allocate too much memory for the 
> >>>>> edge cases when a user puts something like "Use 99% of available 
> >>>>> memory" in their system preferences.
> >>>>>
> >>>>> Sean
> >>>>>
> >>>>>> 64m does not work on a mac, so the default has to be 128.  What if that
> >>>>>> was the default for all OS's?
> >>>>>>
> >>>>>> Don
> >>>>>>
> >>>>>> On 4/3/12 10:00 AM, Unidata IDV Support wrote:
> >>>>>>> Hi Don,
> >>>>>>>
> >>>>>>> The default minimum for the JVM is 64 MB. I think as long as the IDV 
> >>>>>>> can startup at 64 MB, we set it to that. We can recommend that Mac 
> >>>>>>> users bump it up to 128 MB (or maybe we can edit the Mac installer to 
> >>>>>>> use 128 MB by default). I just think it would be nice to get this in 
> >>>>>>> the release. As far as git, if you'd like to sit down and go through 
> >>>>>>> a few "branch, hack, commit, push" cycles, I'd be happy to do so!
> >>>>>>>
> >>>>>>> Sean
> >>>>>>>
> >>>>>>>> Sean-
> >>>>>>>>
> >>>>>>>> The tricky think is what to use as a minimum.  I think that varies 
> >>>>>>>> from
> >>>>>>>> system to system.  On the mac, it seems like 128 is a bare minimum.  
> >>>>>>>> And
> >>>>>>>> then you have to worry about how much total memory the user has.  I 
> >>>>>>>> have
> >>>>>>>> versions of runIDV and runIDV.bat which go along with the user
> >>>>>>>> preference.  I can send those to you if you want.  I'm not having fun
> >>>>>>>> with Git - otherwise, I'd make a branch. ;-)
> >>>>>>>>
> >>>>>>>> Don
> >>>>>>>>
> >>>>>>>> On 4/3/12 9:01 AM, Unidata IDV Support wrote:
> >>>>>>>>> Hi Don!
> >>>>>>>>>
> >>>>>>>>> I was (finally) looking into the PermGen stuff and noticed that you 
> >>>>>>>>> checked in some code back in December to handle this in the System 
> >>>>>>>>> Preference Tab, as well as code to handle saving the preference - 
> >>>>>>>>> awesome! I did some testing with jvisualvm to see how the PermGen 
> >>>>>>>>> space changed while abusing the IDV, and with 6 GB I was able to 
> >>>>>>>>> get 108 MB (I loaded about 20 or so bundles, has 12 views as well 
> >>>>>>>>> as data panels, etc.), so I would say 512 MB is a good upper limit. 
> >>>>>>>>> If you want, I can open a ticket on Redmine for discussion on the 
> >>>>>>>>> topic so that we can get your work in the tree...this will really 
> >>>>>>>>> help windows users, as they get a bit finicky when they have to 
> >>>>>>>>> edit the runIDV.bat script using admin privileges.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>> Sean
> >>>>>>>>>
> >>>>>>>>>> Hi Sean-
> >>>>>>>>>>
> >>>>>>>>>> Thanks for looking into this.  It's becoming more and more of an
> >>>>>>>>>> annoyance (especially on MacOS) and I'm surprised that no one has
> >>>>>>>>>> written in to complain (besides me).
> >>>>>>>>>>
> >>>>>>>>>> I've been using 256 and 512 (depending on how much memory I 
> >>>>>>>>>> allocate to
> >>>>>>>>>> the IDV which varies between 2 and 8 GB).
> >>>>>>>>>>
> >>>>>>>>>> I think we need to allow the user to set it through the system tab 
> >>>>>>>>>> of
> >>>>>>>>>> the user preferences in case the default doesn't work.  I think 
> >>>>>>>>>> starting
> >>>>>>>>>> small (i.e. 128) would be fine and the UI could just be an editable
> >>>>>>>>>> combo box of 128, 256, 512 (editable so the user can type in their 
> >>>>>>>>>> own
> >>>>>>>>>> amount if they want).  The selected value would be used in the
> >>>>>>>>>> runIDV/runIDV.bat scripts.
> >>>>>>>>>>
> >>>>>>>>>> Let me know if you have any questions about how to implement this.
> >>>>>>>>>>
> >>>>>>>>>> Don
> >>>>>>>>>>
> >>>>>>>>>> On 10/7/11 2:02 PM, Unidata IDV Support wrote:
> >>>>>>>>>>> Don,
> >>>>>>>>>>>
> >>>>>>>>>>> What would be the downside of adding -XX:MaxPermSize=128m to 
> >>>>>>>>>>> runIDV? 128 MB seems to keep things under control on my system (I 
> >>>>>>>>>>> think the default is 64MB).
> >>>>>>>>>>>
> >>>>>>>>>>> Sean
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi-
> >>>>>>>>>>>>
> >>>>>>>>>>>> During the recent AAP workshop, we were running the IDV under 
> >>>>>>>>>>>> CentOS in
> >>>>>>>>>>>> a virtual machine set up under Oracle's Virtual Box.  The native 
> >>>>>>>>>>>> system
> >>>>>>>>>>>> had 2 GB of memory and the VM's were set up to allocate 1.5 GB.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The first problem was that the IDV only allocated 512 MB of the 
> >>>>>>>>>>>> 1.5
> >>>>>>>>>>>> available.  This caused problems loading satellite images and 
> >>>>>>>>>>>> any other
> >>>>>>>>>>>> raster data.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The second issue was getting PermGen errors.  This seems to be 
> >>>>>>>>>>>> cropping
> >>>>>>>>>>>> up more and more.  At the very least, there should be an FAQ 
> >>>>>>>>>>>> item on
> >>>>>>>>>>>> this, but it might be useful to have a permgen setting in the 
> >>>>>>>>>>>> System tab
> >>>>>>>>>>>> of the Preferences so the users don't have to edit runIDV to 
> >>>>>>>>>>>> account for
> >>>>>>>>>>>> this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Don
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Don Murray
> >>>>>>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>>>>>> 303-497-3596
> >>>>>>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Ticket Details
> >>>>>>>>>>> ===================
> >>>>>>>>>>> Ticket ID: SRS-784269
> >>>>>>>>>>> Department: Support IDV
> >>>>>>>>>>> Priority: Critical
> >>>>>>>>>>> Status: Open
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Don Murray
> >>>>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>>>> 303-497-3596
> >>>>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Ticket Details
> >>>>>>>>> ===================
> >>>>>>>>> Ticket ID: SRS-784269
> >>>>>>>>> Department: Support IDV
> >>>>>>>>> Priority: Critical
> >>>>>>>>> Status: Open
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Don Murray
> >>>>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>>>> 303-497-3596
> >>>>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Ticket Details
> >>>>>>> ===================
> >>>>>>> Ticket ID: SRS-784269
> >>>>>>> Department: Support IDV
> >>>>>>> Priority: Critical
> >>>>>>> Status: Open
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Don Murray
> >>>>>> NOAA/ESRL/PSD and CIRES
> >>>>>> 303-497-3596
> >>>>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: SRS-784269
> >>>>> Department: Support IDV
> >>>>> Priority: Critical
> >>>>> Status: Open
> >>>>>
> >>>>
> >>>> --
> >>>> Don Murray
> >>>> NOAA/ESRL/PSD and CIRES
> >>>> 303-497-3596
> >>>> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>>>
> >>>>
> >>>
> >>>
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: SRS-784269
> >>> Department: Support IDV
> >>> Priority: Critical
> >>> Status: Open
> >>>
> >>
> >> --
> >> Don Murray
> >> NOAA/ESRL/PSD and CIRES
> >> 303-497-3596
> >> http://www.esrl.noaa.gov/psd/people/don.murray/
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: SRS-784269
> > Department: Support IDV
> > Priority: Critical
> > Status: Open
> >
> 
> --
> Don Murray
> NOAA/ESRL/PSD and CIRES
> 303-497-3596
> http://www.esrl.noaa.gov/psd/people/don.murray/
> 
> 


Ticket Details
===================
Ticket ID: SRS-784269
Department: Support IDV
Priority: Critical
Status: Open