[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IDV #XXB-855611]: consider this bundle for your memory profiling
- Subject: [IDV #XXB-855611]: consider this bundle for your memory profiling
- Date: Fri, 08 Feb 2013 12:20:06 -0700
Brian,
Thanks for submitting this bundle for analysis. The IDV team has scrutinized
this bundle and we would like to tell you about our conclusions.
This bundle is comprised 21 remote data sources from a NASA GrADS-DODS Server.
As you know, these data are comprised of monthly, 3 hourly, and hourly data
spanning several decades. We count 2,202,131 time steps. While the IDV does not
load the entire data source, it does load the times for use in the Field
Selector. It is the loading of the times that accounts for the bulk of the
activity observed in the IDV both in terms of memory usage and time to load. On
my Macbook Pro, it takes about 2 minutes to load these times.
In addition, we have examined the memory usage numbers you reported from the
memory monitor. Without getting into too much detail, unfortunately, the 3
gigabyte number is deceptive because is comprises both "live" and "unreachable"
memory. The real memory usage to load these times is more around ~350
megabytes. (Indeed, you can load this bundle while only allocating 500
megabytes to the IDV). The rest is memory that was once used by the IDV, but is
now waiting to be reclaimed (garbage collected in computer science parlance).
In sum, we did not uncover any anomalous memory consumption problems here.
Over the course of this analysis, we did not uncover problems with the IDV.
This bundle is simply loading a lot of times from a remote source.
Please let us know if you are seeing performance issues with other bundles.
Best Wishes,
Unidata IDV Team
> Here's a little bundle that merely loads some URL data sources.
> No displays or views are created.
>
> They are big datasets - hourly for decades - so just indexing them seems
> to use a lot of memory (like 3G according to the memory counter).
> I'm not sure if that is really a problem, or just a virtual one.
> Don suggested I submit this for your profiling activity.
>
>
>
> Brian Mapes
> address@hidden or address@hidden
Ticket Details
===================
Ticket ID: XXB-855611
Department: Support IDV
Priority: Critical
Status: Closed