Re: netCDF 3.2a speed on Cray IEEE T90

On Dec 4,  4:57pm, John Sheldon wrote:
> Hi again Glenn-
>
> Well, I finally managed to successfully merge versions 2.4.3 and 3.2a.
> One major holdup was Cray's "make" which sometimes refuses to update
> targets despite an undeniable need to do so - I wasted 2 days on
> that...
>
> Anyhow, I ran a couple of timing tests to check out the speed of 3.2.
> There's good news and bad news.  The good news is that prefill no
> longer takes inordinately longer than regular writes (in fact, it seems
> to take amlost no extra CP time, and the wall-clock time is about
> double, as expected).  The bad news is that regular writes take about
> 20x more CP time than did version 2.4.2.  I tried increasing
> NC_PG_CHUNK bay factors of 16 and 128, and made the change to NFILL as
> well.  Here is a "ja -clt" for 2.4.2 vs 3.2a/2.4.3 for the case where I
> "#define NC_PG_CHUNK 2097152":
>
>             Elapsed    User CPU   Sys CPU   I/O Wait I/O Wait    CPU MEM
>             Seconds     Seconds   Seconds   Sec Lck  Sec Unlck   Avg Mwds
>            ========= ========== ========== ======== ========== =========
>  2.4.2              35.1762     6.4618     1.3875   20.8732    6.4732
   2.4692
>  2.4.3/3.2a  161.1328   112.0403     4.2477   15.6226   29.3560    1.6070
>
> In both cases, I have setenv NETCDF_FFIOSPEC set to "cachea:512:4".
>
> Any suggestions?  I am going to try "#define NC_PG_CHUNK 67108864" next...

It is hard to tell what to suggest without profiling.

I asked NCAR's Cray site rep to try to get me an account on a T90, but
he hasn't responded. I think it is in our best interest to
jump through the hoops. Can you get things rolling to get me an account on

Thanks.

-glenn