[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[python #XMR-920755]: Weather forecast made in past

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [python #XMR-920755]: Weather forecast made in past
  • Date: Mon, 14 Jun 2021 14:59:32 -0600
Greetings!

Since GRIB is a message-based file format, you should be able to directly 
concatenate your desired files together and then open them. This should work on 
a unix-like system as:

    cat *.grb > concatenated_dataset.grb

You should then be able to open that with pygrib or xarray + cfgrib. As far 
converting to CSV, you may be able to do this by opening the GRIB file using 
xarray and cfgrib, converting to a Pandas dataframe, and then writing the 
dataframe to CSV. Something like this *might* work:

    import xarray as xr
    ds = xr.open_dataset('concatenated_dataset.grb', engine='cfgrib')
    df = xr.to_dataframe()
    # MAKE SURE YOU SUBSET HERE
    df.to_csv('data.csv')

It's important to subset your data before writing to CSV, or you'll end up with 
a CSV file that is roughly 10x the size of the original GRIB file. If xarray + 
pandas isn't something you can use, you'll need to either use Python's built-in 
`csv` module, or manually open a text file for writing and construct the CSV 
manually like:

   with open('data.csv', 'wt') as outfile:
      for row in subsetted_data:
        output_line = #FORMAT DATA HERE
        outfile.write(output_line + '\n')

Hope this helps,

Ryan

> Hi,
> Thank you for the reply. I got what i need from 
> https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs.
>  It gives a tar file containing many timestamped grib files. I tried opening 
> those with xygrib but it takes only 1 grib file at a time. Do you know a way 
> to open multiple grib files and store the temperature and precipitation data 
> in csv .
> 


Ticket Details
===================
Ticket ID: XMR-920755
Department: Support Python
Priority: Low
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.