[python #NAQ-102319]: How to pull GLM data through Siphon without a fixed URL

Greetings!

So the next release of siphon (overdue at this point) will have the ability to 
use a custom regex and string parsing format, so this will do what you want:

    from siphon.catalog import TDSCatalog
    from datetime import datetime, timedelta

    cat = 
TDSCatalog('https://thredds.ucar.edu/thredds/catalog/satellite/goes/east/products/GeostationaryLightningMapper/CONUS/current/catalog.xml')
    dt = datetime(2020, 8, 2, 16, 45)
    cat.datasets.filter_time_nearest(dt, regex='_s(?P<strptime>\d{13})', 
strptime='%Y%j%H%M%S')

You can get that today if you install siphon from master with:

  pip install git+git://github.com/Unidata/siphon@master#egg=siphon

Alternatively, you can loop over all the datasets in the catalog and filter 
just like the function above does:

    import re

    regex = re.compile('_s(?P<strptime>\d{13})')
    min_diff = float('inf')
    closest_ds = ''
    for ds in cat.datasets:
        match = regex.search(ds)
        if match:
            diff = datetime.strptime(match.groupdict()['strptime'], 
'%Y%j%H%M%S') - dt
            if abs(diff.total_seconds()) < min_diff:
                min_diff = abs(diff.total_seconds())
                closest_ds = ds

Hope this helps,

Ryan

> Hi UCAR folks,
> 
> I'm trying to pull GLM data from this THREDDS link:
> https://thredds.ucar.edu/thredds/catalog/satellite/goes/east/products/GeostationaryLightningMapper/CONUS/20200719/catalog.html
> .
> 
> I usually just code up a url string and feed it into TDSCatalog(url), but
> the urls for the GLM are not predictable. In the last part of the string
> (starting with "c"), the seconds change each time. (It seems like on this
> particular day, the seconds vary between about 32 and 39.)
> 
> Is there a good way to pull the desired file by just the start "s" and end
> "e" minute and second, sort of treating the last part of the string like a
> wildcard?
> 
> I've attached a snippet of what I'm trying so far using a list of possible
> urls and a try/except, but 1) I don't think I'm using try & excepts
> correctly and 2) even if I am, there has to be a better way than this,
> right?
> 
> Thanks!
> Ryan
> 
> --
> Ryan Connelly
> Meteorologist
> NWS WFO Marquette, MI
> 
> 


Ticket Details
===================
Ticket ID: NAQ-102319
Department: Support Python
Priority: Low
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.