Re: Recovery Assistance (NOAAPort NWSTG Data - DSI 6431_03)

Margaret,

This data's different and thanks for looking!

On Fri, Nov 21, 2025 at 12:34 PM Margaret Curtis - NOAA Federal <margaret.curtis@noaa.gov> wrote:

Is this the same or different than the request to recover " TAC/GTS data from 08/26 - 08/29. We ingest the TAC/GTS from ftpprd.ncep.noaa.gov/pub/data2/NCDC/rawobs and files should look like 20250822.sonde."   from  Pichai Polprasert ?   @Jesse Marks - DF  has staged a bunch of data for them, we can look for this one too. 

Margaret

On Fri, Nov 21, 2025 at 7:05 AM Holly Uhlenhake - NOAA Federal <holly.uhlenhake@noaa.gov> wrote:

Morning Shane,

It's not always easy to know what datasets we need to keep, but we do have some data for you.  Dates I see on the supercomputer are from August 25th to Sept 8th. Hopefully that gets most of what you need.  I'll talk to both teams about how we normally get this data back to you.

Thanks,

Holly 

On Thu, Nov 20, 2025 at 10:50 PM Carissa Klemmer - NOAA Federal <carissa.l.klemmer@noaa.gov> wrote:

Casting a wider net to have the teams assist with this. 

IDP, HPC, can you together figure out what needs to be done to help get the data back. 

Carissa Klemmer
NCEP Central Operations
IDSB Branch Chief

301-683-3835

On Wed, Nov 19, 2025 at 9:59 AM Shane Taitt - NOAA Federal <shane.taitt@noaa.gov> wrote:

Good morning ma'ams!

Now that we've gotten NCEI's Common Ingest system stable and are able to invest more time in data recovery, I was wondering if it'd be possible to provide us with Subject data from 08/23-08/30.  I know this is a few months back but we're severely undermanned and have been spending the last three months trying to restore not only Common Ingest but other data stewardship systems and haven't had the time or resources to routinely monitor and steward data like we normally would.

Have a great day!

Shane

--

Shane Taitt

Data Scientist

NOAA's National Centers for Environmental Information (NCEI)

Data Stewardship Division | Data Operations Branch

shane.taitt@noaa.gov 

(828) 271-4335

Stonie,

For clarification, you do provide us with our NOAAPort NWSTG data via IDS|DDSPLUS but just can't re-provide/reproduce old data?

Shane

On Mon, Dec 22, 2025 at 7:39 PM NSF Unidata Local Data Manager (LDM) <support-ldm@unidata.ucar.edu> wrote:

Shane and Stonie,

I just wanted to jump in here to possibly provide a little bit of helpful information. The general flow of Canadian data as I understand it from its source -> NCEI works like this...

• Canada -> NCEP/NWSTG -> NCF -> SBN -> Unidata -> NCEI (via LDM)

My group (IDP Dataflow Team) is essentially the operators of the NWSTG who receives data from Canada and transmits data to the NCF where it is uploaded to the NWS SBN (Satellite Broadcast Network).

August 19th was an important date for the NCF/SBN as that was when the final major realignment of datasets and channels on the SBN took place, with my group making upstream dataflow changes as a result of the realignment plan. 

An NWS SCN describing the SBN changes was issued and is available here: https://www.weather.gov/media/notification/pdf_2025/scn25-59_channel_realignment_to_SBN_no3.pdf

As the above SCN indicates, one of the major changes planned was to move all US/Canadian/Caribbean radar data on the SBN to the EXP channel. After moving the Canadian and Caribbean radar products to the new SBN channel though, there was enough bandwidth improvement on the original NMC channel (which has highest priority) to keep US radar data there as described in this SCN update: https://www.weather.gov/media/notification/pdf_2025/scn25-59%20_channel_realignment_to_SBN_number_3_aaa. 

With the Canadian radar data moving to the SBNs' EXP channel, it is possible that change caused it to default to a different LDM feedtype (IDS|DDSPLUS) when the downstream Unidata SBN ingestor processes put the data into their LDM. This is where the exclude pattern that Stonie had provided you earlier in this thread would come in handy to have excluded the SDCN01 CWAO WMO-headed radar products from your data request. I'm not sure if you are able to exclude any SDCN01 products from the bundles of data NCEI already possesses in order to make them usable, but that would probably be the best solution here if you are able to do that. The Canada radar products would have been a major increase in size if you were only expecting smaller text products/observations in general when requesting feedtype IDS|DDSPLUS from Unidata's LDM.

Hope this info helps!

On Wed, Jan 7, 2026 at 4:25 PM NSF Unidata Local Data Manager (LDM) <support-ldm@unidata.ucar.edu> wrote:

--

Dustin Sheffler

NCEP Central Operations - Dataflow Team

Email: dustin.sheffler@noaa.gov

Google Voice: (301) 683-3827