Reading the winget Source Index From source2.msix and Its SQLite Package Table

Windows Package Manager publishes its package index as a downloadable artifact. Reading that artifact directly is more reliable than shelling out to winget search when you need name-to-ID resolution at server scale. This post documents the artifact's location and format, the SQLite table that matters, and the name normalization needed to match real-world display names against it. It also covers the refresh workflow ET Ducky uses to keep a local copy current. Everything here was learned building the software-catalog feature that maps installed-application inventory to winget package IDs.

Where the index lives

The winget client itself fetches its source index from a public CDN. Two artifacts matter:

https://cdn.winget.microsoft.com/cache/source2.msix
https://cdn.winget.microsoft.com/cache/source.msix

source2.msix is the current-generation index. source.msix is the older artifact that remains published. A robust fetcher requests source2.msix first and falls back to the older artifact. The ET Ducky harvester does that, with a plain User-Agent and a long timeout, because the artifact is tens of megabytes and the CDN occasionally serves slowly. No authentication is required. This is the same download every winget client performs.

What is inside the msix

An .msix is a ZIP archive. Inside it, alongside the package manifest and signature files, is a SQLite database (the file ends in .db; locating it by extension rather than by hard-coded path survives layout changes between index revisions). That database is the entire searchable index the winget client uses.

The table that matters for name resolution is packages. The columns worth reading:

SELECT id, name, moniker, latest_version FROM packages

• id is the winget package identifier (Publisher.Product form, e.g. Mozilla.Firefox).
• name is the display name (Firefox).
• moniker is the short alias where one exists (firefox). Some revisions store it as the literal string "None" instead of NULL, which a reader must normalize.
• latest_version is the current version string in the index.

Earlier index generations normalized data across more tables, using separate name, ID, and moniker tables joined by rowid maps. The flat packages table in the current index makes a single SELECT sufficient. If you target both artifacts, verify the table layout per file rather than assuming.

Matching inventory display names to index entries

The reason to read the index at all is matching. You have display names from installed-software inventory ("Notepad++ (64-bit x64)", "Microsoft Visual C++ 2015-2022 Redistributable (x64)") and you want winget IDs. Exact string equality fails constantly. Inventory names carry architecture suffixes, registered trademarks, version fragments, and inconsistent spacing that the index's display names do not.

The approach that works is key folding. Strip every character that is not a lowercase letter or digit, and match on the folded key.

NORM = re.compile(r'[^a-z0-9]+')
def norm(s):
    return NORM.sub('', (s or '').lower())

Folding both sides brings "Notepad++" and "notepadplusplus"-adjacent variants together and makes punctuation, casing, and spacing differences irrelevant. The harvester computes folded keys for both name and moniker at ingest time and stores them as indexed columns. Lookups at match time are then a single indexed equality rather than a scan with normalization on the fly. Folded-key collisions exist, for example two products whose names differ only in punctuation. They are rare enough in practice to resolve by preferring the entry whose unfolded name is closest.

The refresh workflow

The index changes as packages are added and updated, so a local copy needs scheduled refresh. The ET Ducky harvester runs weekly from Task Scheduler and is safe to run at any time. It downloads the msix to a temp directory, extracts and reads the SQLite packages table, and computes the folded keys. It then atomically replaces the contents of the serving table (Postgres, in our case) in one transaction. Atomic replacement rather than incremental merge keeps the logic small and makes a partially-failed refresh impossible. Either the new snapshot lands complete or the old one keeps serving.

The packages table carries thousands of rows and is small once extracted. The cost of a refresh is the msix download rather than the data handling. There is no delta mechanism. Every refresh is a full snapshot, which is why weekly is a reasonable cadence and hourly would be wasteful.

What this does not cover

The index maps names to IDs and latest versions. It does not carry installer URLs, hashes, or switches. Those live in the per-package manifests in the winget-pkgs repository, keyed by the ID resolved from the index, so install detail requires a second lookup. Packages outside the community repository (private sources, vendor-direct installers) are absent by definition. In ET Ducky, apps that no package manager tracks are held in a separate per-organization software catalog with vendor-script installs, and the winget index answers only the "is this managed, and by what ID" question. The msix signature covers the artifact you downloaded. If your threat model includes the CDN path, verify the msix signature after download rather than trusting transport alone.

Summary

Download source2.msix (fall back to source.msix), unzip, find the .db, read id, name, moniker, latest_version from packages, normalize "None" monikers to NULL, fold match keys to [a-z0-9], and replace your serving table atomically on a weekly schedule. The harvester is under a hundred lines of Python. It avoids both the fragility of parsing CLI output and the per-query latency of shelling out to winget on demand.