Skip to content

Conversation

@wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Jan 7, 2026

This partially addresses 1409.

This is taking the majority of the pkg/ package from anchore/grype-db and migrating it to here. This is mainly a lift-and-shift of the build and package aspects of grype-db (pull will remain in grype-db).

What's being added to grype

  • Transformers/Unmarshalers/Processors for all the vulnerability data sources we ingest (NVD, OSV, OS advisories, GitHub, MSRC, OpenVEX, etc.)
  • DB writers and build infrastructure
  • Codename lookup generation (the debian/ubuntu version-to-codename mapping) + weekly workflow to keep codename data up to date via automated PRs.

A similar question, what is NOT being added to grype:

  • anything related to running vunnel or the pull process in grype-db
  • DB acceptance tests, these will remain in grype-db for the meantime

Why do this?

  1. The grype-db lib has a lot of code that is responsible for taking data and writing it to the DB. Today the DB schema and surrounding code lives in the grype repo. This can lead to odd situations where you've made a change to the DB code in grype and released it but it doesn't function practically in the grype-db repo, what is responsible for making production DBs. This is the first step in moving all of the logic needed to write to a DB in one spot, so that we can catch these kinds of issues as early as possible.

  2. Unit/Integration tests could benefit from being able to build realistic DBs from test fixture data in the same manner that we build DBs in production. Today we utilize a mocking layer, which is very cumbersome and not providing a lot of value. By cumbersome, I mean that it takes many lines of code to express a test + test-fixture (which needs to be manually written). By "not providing a lot of value" I mean that it is easy to write a test that will pass but does not reflect what will happen in production (thus is not a good safety net to continue to develop against).

  3. Simplifies development workflow. Instead of needing to implement logic across two repos, it's implemented in one (grype). Grype-db still needs to pull in the changes, but only via a go-module bump, not with additional code/test review.

Questions

  • Will we migrate the remaining grype-db functionality (commands) into grype?... Maybe. There are good reasons for and against this. The most minimal/impactful action is to get the underlying DB writing functionality migrated to grype in order to improve our test posture.

  • Does this refactor tests with a new DB build infrastructure? Not yet, that will be a follow up PR(s).

  • Does this internalize the new API? Not yet, only a few areas have been put into internal packages. This will be a focus of the grype 1.0 work as a whole (more things will be put into internal over time).

  • Will this new code be removed from grype-db? Yes.

Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
@wagoodman wagoodman requested a review from a team January 7, 2026 16:40
Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -0,0 +1,37 @@
package tarutil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unfortunate this became duplicated code. Note: populate is no longer used and could be removed from grype-db, I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants