Conversation
Sbozzolo
added
ci
Sbozzolo
force-pushed
the
gbozzola/cleanup_ghcr
branch
2 times, most recently
from
b64dfe7 to
3828528
Compare
Sbozzolo
force-pushed
the
gbozzola/cleanup_ghcr
branch
from
3828528 to
789f36c
Compare
cameronrutherford
left a comment
cameronrutherford
left a comment
There was a problem hiding this comment.
I think the first few comments I left are just naive. I would be curious to hear if you can modify the condition for deletion slightly.
| # We used `crane mutate --label` to add this to a tag in the package, which | ||
| # caused GHCR to auto-link the package to this repo. Once linked, the | ||
| # association persists at the GitHub level regardless of individual tag labels. |
There was a problem hiding this comment.
Maybe I am just naive but who is "we" here? Our CI jobs do this?
| # gh api orgs/awslabs/packages/container/palace-develop-testing \ | ||
| # --jq '.repository.full_name' | ||
| # | ||
| # Or as a GitHub Actions step: |
There was a problem hiding this comment.
I am interpreting this as "one needs to do this one-off operation on the index.spack in order to delete binaries in the future"?
Which begs the question - why don't we do this then in our GH Actions?
There was a problem hiding this comment.
Correct. This is a one-off operation, but it it not cheap (if you do it naively). I thought about adding to our actions but I found that processing all the packages we have in our binary cache would take several hours.
We could do something smarter, but I figured that we won't need to do this ever again (unless we change the name of the palace-develop-testing package), so I didn't bother
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 6,18 * * *' # Twice daily at 6:00 and 18:00 UTC |
There was a problem hiding this comment.
Maybe just once a week is fine? Curious how you decided on this cadence.
There was a problem hiding this comment.
The action can only remove up to 100 packages every time it runs.
| # Each Spack spec is a tagged version in the container package. We keep | ||
| # the most recent 500 versions (roughly 5-10 full build sets) and delete | ||
| # older ones. The action deletes at most 100 per run. |
There was a problem hiding this comment.
I would love to be a bit more scientific about this if possible.
I am reading this as it deletes anything past n=500.
That seems potentially too many deletions if we end up expanding the matrix of builds that we want to support. Ideally I would hope that one can just delete based on the age of an installation / the time since an installation was last used.
In other words, it would be ideal to cleanup based on a different criteria:
Any binary older than a week, that also hasn't been used in the last week
Sbozzolo
added
the
no-long-tests
2 participants
The GHCR is owned by the organization, so I needed to use
craneto link the build artifacts to the repo. Then, I could delete them. Now,palace-develop-testingis linked to the repo and can be cleaned.In this, I also fixed skipping the container workflow (because its status is required for merge)
Closes #524