Skip to content

Convert store and param registry to sqlite#114

Draft
WarmCyan wants to merge 10 commits intomainfrom
sqlite
Draft

Convert store and param registry to sqlite#114
WarmCyan wants to merge 10 commits intomainfrom
sqlite

Conversation

@WarmCyan
Copy link
Collaborator

This has previously been an issue because large stores take a second to write out new information - if the experiment gets ctrl-C'd before this completes, the store corrupts and you have to delete it (resetting all run numbers to zero)

Closes #25

@WarmCyan
Copy link
Collaborator Author

WarmCyan commented Nov 9, 2023

Consider if using duckdb is a better alternative to sqlalchemy - I don't love sqlalchemy's API so far and I think duckdb is a little more intuitive. Weight pros and cons.

This makes it a lot easier to interface with the sql database since we
can use dataclasses' asdict, and have a function to sanitize if needed.

Also this partially begins to address #65
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use sqlite for experiment store instead of json

1 participant