Skip to content

Fix module caching issue causing intermittent empty output (issue #16)#35

Merged
ronenlu merged 4 commits intomasterfrom
copilot/fix-5db8360a-d1f7-4113-a460-08123a2100d4
Aug 20, 2025
Merged

Fix module caching issue causing intermittent empty output (issue #16)#35
ronenlu merged 4 commits intomasterfrom
copilot/fix-5db8360a-d1f7-4113-a460-08123a2100d4

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Aug 20, 2025

Fixes #16 - resolves the intermittent issue where atlas-provider-sqlalchemy would sometimes output nothing, particularly after git branch switches.

Problem

The issue was caused by Python's module caching mechanism in the get_metadata() function. When importing SQLAlchemy model files using importlib.util.spec_from_file_location(), the function used non-unique module names (just file_path.stem), which could lead to stale cached modules being returned instead of fresh imports of changed files.

This manifested as:

  • Empty output on first run after git branch switches
  • Inconsistent behavior where second runs would work correctly
  • The workaround PYTHONDONTWRITEBYTECODE=1 fixing the issue (confirming bytecode caching was the culprit)

Solution

The fix implements unique module naming and cache invalidation:

  1. Unique module names: Instead of using file_path.stem (e.g., "models"), now uses f"atlas_dynamic_module_{abs(hash(str(file_path.absolute())))}_{file_mtime}" to ensure each import gets a unique module name based on file path and modification time.

  2. Cache invalidation: Explicitly clears stale modules from sys.modules and calls importlib.invalidate_caches() to force fresh loading.

  3. File change detection: Includes file modification time in the module name to detect when files have changed (e.g., after git branch switches).

Testing

Added comprehensive test suite (tests/test_caching_fix.py) that validates:

  • File changes are properly detected and reflected
  • Multiple files with same stem names don't interfere
  • File modification time tracking works correctly

All existing tests continue to pass, ensuring no regression in functionality.

Example

Before the fix, this scenario could fail:

git checkout feature-branch
atlas-provider-sqlalchemy --path models --dialect postgresql
# Sometimes outputs nothing

atlas-provider-sqlalchemy --path models --dialect postgresql  
# Second run would work

After the fix, both runs consistently produce the correct output.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] read this isuue: @ariga/atlas-provider-sqlalchemy/issues/16 and try to fix it Fix module caching issue causing intermittent empty output (issue #16) Aug 20, 2025
Copilot AI requested a review from ronenlu August 20, 2025 08:27
Co-authored-by: ronenlu <63970571+ronenlu@users.noreply.github.com>
@ronenlu ronenlu force-pushed the copilot/fix-5db8360a-d1f7-4113-a460-08123a2100d4 branch from 947431b to 9d0b395 Compare August 20, 2025 08:47
@ronenlu ronenlu marked this pull request as ready for review August 20, 2025 08:54
@ronenlu ronenlu requested a review from noamcattan August 20, 2025 08:59
@QuintenBruynseraede
Copy link
Copy Markdown

This fix seems to work 👍

ronenlu and others added 2 commits August 20, 2025 14:04
Co-authored-by: Noam Cattan <62568565+noamcattan@users.noreply.github.com>
@ronenlu ronenlu requested a review from noamcattan August 20, 2025 11:08
@ronenlu ronenlu merged commit 4b5904f into master Aug 20, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sometimes atlas-provider-sqlalchemy outputs nothing

4 participants