Sync-gated content loading after join#109
Merged
ThomasHalwax merged 29 commits intomainfrom Mar 23, 2026
Merged
Conversation
Wire up E2EE in ODIN's replication layer:
1. package.json: matrix-client-api → github:syncpoint/matrix-client-api#feature/e2ee
2. ipc.js: safeStorage IPC handlers
- ipc:replication/encryptPassphrase → safeStorage.encryptString()
- ipc:replication/decryptPassphrase → safeStorage.decryptString()
- Uses OS keychain (DPAPI/Keychain/libsecret) for at-rest protection
3. preload/modules/replication.js: expose encrypt/decryptPassphrase to renderer
4. Project-services.js: passphrase lifecycle
- Reads crypto:enabled from session LevelDB
- If enabled: loads or generates passphrase, encrypts via safeStorage
- Passes encryption config to MatrixClient factory:
{ enabled: true, storeName: 'crypto-<uuid>', passphrase }
- MatrixClient handles CryptoManager init + persistent IndexedDB store
Secure by default: E2EE checkbox is checked by default when sharing.
User can opt out by unchecking 'Encrypt project data (recommended)'.
Components:
- ShareDialog.js: Modal with checkbox + explanatory text
- ProjectList.js: Share button opens dialog instead of sharing directly
doShare() passes { encrypted: true/false } to replication.share()
- ProjectList.css: Dialog overlay + styling
E2EE preference storage:
- ipc.js: new handler ipc:put:project:crypto/enabled writes to
project's session LevelDB
- replication.js (preload): setCryptoEnabled() exposed to renderer
- Project-services.js reads crypto:enabled on project open
Layer encryption inheritance:
- toolbar.js: shareLayer() inherits encrypted flag from project
(checks if cryptoManager is active on the replicatedProject)
Webpack's electron-renderer target activates the 'node' export condition, which loads node.mjs (uses fileURLToPath — incompatible with bundling). Set conditionNames to ['browser', 'import', 'default'] to force the browser/wasm entrypoint (index.mjs) which runs natively in Chromium.
conditionNames override broke CommonJS modules (jexl). Use a targeted alias to point @matrix-org/matrix-sdk-crypto-wasm directly at index.mjs (browser/wasm entrypoint).
Chromium's --user-data-dir flag changes the session context, which
breaks custom protocol handlers (app://) registered on the default
session. We now strip the flag from process.argv and use Electron's
app.setPath('userData') instead.
When Alice joins Bob's encrypted project, the crypto:enabled flag is now written to Alice's session store. Without this, Alice's ODIN instance would not initialize a CryptoManager, causing any layers she creates to be unencrypted.
Content is no longer loaded immediately after joinLayer() in the toolbar handler. Instead, the upstream handler's new selfJoined callback loads content when the sync stream confirms our own membership join — ensuring the server has processed the join before we request content. This fixes empty content on initial join, especially with federated servers or slow homeservers like Tuwunel.
The selfJoined stream approach doesn't work due to filter timing. Restore the direct content() call after joinLayer() — the HTTP join is synchronous so the messages endpoint should have content. Historical keys are received via to_device before the join, so decryption should work.
Content loading is back in toolbar.js. The selfJoined event approach didn't work and left dead code.
The join handler stored the role but never called store.restrict() for READER roles. This allowed local edits that the homeserver would reject, causing a gap between local ODIN state and server. Now applies rolesReducer after join, same as the hydrate path.
Features imported after join were not restricted even though the layer was marked as restricted. Now applies store.restrict() to all imported operation keys when the layer is restricted.
Extract importOperations() as shared function that handles both import and restriction check. Used by: - toolbar.js join handler (initial content load) - upstream.js received handler (stream events) Single code path, no duplication.
- Update dependency from GitHub branch to npm v2.0.0 - Project: persistent command queue using sublevel of project DB - ProjectList: in-memory command queue (no persistent operations) 327 unit tests passing, webpack build clean, eslint clean.
The B column cells were hardcoded empty — state.B1, state.B2, state.B3
were never rendered. This caused the replication offline feedback
('Looks like we are offline!') to be silently dropped since it
targets cell B2.
After joining an E2EE layer, historical keys arrive via the sync stream's to_device events in parallel. Without waiting, content() tries to decrypt before the keys are imported, resulting in 0 operations on first join. This is a temporary workaround (1s delay). A proper fix should use a dedicated sync cycle after join to ensure keys are available.
Remove the 1s setTimeout workaround and use the fix/decrypt-retry branch of matrix-client-api instead. The library now retries decryption up to 5 times (500ms intervals) when keys are not yet available, which handles the race condition properly.
- Remove content() call after joinLayer() in toolbar handler - Content now arrives via the received() stream handler after matrix-client-api detects the room in the next sync cycle - Permissions are still applied immediately after join (before content arrives) - Point to matrix-client-api feature/sync-gated-content branch
Includes sync-restart-on-join fixes: - Awaitable restartSync() to prevent race condition - Backward pagination with prev_batch for federation backfill - Content fetch retry across sync cycles for key delivery - Increased retry window (20x10s) for slow federation servers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the previous immediate
content()call afterjoinLayer()with a sync-gated approach that reliably handles federation lag and E2EE key delivery timing.Problem
When joining a layer hosted on a federated server (e.g. matrix.org ↔ syncpoint.io), the content fetch after join would often return empty because:
dir=fwithout a token instead ofdir=bwithprev_batchSolution
matrix-client-api v2.2.0
restartSync()returns a Promise that resolves when the new filter is appliedcontent()accepts afrompagination token for backward paginationsyncTimeline()exposesprevBatchesfrom sync responsesODIN (this PR)
joinLayer()awaitsrestartSync()before callingjoin()pendingContentMap)prev_batchtoken from sync for backward paginationAlso included
Testing