Skip to content

Conversation

@ercsnmrs
Copy link
Contributor

@ercsnmrs ercsnmrs commented Aug 31, 2025

Key Features

  • Optional: Encryption can be disabled if not needed
  • Transparent: No changes to client API required
  • Secure: Uses AES-256-GCM with random nonces
  • Performance Optimized: Only encrypts message content field
  • Error Handling: Graceful fallback if encryption fails

Security Considerations:

  • Message content only: Metadata remains unencrypted for search/indexing
  • Database admins: Can still see message metadata but not content
  • Key management: Critical - keys must be stored securely
  • Performance: Minimal overhead (~1-5ms per message)
  • Recovery: Lost keys mean lost message content

[#967]

@ercsnmrs
Copy link
Contributor Author

I will post the test results + load test.

@or-else
Copy link
Contributor

or-else commented Aug 31, 2025

Do you want me to review this now or later, after the tests?

@ercsnmrs
Copy link
Contributor Author

@or-else please do after my testing.

@ercsnmrs
Copy link
Contributor Author

@or-else thinking about it, I would appreciate early feedbacks, we can discuss more technical details afterwards.


// Encrypt message content if encryption is enabled
if encryptionService != nil && encryptionService.IsEnabled() {
encryptedContent, err := encryptionService.EncryptContent(msg.Content)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if encryptedContent, err := encryptionService.EncryptContent(msg.Content); err != nil { ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, not really.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not addressed. The encryptionService.IsEnabled() check is redundant.

return adp.MessageGetAll(topic, forUser, opt)
messages, err := adp.MessageGetAll(topic, forUser, opt)
if err != nil {
return nil, err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return messages, err rather than nil, err. No need to change the logic here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not addressed.

mielen636-ui
mielen636-ui previously approved these changes Sep 5, 2025
@or-else or-else changed the base branch from master to devel September 5, 2025 10:35
@or-else or-else dismissed mielen636-ui’s stale review September 5, 2025 10:35

Please don't do this.

@or-else
Copy link
Contributor

or-else commented Sep 15, 2025

Any update on this?

@or-else
Copy link
Contributor

or-else commented Sep 19, 2025

Closing as abandoned

@or-else or-else closed this Sep 19, 2025
@ercsnmrs
Copy link
Contributor Author

@or-else let me open another mr on this one, been a though week. thanks for your input.

@or-else or-else reopened this Sep 19, 2025
@or-else
Copy link
Contributor

or-else commented Sep 19, 2025

If you want to continue, please use this PR. I reopened it. Thanks.

@ercsnmrs
Copy link
Contributor Author

Appreciate it thanks!

@ercsnmrs ercsnmrs force-pushed the feature/encrypt-message-at-rest branch 2 times, most recently from 5d3ce78 to 157476a Compare September 20, 2025 05:01
@or-else
Copy link
Contributor

or-else commented Sep 20, 2025

Could you please respond to each of my comments with done (if resolved) or not done and an explanation why you think it's not a good idea. This way I would know what to look for in the next code review. Thanks!

@ercsnmrs
Copy link
Contributor Author

@or-else sure, currently testing the changes. thanks!

@ercsnmrs ercsnmrs force-pushed the feature/encrypt-message-at-rest branch from 157476a to 4ae0494 Compare September 20, 2025 07:25
@ercsnmrs
Copy link
Contributor Author

Testing Results:

Steps:

  1. Generated and added the encrypted_at_rest config on tinode.conf file
Screenshot 2025-09-20 at 15 30 48
  1. Send Message 1 - Testing User
Screenshot 2025-09-20 at 15 19 11 Screenshot 2025-09-20 at 15 19 11

Result: Encrypted = true

  1. Send Message 2 - second encrypted message
Screenshot 2025-09-20 at 15 20 06

Result: Encrypted = true

  1. Revert Config from Step 1
Screenshot 2025-09-20 at 15 23 44 Screenshot 2025-09-20 at 15 23 27

Result: Encrypted = false

Some observation:

  • I need to refresh the webapp to display the chat
  • Needs thorough testing

CC: @or-else

@ercsnmrs ercsnmrs marked this pull request as ready for review September 20, 2025 07:46
keygen/README.md Outdated
./keygen -encryption -keysize 16

# Generate 24-byte encryption key (AES-192)
./keygen -encryption -keysize 24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example is probably unnecessary. Any reasonable person can guess that specifying -keysize 24 would mean a 24-byte key.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not addressed

}

// initMessageEncryptionFromConfig initializes message encryption service from config file
func initMessageEncryptionFromConfig(jsonconf json.RawMessage) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid code duplication. I'm strongly against copy-pasting code. Please merge this method with initMessageEncryption.

@or-else
Copy link
Contributor

or-else commented Sep 20, 2025

Please move server/tools/encrypt_messages.go to another PR. It will require a lot of work. It can be done later.

@ercsnmrs ercsnmrs marked this pull request as draft September 21, 2025 03:57
@ercsnmrs
Copy link
Contributor Author

@or-else i will continue on this, will push a change tomorrow oct 10. please keep it open. Thanks!

@or-else
Copy link
Contributor

or-else commented Nov 3, 2025

Any update on this?

@or-else
Copy link
Contributor

or-else commented Dec 21, 2025

@ercsnmrs Please let me know if you wish to continue with this PR.

@ercsnmrs
Copy link
Contributor Author

ercsnmrs commented Jan 8, 2026

Had a long break. I will continue working on it @or-else thanks for the wait :D

@ercsnmrs ercsnmrs force-pushed the feature/encrypt-message-at-rest branch 2 times, most recently from 348f806 to 1a92f1a Compare January 9, 2026 14:08
… validation

- Rename generic 'encryption' config to 'encrypt_at_rest' for clarity
- Remove redundant 'enabled' field - key presence determines encryption
- Support all AES key sizes (16, 24, 32 bytes) instead of just 32-byte keys
- Simplify EncryptionService to MessageEncryptionService with cleaner API
- Use []byte fields in EncryptedContent for automatic base64 conversion
- Fix store initialization order: command line flags override config file
- Update keygen tool with proper AES key size validation
- Remove output file option from keygen (use shell redirection instead)
- Fix encrypt_messages tool to use proper store interface methods
- Add nil content handling in EncryptContent method
- Update all tests to work with new MessageEncryptionService API
- Improve error handling and method visibility throughout

[tinode#967]
@ercsnmrs ercsnmrs force-pushed the feature/encrypt-message-at-rest branch from 1a92f1a to 2a87eff Compare January 9, 2026 14:15
- Detect encrypted content by presence of data and nonce fields
- Message, replace with error placeholder instead of returning garbled encrypted content

[tinode#967]
@ercsnmrs
Copy link
Contributor Author

@or-else already addressed your concern on the MR.

i have concern regarding the front-end, what is your expected here?

Screenshot 2026-01-10 at 15 17 12

E.g. admin put encryption then decide to go back.

@ercsnmrs ercsnmrs marked this pull request as ready for review January 10, 2026 07:19
@or-else
Copy link
Contributor

or-else commented Jan 10, 2026

already addressed your concern on the MR.

I don't understand. What's MR?

@ercsnmrs
Copy link
Contributor Author

ercsnmrs commented Jan 10, 2026

@or-else sorry MR is merge request, same meaning of pull request

@or-else
Copy link
Contributor

or-else commented Jan 10, 2026

what is your expected here?

The use case:

  1. The encryption is turned on, a few messages are exchanged and stored encrypted on the server.
  2. The encryption is turned off without decrypting already encrypted messages (tools are not ready).
  3. New messages are unencrypted.

Expectation:

  1. The encryption is handled by the server in such a way that the client is completely unaware of it. The messages appear completely normally in the client.
  2. The encrypted messages become unaccessible. The client should see something like "image not accessible" or "unavailable", not "invalid content".
  3. New messages appear completely normally, like nothing happened.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements optional message encryption at rest for Tinode, encrypting message content stored in the database using AES-GCM symmetric encryption. The feature is transparent to clients, with messages automatically encrypted on save and decrypted on retrieval.

Changes:

  • Added AES-GCM encryption/decryption service for message content
  • Integrated encryption into message save and retrieve operations
  • Added keygen utility support for generating encryption keys
  • Provided comprehensive documentation and test coverage

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
server/store/message_encryption.go New encryption service implementing AES-GCM encryption/decryption for message content
server/store/encryption_test.go Comprehensive test coverage for encryption functionality
server/store/store.go Integration of encryption service into message persistence layer with graceful error handling
server/store/ENCRYPTION.md Documentation covering configuration, key management, migration, and security considerations
server/tinode.conf Added configuration options for encryption at rest with examples (includes unrelated DB config changes)
tinode-db/tinode.conf Added configuration options for encryption at rest (includes unrelated DB config changes)
keygen/keygen.go Extended keygen utility to generate encryption keys with validation
keygen/README.md Documentation for encryption key generation
server/main.go Minor cosmetic change (blank line)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

// "encrypt_at_rest": {
// "key": "base64-encoded-32-byte-key-here"
// },
"use_adapter": "postgres",
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use_adapter has been changed from empty string to "postgres". This appears to be a development-specific change that should not be in a configuration example file. The default should remain as an empty string (or another placeholder value) so users can choose their own adapter.

Suggested change
"use_adapter": "postgres",
"use_adapter": "",

Copilot uses AI. Check for mistakes.
Comment on lines +19 to +22
// EncryptedContent represents encrypted message content with metadata
type EncryptedContent struct {
Data []byte `json:"data"` // Encrypted data (automatically base64 encoded/decoded)
Nonce []byte `json:"nonce"` // Nonce (automatically base64 encoded/decoded)
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments on the EncryptedContent struct claim that Data and Nonce fields are "automatically base64 encoded/decoded", but the struct doesn't implement custom JSON marshaling/unmarshaling. Go's default JSON encoding for []byte fields uses base64 encoding, so this is technically correct, but the comment might be misleading as it suggests custom handling when it's actually Go's default behavior.

Suggested change
// EncryptedContent represents encrypted message content with metadata
type EncryptedContent struct {
Data []byte `json:"data"` // Encrypted data (automatically base64 encoded/decoded)
Nonce []byte `json:"nonce"` // Nonce (automatically base64 encoded/decoded)
// EncryptedContent represents encrypted message content with metadata.
// The Data and Nonce fields are []byte and will be base64 encoded/decoded
// by Go's standard encoding/json package when marshaling/unmarshaling.
type EncryptedContent struct {
Data []byte `json:"data"` // Encrypted data (JSON-encoded as base64)
Nonce []byte `json:"nonce"` // Nonce (JSON-encoded as base64)

Copilot uses AI. Check for mistakes.
Comment on lines +169 to +170


Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Troubleshooting section is empty. This section should include common issues and their solutions, such as: 1) What to do if encryption key is lost, 2) How to handle decryption errors, 3) What to do if messages appear as "[Message content could not be decrypted]", 4) Performance troubleshooting, etc.

Suggested change
### Lost or Rotated Encryption Key
The encryption key is required to decrypt any message content that was stored while that key was active.
If the key is lost or changed without a proper migration, previously stored messages **cannot** be decrypted.
Recommended practices:
- **Back up the key securely** before enabling `encrypt_at_rest` in production.
- Store the key in a secure secret manager or hardware security module (HSM) where possible.
- Restrict access to the key to only the server processes that need it.
If the key is already lost:
- Existing encrypted messages will remain in the database but their content is unrecoverable.
- You may configure a new key in `tinode.conf` and restart the server; all **new** messages will use the new key.
- If you have a database backup and a backup of the original key, you can:
- Restore the database snapshot that corresponds to the time when the old key was still available.
- Restart the server with the old key to access the data, or
- Implement a one‑time export/re‑encrypt procedure outside of this document.
### Decryption Errors
If the server logs decryption errors for message content:
1. **Verify the key format**:
- Ensure the `encrypt_at_rest.key` value is valid base64.
- After base64 decoding, the key must be exactly 16, 24, or 32 bytes for AES‑128/192/256.
2. **Check for configuration drift**:
- Confirm all server instances use the **same** encryption key.
- If you changed the key, verify that all nodes were restarted and are reading the updated configuration.
3. **Inspect for data corruption**:
- Validate that the stored JSON object contains both `data` and `nonce` fields.
- Ensure both fields contain valid base64 without truncation or manual editing.
4. **Review server logs**:
- Look for details such as "authentication failed", "invalid nonce size", or "ciphertext too short".
- These messages can help distinguish between key mismatch, corrupted data, and programming errors.
If decryption consistently fails for all messages after a deployment or configuration change, roll back to the previous configuration (and key) that was known to work, then re‑apply changes carefully.
### Messages Displayed as `[Message content could not be decrypted]`
When clients see a placeholder such as `[Message content could not be decrypted]`, it usually means:
- The server failed to decrypt the stored `content` field, or
- The content field is missing or corrupted in the database.
Steps to investigate:
1. Identify an affected message and locate it in the database.
2. Confirm that the `content` JSON has both `data` and `nonce` fields and that they are valid base64.
3. Check that the server is running with the same encryption key that was used when the message was stored.
4. Review server logs around the time the message was fetched for any decryption errors.
If only some messages show this placeholder while newer ones are fine, this typically indicates:
- A key change without migrating old data, or
- Partial database corruption or manual edits to stored messages.
In that case, only the affected messages are unrecoverable; new messages should decrypt normally once the key and configuration are correct.
### Performance Troubleshooting
Encryption at rest adds CPU overhead and increases stored message size. If you observe high latency or load:
- **Measure baseline performance**:
- Compare message send/receive latency with `encrypt_at_rest` enabled vs disabled in a non‑production environment.
- **Monitor resource usage**:
- Check CPU utilization on the Tinode server nodes during peak traffic.
- Ensure there are enough CPU cores for the expected encryption/decryption workload.
- **Review message sizes**:
- Very large message payloads incur more encryption cost and storage overhead.
- Consider enforcing reasonable limits on message size.
- **Scale horizontally**:
- If CPU is the bottleneck, add more server instances and ensure they all share the same encryption key.
- **Check configuration**:
- Verify that the key length is appropriate; AES‑256 may be slightly slower than AES‑128 but usually not dramatically so.
When investigating performance issues, always change one variable at a time (e.g., disable encryption in a staging environment only) and compare metrics to isolate the impact of encryption.

Copilot uses AI. Check for mistakes.
Comment on lines +14 to +15
"database": "tinodesvc",
"dsn": "postgresql://user:pass@localhost:5432/tinodesvc?sslmode=disable"
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The database configuration has been changed from default values. This change appears to be specific to a development environment and should not be committed. The database name changed from "tinode" to "tinodesvc", and credentials from "postgres:postgres" to "user:pass". These should remain as the default examples unless there's a specific reason to change them.

Suggested change
"database": "tinodesvc",
"dsn": "postgresql://user:pass@localhost:5432/tinodesvc?sslmode=disable"
"database": "tinode",
"dsn": "postgresql://postgres:postgres@localhost:5432/tinode?sslmode=disable"

Copilot uses AI. Check for mistakes.
Comment on lines +270 to +274
"User": "user",
"Passwd": "pass",
"Host": "localhost",
"Port": "5432",
"DBName": "tinode",
"DBName": "tinodesvc",
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The database configuration has been changed from default values. This change appears to be specific to a development environment and should not be committed. The database name changed from "tinode" to "tinodesvc", username from "postgres" to "user", and password from "postgres" to "pass". These should remain as the default examples unless there's a specific reason to change them.

Copilot uses AI. Check for mistakes.
// See server/store/ENCRYPTION.md for more information.
// "encrypt_at_rest": {
// "key": "base64-encoded-key-here"
// },
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a trailing comma after the closing brace. While this is commented out, if users uncomment these lines to enable encryption, the trailing comma will cause a JSON parsing error. The comma should be removed or a comment should be added to warn users to remove it.

Suggested change
// },
// }

Copilot uses AI. Check for mistakes.
Comment on lines +150 to +158
// GetKey returns a copy of the encryption key (for testing purposes)
func (es *MessageEncryptionService) GetKey() []byte {
if es == nil || es.key == nil {
return nil
}
keyCopy := make([]byte, len(es.key))
copy(keyCopy, es.key)
return keyCopy
}
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GetKey method is exported and allows retrieving a copy of the encryption key. While the comment indicates it's for testing purposes, being exported makes it accessible in production code, which is a security risk. This method should either be removed if not needed, or made unexported (lowercase 'getKey') and kept in a test file if needed for testing.

Copilot uses AI. Check for mistakes.
Comment on lines +160 to +161
// GetMessageEncryptionService returns the current message encryption service instance
func GetMessageEncryptionService() *MessageEncryptionService {
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GetMessageEncryptionService function is exported but not used anywhere in the codebase. Exposing the encryption service instance could allow external code to access encryption methods or the key (via GetKey), which is a security risk. This function should either be removed if not needed, or made unexported (lowercase name) if it's only needed internally.

Suggested change
// GetMessageEncryptionService returns the current message encryption service instance
func GetMessageEncryptionService() *MessageEncryptionService {
// getMessageEncryptionService returns the current message encryption service instance.
// It is kept unexported to avoid exposing the encryption service outside this package.
func getMessageEncryptionService() *MessageEncryptionService {

Copilot uses AI. Check for mistakes.
// DB adapter name to communicate with the DB backend.
// Must be one of the adapters from the list below.
"use_adapter": "",
"use_adapter": "postgres",
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use_adapter has been changed from empty string to "postgres". This appears to be a development-specific change that should not be in a configuration example file. The default should remain as an empty string (or another placeholder value) so users can choose their own adapter.

Suggested change
"use_adapter": "postgres",
"use_adapter": "",

Copilot uses AI. Check for mistakes.
@ercsnmrs
Copy link
Contributor Author

Cool. Thanks for the feedback @or-else let me address that use case for this point:

  1. The encrypted messages become unaccessible. The client should see something like "image not accessible" or "unavailable", not "invalid content".

I agree that the server should return other message instead.

@or-else
Copy link
Contributor

or-else commented Jan 10, 2026

I agree that the server should return other message instead.

It should not return the message with such text. It should return an indicator that the message cannot be decrypted, probably in the message header. Including such text into message body is undesirable because it has to be i18n-ed and that's easier if it's just a header.

@ercsnmrs ercsnmrs marked this pull request as draft January 10, 2026 12:35
@or-else
Copy link
Contributor

or-else commented Jan 10, 2026

Please address every unresolved comment. If you agree and it's resolved, say "done", if you disagree, explain why. Please do address every unresolved issue with either "done" or an explanation why it's a bad idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants