GMime 4.0 Plan

[jstedfast]

GMime 4.0 ideas:

• Add padding to all object and class structs to allow for future expansion
• Rename g_mime_object_encode() to g_mime_object_prepare() for API consistency with MimeKit?
• Add a GMimeVisitor class and GMimeObjectClass::accept virtual method? If we do, we'll also want:
  • Add GMimeMultipartAlternative
  • Add GMimeMultipartRelated
  • Add GMimeMultipartReport
  • Add GMimeMessageDeliveryStatus
  • Add GMimeMessageDispositionNotification
  • Add GMimeTextRfc822Headers
• Add g_mime_message_get_text_body() and g_mime_message_get_html_body() like MimeKit has? These would replace g_mime_message_get_body()
• Update GMimeSignature and GMimeCertificate to use a gint64 (internally at most) or GDateTime (public API for sure and maybe internally as well?) for the timestamps. (issue GMimeSignature created and expires are time_t, fail in Y2038 on platforms with 32-bit signed time_t #68 and issue in GMime 4.0, timestamps should use some sort of robust/native GLib representations #73)
• Merge GMimeDecryptFlags and GMimeVerifyFlags (issue in GMime 4.0, consolidate GMimeVerifyFlags and GMimeDecryptFlags as GMimeCryptoFlags #70)?
• Add support for application/pkcs7-mime; smime-type=compressed-data (issue handle PKCS#7 CompressedData #66)
• Add support for application/pkcs7-mime; smime-type=authEnveloped-data (issue handle PKCS#7 authEnveloped-data #65)
• Can we get rid of g_mime_stream_close()? Disposing of the stream can (and already does to an extent) take the place of close(). This might eliminate confusion of reading/writing to a closed stream.
• Should g_mime_crypto_context_sign() take an InternetAddressMailbox instead of const char *userid? Same idea for g_mime_crypto_context_encrypt() for both the userid and recipients arguments.
• Should g_mime_object_write_to_stream() be fixed to return gint64?
• Should GMimeStream's read/write functions take GError and/or GCancellation arguments? Same for GMimeParser APIs? And g_mime_object_write_to_stream(), etc.

[jstedfast]

The more I think about it, the more I think I'm probably going to pass on adding a GMimeVisitor API.

I'm probably also going to pass on the GError and GCancellation stuff as well.

[albrechtd]

Disclaimer: the remarks below come from the requirements for Balsa as well as for an other project (non-FOSS) I'm working on for my current employer (a German Government agency) which has somewhat different requirements as it is not a MUA.

Basically, GMime 3.2.4 (the latest for Debian Bullseye) provides almost everything I need – the exception actually is crypto support. From my POV, it would be ideal if GMime would simply provide access to the excellent GpgME data structures. I understand that you want to hide all the complexity of GpgME behind your own interfaces for a painless implementation of trivial use-cases. However, as this approach also completely hides the full flexibility and power of GpgME, the GMime crypto interface is not usable for more complex tasks like mine.

I presume you want to keep your stripped-down crypto interface in ver 4.0 as to support other projects. In order to make is usable for more demanding use cases, a solution could be adding methods to access to the underlying GpgME objects, plus several callbacks and/or signals. If you want to avoid a mandatory dependency of GMime headers on gpgme.h, you could wrap the “advanced” interfaces into #ifdef GPGME_VERSION, i.e. they are visible only if gpgme.h is included before.

1. GpgME context manipulation

GMime provides no access to the properties of the underlying GpgME contexts and is thus unusable for me. It must be possible to configure the crypto engine, including the working folder, flags, etc., plus some more nasty changes of the engine. A method for accessing the GpgME context, e.g. gpgme_ctx_t g_mime_crypto_context_gpgme_ctx(GMimeCryptoContext *ctx), should be sufficient. If a context is created automatically (e.g. for g_mime_multipart_signed_verify() or similar) a callback must be called (or a signal fired) immediately after creating the context. The callback (or signal handler) must receive the GpgME context and the purpose it has been created for.

2. Key selection for signing and/or encryption

If I understand the key selection code correctly, it simply returns the first matching key on success, or just bails out with an error. This would be fine in a perfectly ideal world, but fails for the following common cases and is thus not usable:

1. There is more than one valid key for a certain uid. This happens if e.g.
   • legacy keys which have never expired or been revoked exist in the key ring (according to German laws, access to documents must be provided –depending upon the type of the document– for up to 30 years, so this case is very common),
   • a key is in use for legacy crypto hardware which supports 2k (or even less) keys only (quite common), or where high-grade keys must not be used for security reasons (think of Windows 10 and its nasty “telemetry” spying feature), and a different high-grade key is in use for modern, safe systems (see also item 3 below), etc.
2. The GpgME key listing does not find a key, because the user simply forgot to add a mail alias to the uid's, because the uid is obfuscated (both very common cases, too), or the crypto operations are performed by a gateway using some kind of “collective key”.
3. Even if there is exactly one matching key, it may be unusable due to further restrictions (e.g. key size, curve type, etc. – this is a quite special use case; see item 3 below).

As solution, GMime's key listing must call a callback with a list of keys (i.e. gpgme_key_t) and the reason for calling it (see the Balsa code for an example). The list shall contain

• in case 1: the matching keys. This callback must always be called;
• in case 2: all usable (not expired, revoked, etc.) keys. The context might be configured to bail out in this case instead of calling the callback, although I don't see why this behaviour should be of any real-life use;
• in case 3: the matching (single) key. This should be optional (configurable).

Furthermore, the key listing mode and properties of the context must be configurable, but this would be covered by solving item 1 above.

BTW, a key selection based upon InternetAddressMailbox items is really a very bad idea: a typical use case is to configure the key id (instead of the mail address) in user profiles as to force using a specific key, which would of course be impossible when requiring InternetAddressMailbox arguments.

3. Key properties

The access to (sub-) key properties in the GMime interface (“GMimeCertificate”) is extremely limited. In my projects, I must have access

• to all key properties, in particular the flags,
• to all subkeys, including its properties, in particular (but not limited to) to is_de_vs (German Government classified at “Restricted” level = “VS-NfD”),
• to all uid's, including its properties, in particular the key signatures and the tofu information.

Note that this information is required both for key listings (i.e. in the callback outlined above), and for decryption and signature verification data.

Providing access to the GpgME data via a method, e.g. const gpgme_key_t g_mime_certificate_gpgme_key(GMimeCertificate *cert), should be sufficient.

4. Signature verification info

The signature information provided by GMime is incomplete and therefore unusable for me. Access to in particular the PKA data, chain model information, notation and policy data is required. Again, a method like const gpgme_verify_result_t g_mime_signature_gpgme_result(GMimeSignature *sig) should be sufficient.

5. Decryption result

The decryption result lacks the flags (e.g. VS-NfD, see item 3 above). Please provide access to the full data, e.g. const gpgme_decrypt_result_t g_mime_decrypt_result_gpgme_result(GMimeDecryptResult *result) or similar.

---

Note: the following points are not blockers, but more a wish list. I have implementations for them (using my own GpgME interface, of course), but they might also be of interest for other projects.

6. RFC 4880 messages

It would be great if GMime could provide support for “old” RFC 4880 signed and/or encrypted messages. For sending, this should be limited to just signing and/or encrypting a single-part (text/plain) message.

The interpretation of such parts is a lot more complex, as a message part may contain a mixture of signed and/or encrypted and unsigned/unencrypted data. A common case are mailing list processors which append a few lines of information after the signed stuff, but it is also a method used in attacks (see the Johnny, you are fired! and Re: What's Up Johnny? papers). A method interpreting such message parts therefore should return a list of chunks, e.g. for the simple mailing list example a text “part” with signature information (the original message), plus a text “part” without it (the stuff appended by the list processor).

7. Autocrypt keys via key id or mailbox

The Autocrypt code is very nice, but it would be great if I could have a method which loads the minimal key (as requested by the Autocrypt specs) based upon the key id (or mail address, but see the remark about InternetAddressMailbox above), e.g. void g_mime_autocrypt_header_set_keydata_by_id(GMimeAutocryptHeader *ah, const gchar *key_id). However, exporting the minimal key is a little tricky, as it is not supported directly by GpgME, and depends upon the underlying gpg version (see the Enigmail or Balsa sources).

8. Standards compliance

I believe that GMime is really well-tested and documented. However, for including GMime in (non-FOSS) projects requiring formal approvals it would be helpful if you could prove that GMime complies with usual safety/security standards like MISRA C:2012 or the SEI CERT C Coding Standard, i.e. if you could provide a test/compliance report from a tool like Flexelint, LDRA or similar – typically not in the toolbox of OSS programmers, though, so this is really a “nice-to-have” topic.

[bernhardreiter]

Would be cool to have support for getting pubkey with https://gnupg.org/WKD (Web key directory).
It probably just works already when setting the right gpgme options for external key listings, but some examples would be helpful.

[sigasigasiga]

it could be nice to add an ability to copy GMimeObject class. AFAIK it couldn't be implemented in 3.x because of ABI breakage https://www.mail-archive.com/gmime-devel-list@gnome.org/msg00506.html

[gouthamkrishnakv]

Is there a plan to move to Meson or CMake?

Most GNOME applications do use Meson, and I was thinking of writing meson build files to integrate it into my project. Automake seemed to be difficult and cumbersome for my use-case, and so this might probably make it easier to integrate it later on for future projects, also.

[jstedfast]

@gauthamkrishna9991 I haven't been keeping up with GNOME development trends for quite a while so haven't even heard of Meson and I haven't used CMake enough to have an opinion on that (although I have at least heard of it).

[gouthamkrishnakv]

@jstedfast Okay, cool cool. I'll probably have to look into integrating Autotools into my project, then. I was thinking maybe you could check this out for a major version bump, as this would be a good time to check out meson itself.

Just to give you an idea, this is how a meson.build file looks like (in project root).

project('gmime', ['c', 'cpp'], version: '4.00')

# Dependencies
x1_dep =  dependency('x1', version: '>= 2.0', fallback = ['x1', 'x1_dep'], required: true)

# Go inside 'gmime', run 'gmime/meson.build`
subdir('gmime')
# Inside `gmime/meson.build`: sources = files('gmime.c', 'gmime-message.c', ...)

# Initialize Shared Library
shlib = shared_lib('gmime',
  dependencies: [x1_dep],
  install: true,
)

# Run tests
subdir('tests')

# Declare GMime Dependency, Everything else is taken care of by Meson.
gmime_dep = declare_dependency(
  include_directories: include_directories('gmime'),
  link_with: shlib
)

In any case, I'll end up checking for Autotools integration for my personal project, as that seems to be your preference.

[c-alpha]

@gauthamkrishna9991 I haven't been keeping up with GNOME development trends for quite a while so haven't even heard of Meson and I haven't used CMake enough to have an opinion on that (although I have at least heard of it).

In https://github.com/astroidmail/astroid we migrated from meson to CMake. Automake is certainly showing its age. That said, my recommendation would be to go with CMake.

[dagle]

Refcounted format and parser options.
Atm these boxed types are then copied multiple times when used.
I propose that we should reference count these object instead of copying.

If a user wants to pull out parser from a structure, change a values without wanting it propagated,
a g_mime_(format|parser)_options_copy should be used (which already exists for format).

It also simplifies the semantics for callbacks and NotifyDestroy, since copy wouldn't
copy the callback and clone (ref) would just increase a reference counter.

These change need to be stated in the doc strings and changing the clone/free to ref/unref
should also make it more clear what is going on.

I have written a test version with these changes and I haven't run into any problems so far.

[jstedfast]

@dagle

The reason they are cloned instead of ref-counted is because GMimeHeader (and other objects that keep refs to them) need to be sure they haven't changed since they were referenced.