Build Crashes when `src` has Submodules

[ecyht2]

The build will crash when my source directory uses a git submodules.

$ nix-build
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:37:12:
           36|
           37|   strict = derivationStrict drvAttrs;
             |            ^
           38|

       … while evaluating derivation 'my-project-0.1.0'
         whose name attribute is located at /nix/store/p893dkrzm5rxvhnqh092prgi1a7dzmcy-source/pkgs/stdenv/generic/make-derivation.nix:480:13

       … while evaluating attribute 'src' of derivation 'my-project-0.1.0'
         at /nix/store/0mb5pvalg6dg8fr8vbva433aqx9rz9rq-source/flake.nix:107:9:
          106|
          107|         src = pkgs.lib.fileset.toSource {
             |         ^
          108|           root = ./.;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: lib.fileset.unions: Element 0 (/nix/store/0mb5pvalg6dg8fr8vbva433aqx9rz9rq-source/ggml) is a path that does not exist.
           To create a file set from a path that may not exist, use `lib.fileset.maybeMissing`

My default.nix:

(import (fetchTarball "https://github.com/edolstra/flake-compat/archive/master.tar.gz") {
  src = ./.;
})
    .defaultNix

My package:

    packages = forAllSystems ({
      pkgs,
      system,
    }: {
      cpu = pkgs.stdenv.mkDerivation {
        pname = "my-project";
        version = "0.1.0";

        buildInputs = [
        ];
        nativeBuildInputs = [
          pkgs.cmake
          pkgs.clang
          pkgs.git
        ];

        src = pkgs.lib.fileset.toSource {
          root = ./.;
          fileset = pkgs.lib.fileset.unions [
            ./ggml/.
            ./src/.
            ./CMakeLists.txt
          ];
        };
      };

      default = self.packages."${system}".cpu;
    });

.gitmodules file:

[submodule "ggml"]
	path = ggml
	url = https://github.com/ggml-org/ggml.git