Question: Are documentation-related dependecies being installed by default?

[Kakadu]

This project was mentioned at https://discuss.ocaml.org/t/github-ci-execution-times-for-ocaml-projects/17946/11?u=kakadu and I decided to give it a try.

I have a project, where I want to build documentation in CI. There is (odig :with-doc) in a dune-project file, but I'm getting odig: No such file or directory. Should it be installed? Maybe I need to wrap the call into something, like opam exec -- ?

The ill CI log is here: https://github.com/Kakadu/zanuda/actions/runs/24001053946/job/69996995275

[mbarbin]

Hi @Kakadu -

I looked a bit at your CI and from what I can tell, this sounds like you are hitting some rough edges of dune-pkg specifics rather than setup-dune. Indeed, already trying to make the equivalent of the ci run locally solely by using dune-pkg reveals some challenges for me.

What I tried:

• You created a dune-workspace.ci file but you need to specify it in the CI as input otherwise it won't get used (there is an action input for that).
• By default it would seem that with the dune I used on my machine to try, dune pkg lock won't include packages dependencies that are marked with :doc. So I tried:

  (lang dune 3.18)                                                                                                                                                                     

  (lock_dir                                                                                                                                                                            
+ (solver_env                                                                                                                                                                         
+  (with-doc true))                                                                                                                                                                   
    (constraints                                                                                                                                                                        
      (ocaml                                                                                                                                                                             
       (= 4.14.2))))  

which made the dune pkg lock succesfully include the odig and odoc packages, but then building fails with some cyclic dependency error, which seems to me like a bug of dune-pkg that would be worth reporting.

What I would try next:

• For similar issues I have encountered in the past, I resorted to stopping to use :with_doc in packages and instead use dedicated -dev packages where I put all my dev dependencies there as regular dependencies. Perhaps that could work for you.
• To run binaries installed by dune pkg I think the invocation is dune exec -- ...
• If building your doc can be achieved with regular dune build @doc maybe that could save you the odig ci dependency, and managed to make your CI fall within more common and supported path (not satisfying long term perhaps, but useful if you want to get going).

If you want to see some example of another OCaml project that uses setup-dune to build doc, you may have a look at https://github.com/mbarbin/dunolint

If you wanted to open a PR in your project and flag me there I'm happy to team up with you and try to contribute towards that experiment, find out more of blockers etc. I'm interested to have other folks join as I believe this may benefit the community long term. Thanks!

[Kakadu]

@mbarbin thank you for your fast response.

I tried to follow your recommendations and added odig dependency without any special flags. It is runnable now, but to use it as planned I need to perform dune install. Unfortunately, it crashes dune: https://github.com/Kakadu/zanuda/actions/runs/24007368502/job/70013164177 I don't know who's fault it is: dune's, setup-dune's or combination.

Internal error! Please report to https://github.com/ocaml/dune/issues,
providing the file _build/trace.csexp, if possible. This includes build
commands, message logs, and file paths.
Description:
  ("Unexpected build progress state (expected [Building _])",
   { current = Initializing })
Raised at Stdune__Code_error.raise in file "stdune__Code_error.ml", line 11,
  characters 30-62
Called from Dune_engine__Build_system.Exported.execute_rule_impl in file
  "dune_engine__Build_system.ml", line 511, characters 41-66
Called from Fiber__Scheduler.exec in file "fiber__Scheduler.ml", line 77,
  characters 8-11
Re-raised at Stdune__Exn.raise_with_backtrace in file "stdune__Exn.ml"
  (inlined), line 39, characters 27-56

I also tried to leave :with-doc and add (solver_env (with-doc true)) as you suggested. It doesn't crash, but gives a cyclic error about external dependencies.
https://github.com/Kakadu/zanuda/actions/runs/24008944284/job/70017226050

Install the external dependencies
  Error: Fatal error: got "Error: Dependency cycle between packages:
     angstrom.0.16.0
  -> bigstringaf.0.10.0
  -> dune-configurator.3.22.1
  -> odoc.3.1.0
  -> tyxml.4.6.0
  -> odoc.3.1.0" when listing depexts

It looks like today the best outcome will be a few bug reports. Initially I planned to check if builds will be faster, but at the moment running tests is 5m 9s and the output is too rigid to understand why. Maybe this looks this way because I never used dune-powered package management, who knows...

[mbarbin]

Ah! Unfortunate it's not an easier time for your first try! On the bright side indeed I think these reports will be helpful for dune-pkg.

It's very possible I may not have given you the most helpful recos, as I haven't looked at what the CI is achieving. I'll try forking your repo and play around on my side when I get a chance (e.g. I don't understand why you need installing).

Just in case this unlocks things to the next step, when you use setup-dune with a custom workspace file, I'd expect that next invocation shall use the same workspace file. I think you may have a couple ways to ensure that, either supply the --workspace option for all dune invocations but maybe not practical, so perhaps look into setting this as a workflow env var DUNE_WORKSPACE=dune-workspace.ci instead.

Until you get a successful run the cache is not going to be saved so we'll only get to know the happy-path and cached performances after achieving a first green run.

For getting more info about the build I think recent development in the repo added some support for downloading a trace file but I can't tell you much more about this as I haven't used this yet. Talk soon!