[INSTALL]: yaml-cpp

[benkozi]

Package name

yaml-cpp

Package version/tag

(most recent okay)

Build options

...

Installation timeframe

Installation in the next release is just fine. Thank you!

Other information

No response

[climbfuji]

We have libyaml/0.2.5 and py-pyyaml/6.0.2 in spack-stack-2.1 / develop. Do we really need another yaml library? What does it provide that the others don't?

[benkozi]

yaml-cpp is used by two new planned components to the WM: CATChem and CECE. ESMF also builds an internal version and could use the external. libyaml is C-based, which is not as programmer-friendly for C++ applications. In general, it would seem better to have the libraries share a single yaml-cpp build rather than compile their own copies or fetch from external sources.

[mathomp4]

@theurich Do you know if having an external yaml-cpp package would interfere with the internal yaml-cpp ESMF carries around?

Or, I guess, is there an option to point to an external package?

Of course, if your yaml-cpp is the same version as the "canonical" version, maybe it doesn't matter.

[theurich]

@mathomp4, you can set ESMF_YAMLCPP=standard, then point to the externally provided yaml-cpp via ESMF_YAMLCPP_INCLUDE and ESMF_YAMLCPP_LIBPATH. See: https://earthsystemmodeling.org/docs/nightly/develop/ESMF_usrdoc/node10.html#SECTION000104700000000000000

[mathomp4]

@mathomp4, you can set ESMF_YAMLCPP=standard, then point to the externally provided yaml-cpp via ESMF_YAMLCPP_INCLUDE and ESMF_YAMLCPP_LIBPATH. See: earthsystemmodeling.org/docs/nightly/develop/ESMF_usrdoc/node10.html#SECTION000104700000000000000

Ahhh. Okay.

So this looks like we need a new variant in the spack package, like +external-yaml-cpp. Luckily there are a few examples like this in the package around I can piggyback off of.

[mathomp4]

@theurich There is a bit of a twist I need your input on. So I went looking to see what version of yaml-cpp I could use and found the README.md for your internal yaml-cpp. In there it says it's based on v0.7.0 but:

ESMF uses the patched version of yaml-cpp with YAML merge-key-support.

• URL: https://github.com/loot/yaml-cpp/releases/tag/yaml-cpp-0.7.0%2Bmerge-key-support.1

So I went looking and...I'm not sure the yaml-cpp mother repo has that patch yet. I see a PR from @azat (jbeder/yaml-cpp#1279), that has not yet been merged.

Indeed, I'm not even sure it's the same merge key support as the fork-code has. Maybe equivalent, but not the same.

But, then again, maybe this whole "will ESMF's yaml-cpp collide" doesn't matter? I mean, I just did some ldd on the libesmf.so and libesmf_fullylinked.so and no yaml in those. I think you essentially statically compile in your internal yaml-cpp and I don't think ESMF can even see an external yaml-cpp if there was one around.

Indeed, it might be best we don't add external-yaml-cpp support to ESMF's spack package until and if you think it would be useful. If nothing else, you'd want to update the ESMF documentation to say something along the lines that an external yaml-cpp could remove merge key support. Currently in the docs you state:

YAML merge keys "«" are supported.

And, well, they might not be with an external yaml-cpp

[theurich]

@mathomp4 The yaml-cpp bundled with ESMF is pretty much https://github.com/loot/yaml-cpp/releases/tag/yaml-cpp-0.7.0%2Bmerge-key-support.1. However, notice that we only keep those directories that contain sources and headers and that it's integrated into the ESMF build process (instead of keeping its own separate one). So presumably if you were to build an external yaml-cpp installation based on the above version, it should give you the same functionality as with the ESMF provided yaml-cpp version.

Prompted by your question, I did take a closer look at the ESMF yaml-cpp integration and found that yaml-cpp's YAML symbols (C++ mangled) come through libesmf. So that would be an issue if you wanted to link an application against some external yaml-cpp installation while ESMF uses its internal version. To fix that opened PR esmf-org/esmf#574 where I rename the internal yaml-cpp symbols to ESMF_YAML instead. This should prevent symbol conflicts when linking an application explicitly against yaml-cpp.

[mathomp4]

@theurich Ooh. Thanks. I'll try and take a look soon.

So your update would essentially let ESMF co-exist with an external yaml-cpp even if ESMF was built with the internal?

[theurich]

@theurich Ooh. Thanks. I'll try and take a look soon.

So your update would essentially let ESMF co-exist with an external yaml-cpp even if ESMF was built with the internal?

Exactly.

[mathomp4]

@theurich Ooh. Thanks. I'll try and take a look soon.
So your update would essentially let ESMF co-exist with an external yaml-cpp even if ESMF was built with the internal?

Exactly.

Okay. Once your PR is all tested an in a tag/snap, that'll be when I add the option to the spack package.py. Probably with a "warning" or something letting users know they'll lose merge key functionality with an external yaml-cpp.

But for spack-stack, we can just build both yaml-cpp and esmf with the internal yaml-cpp and everyone is happy! 😄

[theurich]

Yes, everyone should be fine with spack-stack providing yaml-cpp and esmf along side (even without the option to use the spack-stack-built yaml-cpp inside ESMF). The user application can use either or both, they should not conflict with what ESMF is doing internally.

Honestly, with potential linking conflicts out of the way, I am not even sure how valuable it is at this time to offer the option to build ESMF against external yaml-cpp as a spack package.py option. I don't think it is a bad idea in general -- we certainly offer it on the ESMF build level -- but with official yaml-cpp not yet having merge-key support, it seems more of an option to shoot yourself in the foot than to gain anything. Just a quick thought.