Bundled zlib installed by OR-Tools prevented graphical login on SELinux (by masking the system library)

[gabriel-gehrke]

TL;DR

Installing OR-Tools from source with BUILD_DEPS=ON and the default /usr/local prefix installed a bundled libz.so.1 into /usr/local/lib64, causing SELinux-related problems due to masking the system installation of zlib in unrelated system services, which ultimately resulted in service restart loops and the desktop environment failed to start (causing the screen to stay black after booting).

My opinion: bundled dependencies with common SONAMEs such as libz.so.1 should ideally not be installed into globally searched library paths. An isolated install location would be the cleaner solution.

What version of OR-Tools and what language are you using?

Version: 9.15

Language: C++

Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)

Not solver-specific. This concerns the CMake build and installation process.

What operating system (Linux, Windows, ...) and version?

Aurora DX / Fedora Atomic 44, x86_64, with SELinux enforcing.

On this system, /usr/local is backed by /var/usrlocal.

What did you do?

Steps to reproduce the behavior:

1. Build OR-Tools from source with bundled dependencies enabled:

   cmake -S . -B build -DBUILD_DEPS=ON
   cmake --build build
   sudo cmake --install build

2. Use the default install prefix, /usr/local.

3. The installation creates files including:

   /usr/local/lib64/libz.so.1.3.1
   /usr/local/lib64/libz.so.1 -> libz.so.1.3.1
   

4. On a system where /usr/local/lib64 is registered in the dynamic-linker cache, check which zlib a system service resolves:

   ldd /usr/lib/systemd/systemd-logind | grep libz

5. The result was:

   libz.so.1 => /usr/local/lib64/libz.so.1
   

What did you expect to see

I expected the OR-Tools installation either:

• not to install bundled general-purpose dependencies such as zlib into a globally searched library directory;
• to install them in an OR-Tools-specific private directory; or
• to emit a clear warning that bundled libraries may mask distribution-provided system libraries.

Unrelated system services should continue resolving zlib from the distribution path:

libz.so.1 => /lib64/libz.so.1

What did you see instead?

The OR-Tools-provided libz.so.1 masked the distribution library for unrelated programs.

systemd-logind attempted to load:

/usr/local/lib64/libz.so.1

which maps to:

/var/usrlocal/lib64/libz.so.1

SELinux denied the confined service access to the locally installed library, which had a generic usr_t context.

This caused systemd-logind to repeatedly fail. Plasma could not create graphical user sessions, resulting in a black screen and failed graphical login.

After moving /usr/local/lib64 out of the linker path and running:

sudo ldconfig

the service resolved the distribution library again:

libz.so.1 => /lib64/libz.so.1

The SELinux denials stopped and the graphical login immediately recovered.

Anything else we should know about your project / environment

The exact impact depends on the distribution and linker configuration.

/usr/local/lib or /usr/local/lib64 is globally searched by default on some distributions, while on others it may be added through /etc/ld.so.conf.d by the administrator or another installer.

The underlying risk is therefore around installing bundled libraries with common SONAMEs into a globally searched /usr/local directory can silently override system libraries for unrelated applications.

SELinux made the problem visible by blocking the confined service from loading the locally installed library. On a system without SELinux enforcement, the replacement library might instead be loaded silently (which could cause other, e.g. version-related problems, I guess).

Possible improvements:

• install bundled dependencies into an OR-Tools-specific private directory;
• warn when bundled dependencies will be installed under /usr or /usr/local;
• document the risk of masking system libraries;
• recommend an isolated CMAKE_INSTALL_PREFIX.