[R-package] [ci] Breaking changes with R-devel

[jameslamb]

Description

Over the last week or so, I've noticed that R-package CI jobs testing against R-devel have often broken because lightgbm's dependencies could not be compiled successfully.

It seems there is a lot of activity in R-devel recently around re-organizing R's headers.

Opening this to track the issues that come up.

[jameslamb]

Issue 1: R_ext/PrtUtil.h reorganization breaking {yaml}

Description

{yaml} failed to build against R-devel like this:

/opt/R/devel/lib/R/include/R_ext/PrtUtil.h:30:49: error: function-like macro 'Rcpp_Version' is not defined
   30 | #if ! defined(COMPILING_RCPP) || RCPP_VERSION > Rcpp_Version(1,1,0)

(clang18 R-devel job)

And this:

In file included from r_emit.c:1:
./r_ext.h:19:10: fatal error: 'R_ext/PrtUtil.h' file not found
   19 | #include "R_ext/PrtUtil.h"
      |          ^~~~~~~~~~~~~~~~~
1 error generated.

(clang17 R-devel job)

Resolution

Looks like this was just resolved on the dev version of {yaml}:

• R devel issue r-lib/yaml#157
• C API compliance r-lib/yaml#158

Not sure when a new version with that fix will be submitted to CRAN.

[jameslamb]

Issue 2: R_ext/Callbacks.h removed

Description

I haven't actually seen this affecting LightGBM, but it might end up affecting one of our dependencies so want to share that there's already a discussion on the r-pkg-devel mailing list about it.

In file included from D:/RCompile/CRANpkg/lib/4.6/Rcpp/include/Rcpp.h:27,
                 from RcppExports.cpp:4:
D:/RCompile/CRANpkg/lib/4.6/Rcpp/include/RcppCommon.h:75:10: fatal error: R_ext/Callbacks.h: No such file or directory
   75 | #include <R_ext/Callbacks.h>
      |          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

Resolution

See https://stat.ethz.ch/pipermail/r-package-devel/2025q4/012119.html

Looks like it was fixed in {Rcpp} in RcppCore/Rcpp#1418, though there isn't a new release on CRAN yet.

[eddelbuettel]

Note that those changes were (partially) reverted too so check your r-devel build and possibly upgrade. For your amusement, the list of R-devel commits Thursday to today gives an idea too.

Rcpp 1.1.0.8, the version from the current main, also no longer includes those 'to be removed' files (that have been partially reverted to unbreak CRAN) so you can fetch that from our r-universe page.

And just to be plain, Rcpp is the poor messenger that is getting shot because files it was able to include for many years all of a sudden disappeared (and have since been reverted back as noted above).

[eddelbuettel]

CRAN twisted my arm so there is now a Rcpp 1.1.0.8.1 in the 4.6.0/ directory on CRAN and used for R-devel; you could pull that too. (It is equivalent to our master branch and what is on r-universe too, but I had to make a tweak to the testing setup because 'more than release number a.b.c implies dev version with more tests' and then "stuff happens" I don't want at CRAN. But that is very inside-baseball now. Let me know if I can help any other way.)

[jameslamb]

Thank you @eddelbuettel !

It looks like test-time LightGBM's dependencies can all be installed with the latest R-devel now (#7100 (comment)), without code here needing to directly build non-released versions of any libraries.

I also noticed at https://cran.r-project.org/web/checks/check_results_yaml.html that CRAN is now taking the approach of warning packages about the use of these headers:

Version: 2.3.11
Check: whether package can be installed
Result: WARN 
  Found the following significant warnings:
    /data/gannet/ripley/R/R-clang/include/R_ext/PrtUtil.h:35:2: warning: non-API header file R_ext/PrtUtil.h is obsolete and will be removed [-W#warnings]

I suspect that {yaml} will also be be asked told to put up a new release on CRAN very soon.

I think that concludes this episode from LightGBM's perspective, thanks as always for your help 😊