Warning message building on Aurora with latest Aurora libraries

[zhangchonglin]

While building PUMIPic at commit 04ade03 on Aurora with kokkos 5.0.0 and newer libraries (mpich, oneapi etc), I encountered the following warning message.

[ 36%] Linking CXX static library libparticleStructs.a                              
[ 36%] Built target particleStructs                                                 
[ 52%] Building CXX object particle_structs/CMakeFiles/smoke_test_particle.dir/test/smoke_test_particle.cpp.o
[ 57%] Building CXX object src/CMakeFiles/pumipic-core.dir/pumipic_input.cpp.o  
[ 57%] Building CXX object src/CMakeFiles/pumipic-core.dir/pumipic_lb.cpp.o         
[ 57%] Building CXX object src/CMakeFiles/pumipic-core.dir/pumipic_part_construct.cpp.o
Compilation from IR - skipping loading of FCL                                   
Build succeeded.                                                                
warning: Undefined function _ZNK7Omega_h4ReadIiEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h4ReadIdEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIdEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h4ReadIiEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIiEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h4ReadIlEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIlEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h4ReadIaEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIaEixEi found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIiEixEi found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h4ReadIiEixEi found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
warning: Undefined function _ZNK7Omega_h5WriteIiEixEi found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
Compilation from IR - skipping loading of FCL                                   
Build succeeded.   

More observations:

• The code was built successfully. But some simple tests crashed (will update more on this).
• Looking at the build log from earlier kokkos 4.6.02 version in Compilation and run issue on ALCF Aurora #147, these warnings were not in the log file when earlier Aurora libraries were used.
• When using the same code versions as before (kokkos 4.6.02, PUMIPic at commit: 1f654d8, omega_h at commit: cbb705a), the above warning messages are in the log file when built with latest Aurora libraries shown below.

Currently Loaded Modules:
  1) libfabric/1.22.0                 5) oneapi/release/2025.2.0       9) gcc/13.3.0                     13) gmake/4.4.1
  2) cray-pals/1.8.0                  6) gmp/6.3.0-mtokfaw       (H)  10) libxml2/2.13.5                 14) cmake/3.31.8
  3) cray-libpals/1.8.0               7) mpfr/4.2.1-gkcdl5w      (H)  11) hwloc/2.12.1-level-zero
  4) gcc-runtime/13.3.0-ghotoln (H)   8) mpc/1.3.1-rdrlvsl       (H)  12) mpich/opt/develop-git.6037a7a

  Where:
   H:  Hidden Module

@Sichao25, @cwsmith: I am wondering if you guys have any idea what could be the issue here?

[cwsmith]

I'm not sure offhand what the problem could be.

Running those warnings through c++filt gives the following, which seems fine as those are just calls to the omegah read array [] operators. Did you happen to see this when linking omegah's binaries?

For the record, are you using AOT compilation or JIT? I'm assuming AOT.

$ c++filt < log 
warning: Undefined function Omega_h::Read<int>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Read<double>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<double>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Read<int>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<int>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Read<long>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<long>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Read<signed char>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<signed char>::operator[](int) const found in /tmp/icpx-59553d4ee6/pumipic_part_construct-da1b96.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<int>::operator[](int) const found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Read<int>::operator[](int) const found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
warning: Undefined function Omega_h::Write<int>::operator[](int) const found in /tmp/icpx-ef61ea8315/pumipic_lb-05e9e0.bc. This may result in runtime errors.
Compilation from IR - skipping loading of FCL          

[zhangchonglin]

@cwsmith: thank you for looking into this.

• I did not remember seeing these warning while building omega_h, but will check again later.
• Some of the unit tests I performed manually on Aurora crashed, which were all passing with earlier Aurora libraries. I will write up on that later once I have more time.
• I am using AOT compilation.

[zhangchonglin]

With kokkos 4.6.02, PUMIPic commit 1f654d8, and enabling tests, the compilation did not finish. Some of the unit tests where the binary files are generated also failed. Complete log file is attached pumipic build and test log.

Currently Loaded Modules:
  1) libfabric/1.22.0                 6) gmp/6.3.0-mtokfaw  (H)  11) hwloc/2.12.1-level-zero
  2) cray-pals/1.8.0                  7) mpfr/4.2.1-gkcdl5w (H)  12) mpich/opt/develop-git.6037a7a
  3) cray-libpals/1.8.0               8) mpc/1.3.1-rdrlvsl  (H)  13) gmake/4.4.1
  4) gcc-runtime/13.3.0-ghotoln (H)   9) gcc/13.3.0              14) cmake/3.31.8
  5) oneapi/release/2025.2.0         10) libxml2/2.13.5

  Where:
   H:  Hidden Module