Native code export for LAMMPS and Python/ASE

[jameskermode]

Summary

This PR adds infrastructure for exporting fitted ETACE potentials to standalone native libraries using Julia 1.12's juliac --trim=safe feature. The compiled libraries can be used with LAMMPS and Python/ASE without requiring a Julia installation at runtime.

Features

Export Pipeline

• Export ETACE models to trim-compatible Julia code
• Compile to native shared libraries (~3 MB) with bundled Julia runtime (~20 MB)
• Hermite cubic spline radial basis export for machine-precision accuracy
• Standard ACE models (ace1_model) must be converted to ETACE via ETModels.convert2et() before export

LAMMPS Integration

• pair_style ace plugin that loads compiled libraries directly
• Uses LAMMPS native neighbor lists (no Julia dependency)
• Supports MPI domain decomposition and OpenMP threading

Python/ASE Integration (ase-ace package)

Three calculator options with different tradeoffs:

Calculator	Backend	Threading	Startup	Julia Required
ACELibraryCalculator	Compiled .so	Single	Instant	No
ACEJuliaCalculator	JuliaCall	Multi	10-30s	Managed
ACECalculator	Socket/i-PI	Multi	5-10s	Runtime

Documentation

• Comprehensive README with quickstart guide
• End-to-end ETACE to LAMMPS tutorial (integrated into Literate.jl docs)
• C API reference for exported functions
• Troubleshooting guide

Directory Structure

export/
├── src/                    # Core export functionality
├── lammps/                 # LAMMPS plugin source and examples
├── ase-ace/                # pip-installable Python package
├── examples/               # Tutorial workflows
│   └── etace_lammps_tutorial.jl
└── README.md               # Comprehensive documentation

benchmark/
├── lammps/                 # LAMMPS input files and run scripts
└── results/                # Benchmark logs

Requirements

• Export: Julia 1.12+ with juliac support
• LAMMPS: Any version with plugin support
• Python: 3.9+, ASE, matscipy (for library calculator)

Test Plan

• Export ETACE model and verify energies/forces match Julia
• Export with Hermite splines for machine precision
• Build and run LAMMPS plugin with compiled library
• Python/ASE calculators work with compiled library
• MPI scaling benchmark with 1-8 processes

[tjjarvinen]

I am not sure are you aware that you can already use ACEpotentials with ASE and LAMMPS.

It is done with IPICalculator.jl and current used by e.g. DFTK for ASE. I have also used ACEpotentials with ASE extensively, so it is well tested.

Using the i-PI socket server protocol is the approach preferred by Ask Hjorth Larsen the ASE maintainer. This was discussed with him during Julia MolSSI Workshop last year and other occasions afterwards. The implementation at IPICalculator.jl is meant to serve all AtomsCalculators compatible calculators in JuliaMolSim ecosystem by providing i-PI protocol suport and general ASE interface.

LAMMPS supports Molssi Driver Interface (MDI) in server mode https://docs.lammps.org/mdi.html, which is compatible with i-PI protocol, with the exception of not having support for Unix pipe. You can use ACEpotentials with IPICalculators in LAMMPS this way. Although I have to admit that the LAMMPS documentation is not clear at all. I have not tested ACEpotentials in LAMMPS this way, but I expect it to work

Thus, I propose that instead of using resources to implement our own ASE and LAMMPS binary interfaces - this PR - we use the existing i-PI protocol interface to connect to ASE and LAMMPS, and make documentation entries for them. This the easiest way to get ASE and LAMMPS interface going and reduces maintenance burden for ACEpotentials.

[jameskermode]

Thanks for letting me know, I was aware of the IPICalculator.jl support but not that it could be used with LAMMPS. Nonetheless, I think native code export I am pursuing here is a good path to explore because it allows MPI-parallel domain decomposition. This should be much more scalable than socket communications for very large systems. ASE is then a fringe benefit that used the same native code export - and also benefits from OpenMP parallelisation that wouldn't be easy to do with IPICalculator. Initial tests give ca. 5x speed up on 8 threads even for small systems. Moreover, so far I only spent a day or so on it and it's nearly working.

[tjjarvinen]

MPI is a valid point. Although there is the executor keyword that allows you to perform distributed calculation already in Julia

# start Julia in parallel and then use DistributedEx() instead of the default ThreadedEx()
energy_forces_virial(system, ace_calc;  executor=DistributedEx(), ntasks=nworkers() )

I am not sure what the performance is with it, but you could compare it to the MPI implementation. (DistributedEx() might not be exported in ACEpotentials so you need to get it from Folds.jl or Transducers.jl)

Also, I am really curious how did you manage get 5x performance from OpenMP? The default Julia version is multi threaded and scales well with number of threads when neighbourlist calculation time is ignored.

[jameskermode]

It would indeed be interesting to compare LAMMPS MPI performance with Julia executor, but my experience in general on HPC machines is that MPI has been heavily vendor optimisated and in comparison's Julia distributed calculations are typically not competitive. This is likely to be particularly true for LAMMPS.

Standard Julia threads can't be used with --trim=safe native code, so I added OpenMP on the loop over atoms outside the Julia call. 5x on 8 cores isn't great but this was only 128 atom system and this is including the neighbourlist overhead. I was intending it for use either from Python, or within multilevel MPI/OpenMP parallelism from LAMMPS, just to give a bit of extra speedup and make full use of available hardware threads.

[jameskermode]

Another advantage of this approach: it looks like we can bundle the Julia runtime libraries, meaning no Julia installation required for end users.

[tjjarvinen]

MPI should be faster when communication limited, if computation limited then MPI and Julia should give about the same results. The point here would be get an idea of when the situation changes to communication limited case.

The point of including Julia runtime is also good. It should make ACEpotentials easier to use when only compute is needed.

• Standard Julia for small system where multi threading inside a single node - sockets with Unix pipe is same as using MPI on local node
• Large system with MPI using several nodes (might even be valid to use MPI in Julia too at some point)
• Easy to use binary package for compute only

I should also work on to finish the new neigbourlist package. It would be faster than the current one, is multithreaded and can be made to work with MPI.

[cortner]

@jameskermode -- this is entirely outside of what I want to have any control over and would be very happy to merge this so we can proceed with the other PRs.