Universal loss function evaluated with PerAtomMAE

[doouv]

Hi, I am currently experimenting with MACE and noticed something unexpected regarding error table selection when using the universal loss function.

According to mace/tools/model_script_utils.py, setting loss="universal" should automatically enable compute_stress=True and change error_table to "PerAtomRMSEstressvirials".

def configure_model(
    args, train_loader, atomic_energies, model_foundation=None, heads=None, z_table=None
):
    # Selecting outputs
    compute_virials = args.loss in ("stress", "virials", "huber", "universal")
    if compute_virials:
        args.compute_stress = True
        args.error_table = "PerAtomRMSEstressvirials"

However, the 2024-01-07-mace-128-L2.sh script uses the universal loss function while computing stress, yet the MACE-MP model seems to have been evaluated using "PerAtomMAE". Additionally, the results in the MACE paper appear to report errors in MAE/atom rather than RMSE.

My Questions:

• Why does MACE-MP retain PerAtomMAE for evaluation, even though universal loss with stress computation should default to PerAtomRMSEstressvirials?
• If energy and force prediction are the primary objectives, is it necessary to use the universal loss function, or would weighted loss function be more appropriate?

I would appreciate any insights you can provide. Thanks for your time!

[ilyes319]

Hey @doouv,

• All the metrics are always logged in the train file in your result folder, so you can always see these statistics. That is how it was extracted in the paper. Note that the paper is quite some version older now, so things have changed since then in terms of logging.
• The universal loss is most useful when you have very noisy dataset with lot of high forces/stresses configs because it uses the Huber loss. If your training set is well behaved you can use the normal weighted loss.