[CoSim] Fix floating-point tolerance in KeepAdvancingSolutionLoop (CoSimAnalysis)

[juancamarotti]

Description

This PR fixes an issue in the CoSimulationAnalysis time loop where an additional time step could be executed due to floating-point round-off errors when reaching the end of the simulation.

Previously, the default implementation of KeepAdvancingSolutionLoop() could continue the simulation even when the current time was numerically equal to the specified end_time (e.g. 1.999999999999999 vs. 2.0). As a consequence, the coupled analysis advanced one extra time step, leading to synchronization issues with external solvers that had already finalized.

This PR overrides KeepAdvancingSolutionLoop() to stop the simulation once the current time reaches the prescribed end_time within a specified tolerance.

Motivation

This issue was observed during partitioned Fluid–Structure Interaction simulations with OpenFOAM through CoSimIO. While the external solver correctly finalized at the last time step, the CoSimulation loop advanced once more due to floating-point inaccuracies, causing the coupled simulation to deadlock while waiting for data from an already finalized solver.

Changes

• Override KeepAdvancingSolutionLoop() in CoSimulationAnalysis.
• Introduce a tolerance-based comparison with end_time to determine whether the solution loop should continue.
• Prevent execution of an additional time step caused by floating-point round-off errors.

Result

The coupled simulation now terminates consistently at the prescribed end time, avoiding unnecessary extra time steps and ensuring proper synchronization with external solvers.

[matekelemen]

Rather than relying on shady floating-point comparisons, I recommend sending an extra signal from each solver that they finished/terminated.

[juancamarotti]

Rather than relying on shady floating-point comparisons, I recommend sending an extra signal from each solver that they finished/terminated.

That is done for the external solver side, and it's not enough. If we don´t do this, kratos tries to advance to the next time step even when the external solver already disconnected.

[matekelemen]

Please wait.

I don't understand. How can the external solver disconnect (without an exception) if CoSim did not tell it to do so. Alternatively, why does Kratos attempt to continue to the next time step if the external solver signaled it to stop?

[juancamarotti]

Please wait.

I don't understand. How can the external solver disconnect (without an exception) if CoSim did not tell it to do so. Alternatively, why does Kratos attempt to continue to the next time step if the external solver signaled it to stop?

The two termination mechanisms are independent. The external solver decides whether to terminate based on the control signals exchanged through the solver wrapper, whereas the Kratos CoSimulation framework decides whether to execute another global time step exclusively through KeepAdvancingSolutionLoop(). Therefore, the external solver can correctly identify that the final time has been reached and finalize, while Kratos may still decide to advance if its own end-time check evaluates to True.

In the example I run, the current time was 1.999999999999999 instead of exactly 2.0 due to floating-point round-off. Since KeepAdvancingSolutionLoop() relied on an exact comparison, Kratos incorrectly started one additional time step, while the external solver had already reached its termination criterion (because a tolerance is also used in the solver wrapper to deal with this).

@sunethwarna @philbucher could you add something more here?

[juancamarotti]

@matekelemen I'll try to explain the issue more clearly.

Kratos and OpenFOAM have independent time-marching algorithms. OpenFOAM uses its own end_time specified in its input file (in this case end_time = 25.0), so it terminates once that time is reached.

On the other hand, due to floating-point round-off, Kratos may reach a time such as 24.999999999999 and still decide to advance one more time step. By the time it does so, OpenFOAM has already terminated. As a result, Kratos waits for coupling data that OpenFOAM will never send, causing the simulation to hang.

Does that make the issue clearer?

[rubenzorrilla]

I wouldn't expose this but hardcode it (this is what we normally do for floating point comparisons of this sort).

[loumalouomega]

Literally i asked for this, you never know how small is going to be the time step considered in the simulation, imagine a coupling of a explicit simulation where 1e-12 could be significative...

[rubenzorrilla]

Well, if you reach the point in which the time step is of the order of the floating point comparison, I'd say that your problem is not the comparison but a different thing... (even though you're explicit) In any case I don't want to block it because of this. Go ahead if you really need this. I just want to keep the I/O as simple as possible.

[matekelemen]

@juancamarotti I'm familiar with the concept of floating point rounding errors. My issue is not that, but problem of which solver is in control.

Forget about time.

In a scenario with two domains-specific solvers A and B, with CoSim C doing the coupling, one has to decide who is in charge of what to do when.

In an implicit coupling scenario, this has to be the coupling solver C, because it is the only entity in this system that knows when coupling converged for a given time step. This means that only C can decide when to terminate the analysis.

Your problem is that B decided (on its own) that the analysis is complete and that it should terminate. This is incorrect, because only C should be able to make that decision. Your proposed fix would only increase the chances that A, B and C just so happen to decide on terminating at the very same time step, given the user correctly configured end times for all solvers.

I hope you understand that this fix does not address the underlying problem at all. I can configure A and B today to correctly stop at the same time. On the other hand, I can just as well misconfigure them, even with your patch, to not terminate at the same time.

In my opinion, the correct fix is making A and B ignore their end times, and only rely on C.

Alternatively, if you really really want to, you could implement a way for A or B to notify C that they're going to terminate. This is OK for exception handling, but under normal circumstances, A and B should not decide to terminate on their own at all.

[juancamarotti]

@matekelemen I'm sorry but I don't agree with you. The solvers can be notified through the solver wrapper when to disconnect (and this already happens with the current openfoam wrapper), but not when to terminate. The termination must happen at the same time step for all of them.

@sunethwarna also agrees on this. You can discuss it in person with him.

[matekelemen]

The solvers can be notified through the solver wrapper when to disconnect (and this already happens with the current openfoam wrapper), but not when to terminate.

I don't get what the difference between disconnecting and terminating is from the perspective of CoSim. It makes very little sense to me that a solver keeps on doing time steps after it has disconnected from CoSim.

Once a solver gets an order to disconnect, it should clean up, do its post-processing or whatever, and terminate.

If you rely on all solvers having a predetermined time at which they terminate,

• you'll never be able to robustly ensure that all solvers actually stop at the same time, because you have no control over how they compute time, and thus have no control over how they accumulate errors. Your tolerance might work for your current case, but I guarantee it does not for others. Plus there's no telling what the right tolerance is in advance.
• You'll never be able to do conditional termination. For example, think of running FSI Mok until it becomes steady, with some check for steadiness. With your current approach, you can only do that if you have apriori an upper bound on the time it takes the system to reach steady state.

[juancamarotti]

@philbucher what's your opinion on this?

[juancamarotti]

The solvers can be notified through the solver wrapper when to disconnect (and this already happens with the current openfoam wrapper), but not when to terminate.

I don't get what the difference between disconnecting and terminating is from the perspective of CoSim. It makes very little sense to me that a solver keeps on doing time steps after it has disconnected from CoSim.

Once a solver gets an order to disconnect, it should clean up, do its post-processing or whatever, and terminate.

If you rely on all solvers having a predetermined time at which they terminate,

• you'll never be able to robustly ensure that all solvers actually stop at the same time, because you have no control over how they compute time, and thus have no control over how they accumulate errors. Your tolerance might work for your current case, but I guarantee it does not for others. Plus there's no telling what the right tolerance is in advance.

• You'll never be able to do conditional termination. For example, think of running FSI Mok until it becomes steady, with some check for steadiness. With your current approach, you can only do that if you have apriori an upper bound on the time it takes the system to reach steady state.

If you want to implement a "better" solution, feel free to do it. This is all the time I will spend on this.

[juancamarotti]

@matekelemen could you please remove your block?

[matekelemen]

@matekelemen could you please remove your block?

I will not.

If you want a quick-and-dirty solution, go increase the end time of your OpenFOAM solver.

If you want this change, open a PR for the core AnalysisStage and convince the TC.