Issue with STLSQ/SR3, weakPDELibrary, and coefficient interpretability

[xskvarkai]

Hello everyone,

I'm encountering some difficulties while using Pysindy, specifically with the STLSQ and SR3 optimizers in conjunction with weakPDELibrary, and I'm hoping for some guidance.

Here's a summary of my problem:

I'm currently working with an ODE system, but I've chosen to use weakPDELibrary because I anticipate extending this work to PDEs in the future. Moreover, I've found weakPDELibrary to be very effective in filtering noise, which is crucial for my data.

1. When I use STLSQ with weakPDELibrary, I obtain physically interpretable coefficients. However, the resulting equations are incorrect. I believe this could be resolved by applying specific constraints, for example, if I know that dx0 = x1.
2. To address this, I attempted to switch to constraintSR3. With constraintSR3, I get coefficients that are orders of magnitude too large and are no longer physically interpretable.
3. I tried setting the nu parameter to the smallest possible value, but this did not resolve the issue; I observed no significant change in the coefficient magnitudes.
4. Further investigation led me to change normalize_columns from True to False. This change did affect the results, but it introduced a new problem: my system became ill-conditioned.
5. While I could potentially solve the ill-conditioned problem using MinMax normalization, I prefer to avoid this approach because it scales the data, making the coefficients lose their physical interpretability. I also want to avoid using MIOSR, even though it might be an option.

My goal is to obtain correct equations with physically interpretable coefficients. Has anyone faced similar issues, or could you provide insights into how to handle such large coefficients with constraintSR3 or overcome the ill-conditioning while maintaining physical interpretability?

Any advice would be greatly appreciated!

Thank you.

[royalasdaircoding]

Hi,

While I haven't used constrainedSR3 much, as I understand you have to be slightly careful about normalising when using constraints. For example, if you have x=y in your original data scaling, then if you normalise the data this equality will no longer be true or would have to be rescaled accordingly. I don't think SINDy does this automatically.

If your problem is ill-conditioned then it depends what your input data looks like. If you have a lot of time-series or they're all highly correlated in time then it could be tricky to get the correct the true equations out, even with constraints.

Also in my experience using weak SINDy you need to choose the integration window size (H_xt) depending on specific time-scales of your data. For instance, if it's too large you will simply average over all the interesting dynamics of your system and not get the correct equations.

You might want to share the exact equations you're aiming for and someone could probably comment on what issues might arise. Sorry I can't really offer much on the Constrained SR3 side of things, but I have found constraining with MIOSR to be conceptually simpler as you don't have to choose parameters like $\nu$.

[Jacob-Stevens-Haas]

I echo what @royalasdaircoding mentioned around what the system you're trying to fit, and also about normalization with constraints. With the new BINDy object, #680 asks for a way for optimizers to know about normalization that BaseOptimizer() calculates, which in the future would allow ConstrainedSR3 to use normalization to adjust the constraints.

• If you're getting good coefficients with STLSQ and not with ConstrainedSR3, you might try to see if SR3 works well to troubleshoot if the issue is the optimizer form or the constraints.
• Some optimizer parameters (e.g. relax_coeff_nu) affect internal fitting behavior and can be tricky to set unless you look at the verbose output.
• As @royalasdaircoding mentioned, what system are you trying to fit? If it's an unknown system, sparsity may not be as important as predictive ability. If it is a known system, MIOSR is a really effective way to validate that the rest of the SINDy algorithm is set up as well as it could be.
• What does physically unrealistic mean in your case? Is it just that dx0 != x1, or are there other things that are wrong?
• Other things can go wrong with regression - see Problem while increasing the polynomial degree in the polynomial library #677