the Convergence condition

[zhaowb12138]

Dear SSCHA developers,
About the Convergence condition, I have some questions. I have seen the FAQs part. But I still have some doubts.
Firse, about [minim.meaningful_factor], how should i set it ? In "PbTe spectral functions tutorial",meaningful_factor = 1e-12.But in " Tutorial for quantum variable cell relaxation of LaH10", minim.meaningful_factor = 0.2.
I have tried different values. And have the following results.
minim.meaningful_factor = 0.2
When SSCHA converged,it print
[Free energy = -1089404.56860448 +- 8.47313583 meV
FC gradient modulus = 535.20615236 +- 3630.57224650 bohr^2
Struct gradient modulus = 0.00000000 +- 2.95575374 meV/A]

And I make minim.meaningful_factor = 0.0000005
When SSCHA converged,it print
[Free energy = -1089409.86916801 +- 7.67132381 meV
FC gradient modulus = 0.00098243 +- 2195.60030305 bohr^2
Struct gradient modulus = 0.00000000 +- 2.29607332 meV/A]
So, here is my second question. The answer in " How do I check if my calculations are well converged? " says, "In general, if the gradient goes to zero and the Kong Liu ratio is above 0.5 probably your calculation converged very well." I confuse about the " the gradient goes to zero ." How small is good enough? 1e-2 or 1e-12.
And the third question, according to the answer in " How do I understand if the free energy hessian calculation is converged? ". I split my ensemble in two, and I print the hessian frequencies about the two ensembles. How small the difference in the frequencies means the hessian matrix is good enough.

Here are my results, is it good enough？
Best wishes,
Wenbo Zhao

[diegomartinez2]

Hi Zhao,
For the first question, the 'meaningful_factor' is related to the stopping criteria. If the 'meaningful_factor' times the stochastic error is lower than the calculated gradient during the minimization, then the code will end. For its value you have to weight the precision you want with the calculation time.

[zhaowb12138]

Thank you for your reply. Yeah, I'm asking these questions because I found in my output file the code end at FC gradient modulus = 535.20615236. I think maybe it is not mean "goes to zero". And I'm afraid if I set minim.meaningful_factor smaller it will take too much time.Even in this precision the code took almost two weeks.

I do my work with vc_relax, it's there any suggestions for speeding up?
I read my output file and found because the pressure has not reached the target pressure,so it keep relax the lattice and calculate a big ensemble.

[diegomartinez2]

In my opinion, the precision kind of goes asymptotic with the number of configurations in the ensemble. You can speed up calculations by reducing the number of configurations in the ensemble to get 'closer' and then finish with a proper calculation.
How many configurations per ensemble are you using?

[zhaowb12138]

Thank you very much for your patience.
It is the output file.

out.zip

In this work at the end of the code.
[Free energy = -1051812.30846232 +- 1.47159644 meV
FC gradient modulus = 3.49837290 +- 786.13558339 bohr^2
Struct gradient modulus = 0.12234322 +- 0.71745553 meV/A
Kong-Liu effective sample size = 945.491192420082]
And there are the parameters I use
[N_configs = 1000
minim.min_step_dyn = 0.05
minim.min_step_struc = 0.05
minim.kong_liu_ratio = 0.5
minim.gradi_op = "all"
minim.meaningful_factor = 0.2]

[diegomartinez2]

As Ion says, increase the 'minim.min_step_dyn' and 'minim.min_step_struc' to make the calculation converge faster. Also 'N_configs = 1000' is OK, you can also calculate with 500 and compare results on the hessian.

[diegomartinez2]

For the second question. First note that the Kong-Liu ratio defines how 'good' is the current ensemble. As the calculation goes during the minimization process, the ensemble no longer describes well the probability distribution and a new ensemble needs to be obtained. So for a good minimization you have both a 'good' ensemble that describes the probability distribution and a pair of gradients (with respect to the trial centroids and harmonic matrix) that are lower than the threshold defined by 'min_step_struc' and 'min_step_dyn'. The threshold is re-defined (scaled) by the user provided variable 'meaningful_factor'.

On the "How small is good enough? 1e-2 or 1e-12", I cannot provide you with a specific answer.

[mesonepigreco]

Thanks diego for replying,
just a simple fix:
min_step_struc and min_step_dyn are deprecated from the line_minimization (v1.1) and they regulate the size of the step in each minimization, not the convergence, which is enteirly decided by meaningful_factor.

[ionerrea]

In order to speed up the minimization probably you need to increase the steps in the steepest descent minimization increasing the parameter min_step_dyn to something close to 1.

Bringing the gradient to 0 (~1e-6) is a guarantee that your auxiliary phonons are not updating anymore. This is good. However, as the method is stochastic, this does not mean that this is the solution. The best practice is to take one half of your population, repeat the minimization, and see if the final converged result agree with the result obtained with the full population.

Diego is right about the Hessian. We usually plot the value of the Hessian eigenvalues as a function of the number of configurations and see if the eigenvalues are converged with the number of configurations.

[mesonepigreco]

Hi Ion, by analyzing the output submitted, the code takes only 6-7 minutes to run (~2.3 seconds per step and converge after overall 166 steps), all the rest of the time is spent in the DFT calculations on a cluster connected remotely, which are 1000 over 5 populations.
The step size in this case is already optimized by the line_minimization

[mesonepigreco]

Hi, As others answered, your calculation is converging very well. Especially the variable cell optimization.
You can look how the Stress tensor evolves in the different population with respect to the target pressure you set up:

$   cat out.dat | grep TENS -A3 -a

The result is

==== STRESS TENSOR [GPa] ==== 
     11.88165629     -0.00000000     -0.00000000                0.04203554      0.00000000      0.00000000
     -0.00000000     11.88165629     -0.00000000    +-          0.00000000      0.04203554      0.00000000
     -0.00000000     -0.00000000     11.88165629                0.00000000      0.00000000      0.04203554
--
 ==== STRESS TENSOR [GPa] ==== 
     11.45394201     -0.00000000     -0.00000000                0.03885907      0.00000000      0.00000000
     -0.00000000     11.45394201     -0.00000000    +-          0.00000000      0.03885907      0.00000000
     -0.00000000     -0.00000000     11.45394201                0.00000000      0.00000000      0.03885907
--
 ==== STRESS TENSOR [GPa] ==== 
     11.01669993     -0.00000000     -0.00000000                0.04443646      0.00000000      0.00000000
     -0.00000000     11.01669993      0.00000000    +-          0.00000000      0.04443646      0.00000000
     -0.00000000      0.00000000     11.01669993                0.00000000      0.00000000      0.04443646
--
 ==== STRESS TENSOR [GPa] ==== 
     10.29507241      0.00000000      0.00000000                0.03177844      0.00000000      0.00000000
      0.00000000     10.29507241      0.00000000    +-          0.00000000      0.03177844      0.00000000
      0.00000000      0.00000000     10.29507241                0.00000000      0.00000000      0.03177844
--
 ==== STRESS TENSOR [GPa] ==== 
     10.00361037     -0.00000000     -0.00000000                0.02655959      0.00000000      0.00000000
     -0.00000000     10.00361037     -0.00000000    +-          0.00000000      0.02655959      0.00000000
     -0.00000000     -0.00000000     10.00361037                0.00000000      0.00000000      0.02655959
--
 ==== STRESS TENSOR [GPa] ==== 
     10.00361037     -0.00000000     -0.00000000                0.02655959      0.00000000      0.00000000
     -0.00000000     10.00361037     -0.00000000    +-          0.00000000      0.02655959      0.00000000
     -0.00000000     -0.00000000     10.00361037                0.00000000      0.00000000      0.02655959

Which you see that the starting point is 11.8 GPa (with quantum fluctuations) and ends to 10.004 +- 0.02 GPa Which is converged with respect your target pressure of 10 GPa.
Then, both the gradiends are converged:

Minimization ended after 166 steps

Free energy = -1051812.30846232 +-       1.47159644 meV
FC gradient modulus =       3.49837290 +-     786.13558339 bohr^2
Struct gradient modulus =       0.12234322 +-       0.71745553 meV/A
Kong-Liu effective sample size =  945.491192420082

The values of FC and Struct gradients both are lower than the error.

This means that your results for the free energy and first derivatives are well converged.
Regarding the time, I think here the issue is the cluster that runs the DFT calculations, as the code is taking about 2.5 second at each step, and it converged after 166 steps = between 6 and 7 minutes.
You are already using the line-minimization (default from version 1.1) so setting min_step_dyn and min_step_struc will not change the time.

Moreover, your code stopped with Kong-Liu at 945 effective configurations (out of 1000 I think), which means that, even if you decrease the meaningful factor to 1e-6, it will most likely remain as it is and not go outside the stochastic threshold any more (therefore not increasing significantly your time).

[mesonepigreco]

Regarding the question about the hessian, the test you made gives you an idea of the error. Take the smallest ones for example (excluding the 3 acoustic modes): You have 1.77e-3 Ry with 1000 configurations, and 1.66e-3 Ry with 500 configurations (if I understood correctly), therefore you can expect a 6 % error in your final results (1.77-1.66)/1.77 ~ 0.06
If you plot this frequency at 250, 500, and 1000 configurations, you can have a more quantitative idea of how it is converging, and if this error is acceptable for your purposes, then you are done :)

[zhaowb12138]

Thanks to all of you's answers I think I may get it now.
[About FC gradient is not zero]
I consider that if it's because a good convergence only needs a good ensemble. For an ensemble, there are two parameters to judge whether it is good enough, the number of configurations and Kong-Liu effective sample size.
The number of configurations bigger, we have higher precision. Kong-Liu effective sample size means the number of effective configurations after updating the free energy compared to the previous CG step. So if KL=945/1000 at convergence like in my work. That means that this ensemble can describe the physical properties of the system pretty well. So in this situation, we do not need FC gradient is exactly zero. I am sorry but I don't know much about Stochastic calculation, so I have no idea if I understand it right.
[About the test of convergence]
May there be no problem with my previous way. But I should do it with more points and plot a picture like this?

[About speed up calculation]
I don't want use 1000 configurations, it will take many time to wait the first output. And for some system may not require as much configurations. I understand that for situations where lattice optimization is not required, we can use a small configuration to converge and then use a large configuration to improve the accuracy. But for vc_relax I'm wondering if using a small configuration would make it to the target pressure. Maybe it's a good idea to use a larger minim.min_step_struc. I don't quite understand your interpretation of minim.min_step_struc.

[zhaowb12138]

Sorry I make a mistake in the x label .It's not "Step", it's "the number of configurations"

[mesonepigreco]

Hi,
Thanks to your feedback, it helps us to improve the documentation.

Regarding the vc-relax:
You can use a very little number of configurations. In some of my calculations, I'm able to converge with populations of just 50 configurations, and then I just do a final step with 800 or 1000 to compute the Hessian matrix (which as you can see from the plot, requires a higher number of configs to converge properly as it is a linear response property). Indeed, this really depends on the system, if you have a lot of atoms in the primitive cell, or few symmetries, it may be impossible to converge with such a little value of configurations, and you need more. Probably, in your case, already with 100 or 200 configurations you will be able to converge the VC-relax with an error lower than 1kbar (0.1 GPa).

Regarding the Free energy Hessian:
Here I think that at least 1000 configurations are needed, you can see that you have imaginary (shown as negative) frequencies if you use 250 or 500 configurations, but they disappear when you go to 1000 configurations. However, you can just use one last calculation with this many configurations, no need to do the full vc-relax using so many configs.

Regarding the FC gradient to zero:
Yes, you get it right.

[zhaowb12138]

I get it. Thank you again for your reply and advice.