Questions about cluster calculation

[nintendo-64]

Dear developers,

I'm a learner of your newly released SSCHA code. After running the H3S example, I have a few questions about the cluster calculation:

1. In cluster calculation, I can choose in how many batch of jobs I want to submit simultaneously, and how many configurations for each job using given parameters. However, I already log into a cluster and want to run the code on the same one, which parameter should I use to achieve the same effect?
2. It seems that the code uses 'SBATCH' for 'SLURM' cluster submission, where and how can I change the default for other systems such as 'LSF' using 'BSUB'?

It will be greatly appreciated if you could spare time to answer my questions.

Yours
Lu Qing

[mesonepigreco]

Hi,
The SSCHA code works in three steps:

1. It extracts an ensemble of random ionic displaced configurations (according to the dynamical matrix)
2. It computes energies and forces for the extracted configurations
3. It performs the free energy optimization and gives in output a new dynamical matrix that represents the density matrix of the system.

These operations are iterated.
The easiest way to do this is to run these steps separately as explained in the PbTe example. This is system independent and you can run on any PC.
Usually, the SSCHA code takes few runs to converge.

I recommend doing everything manually, as we do not have yet implemented an automatic submission with queue systems different from SLURM.

However, if you really need to do automatic calculations (you have really a lot of calculations to run simultaneously and you cannot follow all of them) it could be possible to run the code automatically in your case with a bit of work:

If you run the code directly on the same cluster used for the force and energy calculations and want to do everything automatically, you can try to fake the code, using localhost as the hostname for the connection. In this way, the code will submit the calculation on the same computer on which it is running.
To change the submission from SLURM to any other process, you can change the name of the variables of the Cluster class. This is quite advanced, and I have never done it. You can have a look at the Cluster.py class, and see if you can obtain the desired effect by changing the values of the variables.

for example

my_hcp = sscha.Cluster.Cluster(...)
my_hcp.submit_name="SBATCH" # Change with the other keyword
my_hcp.submit_command = "sbatch --wait" # Change with the command for submitting the calculatio

In any case, the submitting command should remain pending until the end of the job (this is why the sscha code uses the --wait option of SLURM). You should try to translate these commands to match your queue system.

[mesonepigreco]

You are welcome.

Regarding to private/public key autentication, there is a workaround.
The Cluster class supports the password connection if you pass the password keyword when you create the Cluster.
In this case you need to have installed sshpass in the system running the sscha code.
This is not an orthodox solution, as you need to write your password clearly in the python script, but it is a workaround I adopted for clusters that do not support private/public key authentications.

Bests,
Lorenzo Monacelli

[cs2t]

Dear Lorenzo,

I have a follow-up question on the automatic submission on a Cluster.
After the calculations have been submitted to the cluster based on the automatic procedure, is there a way to resume the calculations if the jupyter notebook gets disconnected? Or, in this case, would it be more appropriate to recover the calculations following the "manual" procedure of the iteration?

Thank you in advance.

Sincerely,
Fabio Caruso

[mesonepigreco]

Dear Fabio,
Unfortunately not, the only way is to collect the calculation performed in the cluster and start the first population with the manual procedure. Then you can continue again with the automatic one starting from the next step.

However, even if it could be boring to manually restart the calculation, this is a good way to better control the minimization and to minimize the computational time spent on HPC. This automatic submission is a quite new feature of the code: in almost all the paper we published with SSCHA before 2020, we always used the manual procedure.

[cs2t]

Thank you, Lorenzo!
Best,
Fabio

[nintendo-64]

Dear Lorenzo Monacelli,

After making the comparision, I decide to run it manually for controllability. Many thanks!

Yours,
Lu Qing