The calculation is unable to locate the required pseudopotential

[Prutthi]

Dear expert SSCHA,

I'm interested in utilizing SSCHA in conjunction with Quantum Espresso codes. I've followed the script example for H3S and PbTe, particularly in the context of using vc-relax within Quantum Espresso based on a primitive cell. At this stage, the job was executed successfully. It's worth noting that I employed the command "harmonic_dyn = CC.Phonons.compute_phonons_finite_displacements(structure_relaxed, calculators, supercell = (4,4,4))."

However, during the subsequent calculation with the supercell structure, the process was prematurely terminated, and an error message was encountered: "Error in routine readpp (1): file ./Sc.pbe-spn-kjpaw_psl.1.0.0.UPF not found." This suggests that the calculation is unable to locate the required pseudopotential. I should emphasize that this calculation was carried out on a high-performance supercomputer, and my slurm file indicates the following error: "ase.calculators.calculator.CalculationFailed: Calculator 'espresso' failed with the command 'pw.x -in disp_0.pwi > disp_0.pwo', and it failed in the directory /test4/disp_0 with error code 1." Subsequent to this issue, it appears that the SSCHA relaxation process has not yet commenced. Here is my concise script, as shown below:

# We import the basis modules for the SSCHA
import cellconstructor as CC
import cellconstructor.Structure
import cellconstructor.Phonons

# Import the SSCHA engine (we will use it later)
import sscha, sscha.Ensemble, sscha.SchaMinimizer, sscha.Relax


pseudos = {"Sc" : "Sc.pbe-spn-kjpaw_psl.1.0.0.UPF",
                    "H" : "H.pbe-kjpaw_psl.1.0.0.UPF"}
input_params = {"tstress" : True, # Print the stress in the output
                "tprnfor" : True, # Print the forces in the output
                "ecutwfc" : 35,  #The wavefunction energy cutoff for plane-waves (Ry)
                "ecutrho" : 350, # The density energy cutoff (Ry)
                "mixing_beta" : 0.2,  # The mixing parameter in the self-consistent calculation
                "conv_thr" : 1e-9,    # The energy convergence threshold (Ry)
                "degauss" : 0.02,  # Smearing temperature (Ry)
                "smearing" : "mp",
                "pseudo_dir" : ".",
                "occupations" : "smearing",
                "disk_io" : "none"}

k_spacing = 0.07 #A^-1 The minimum distance in the Brillouin zone sampling

structure = CC.Structure.Structure()
structure.read_generic_file("ScH.cif")

# We override (or add) the key for the calculation type
input_params["calculation"] = "vc-relax"
# Specify the command to call quantum espresso
#command = 'mpprun -np 16 pw.x -npool 16 < espresso.pwi > espresso.pwo'
# Generate once again the Espresso calculator
calculators = Espresso(input_data = input_params, pseudopotentials = pseudos, kspacing = k_spacing)

# Relax the gold structure (useless since for symmetries it is already relaxed)

# Relax the structure (not necessary as it's already relaxed due to symmetries)
relax = CC.calculators.Relax(structure, calculators)
structure_relaxed = relax.static_relax()

# Compute the harmonic phonons
# Note: When running with mpirun, the calculation is parallelized
harmonic_dyn = CC.Phonons.compute_phonons_finite_displacements(structure_relaxed, calculators, supercell=(4, 4, 4))

# Apply the sum rule and remove imaginary modes
harmonic_dyn.Symmetrize()
harmonic_dyn.ForcePositiveDefinite()

# Generate the ensemble and minimizer objects
ensemble = sscha.Ensemble.Ensemble(dyn, 1000, supercell=dyn.GetSupercell())
minimizer = sscha.SchaMinimizer.SSCHA_Minimizer(ensemble)

# Set up minimization parameters
minimizer.min_step_dyn = 0.001
minimizer.kong_liu_ratio = 0.5

# Prepare for automatic relaxation
relax = sscha.Relax.SSCHA(minimizer, ase_calculator=espresso_calc, N_configs=200, max_pop=6, save_ensemble=True)

In light of these challenges, I would greatly appreciate guidance from an expert on how to address this issue with the SSCHA code. Your assistance would be invaluable.

Best,

Prutthipong

[mesonepigreco]

Hi, this error is releated to quantum espresso.
It means that the espresso cannot locate the pseudopotential file:

Sc.pbe-spn-kjpaw_psl.1.0.0.UPF

Which is expected to be located in the same directory in which the script is running.
Have you copied correctly the pseudopotentials in your directory?

[Prutthi]

Hello, Certainly, I have placed the pseudopotentials in the same directory. When I initiate a calculation based on the unit cell, the pseudopotentials are used in the computation. Following the completion of the calculation, I've noticed the creation of a subdirectory named "disp_0." It's worth noting that the "disp_0" subdirectory contains "disp_0.pwi" and "disp_0.pwo" files. Additionally, in "disp_0.pwi," I can see the following lines:

ATOMIC_SPECIES
Sc 44.955908 Sc.pbe-spn-kjpaw_psl.1.0.0.UPF
H 1.008 H.pbe-kjpaw_psl.1.0.0.UPF

This suggests that Quantum Espresso should be able to locate the pseudopotential file.

Nevertheless, the calculation, as indicated in "disp_0.pwo," is abruptly halted due to the unavailability of the file "./Sc.pbe-spn-kjpaw_psl.1.0.0.UPF.

Best,
Prutthipong

[mesonepigreco]

Ok, you are right.
So the problem is that, to compute the harmonic phonons, the code is generating the disp_x directories and performing the espresso calculation inside those directories (which does not contain the pseudopotential).

The solution is to change the value of pseudo_dir in the input to the absolute path of the calculation:

input_params = {"tstress" : True, # Print the stress in the output
                "tprnfor" : True, # Print the forces in the output
                "ecutwfc" : 35,  #The wavefunction energy cutoff for plane-waves (Ry)
                "ecutrho" : 350, # The density energy cutoff (Ry)
                "mixing_beta" : 0.2,  # The mixing parameter in the self-consistent calculation
                "conv_thr" : 1e-9,    # The energy convergence threshold (Ry)
                "degauss" : 0.02,  # Smearing temperature (Ry)
                "smearing" : "mp",
                "pseudo_dir" : "/ABSOLUTE/PATH/TO/THE/PSEUDOPOTENTIAL/DIR",   # <-   EDIT THIS LINE
                "occupations" : "smearing",
                "disk_io" : "none"}

This should solve the issue

[Prutthi]

I deeply appreciate your prompt response and your continued support. I have diligently followed your guidance, and it indeed seemed to address the issue.

Regrettably, it appears that a new problem has arisen during the calculation. After specifying the path to the directory containing the pseudopotentials, I noticed that the "disp_0" subdirectory now only contains a single file, "disp_0.pwi," and the calculation was subsequently terminated. To provide further context, my slurm-file displays the error message "/bin/sh: espresso.pwi: No such file or directory," and it concludes with the final message "ase.calculators.calculator.CalculationFailed: Calculator 'espresso' failed with the command 'mpprun -np 16 pw.x -npool 16 < espresso.pwi > espresso.pwo'."

Currently, I have defined the command for invoking Quantum Espresso as follows:
command = 'mpprun -np 16 pw.x -npool 16 < espresso.pwi > espresso.pwo'

Additionally, I'm unsure whether this might be a factor in the calculation issue, but I noticed that the output.log file shows the following message:

STATIC STRUCTURE RELAX

ITERS ENERGY FORCE GRAD

Number of symmetry inequivalent displacements: 2

As indicated in the message, the calculation was abruptly terminated.

It's worth mentioning that the initial calculation, based on the unit cell, generated the files "espresso.pwi" and "espresso.pwo," while the subsequent calculation with the supercell structure only produced "disp_0.pwi" in the "disp_0" subdirectory.

P.S. The script should be explicitly specified, as in the following example: "TEMPERATURE = 300, N_CONFIGS = 50, MAX_ITERATIONS = 20, NQIRR = 4."

I would greatly appreciate any insights or assistance you can provide in resolving this matter.

[mesonepigreco]

Hi,
You should not modify explicitly the command of the espresso calculator.
Here the problem is that you assumed the name of the file to be espresso.pwi, but the code generates disp_0.pwi (where 0 changes identify the id of the displacement computed)

ASE has a way to exploit mpi parallelization that does not involve explicitly overriding the command:
https://wiki.fysik.dtu.dk/ase/ase/calculators/espresso.html

Alternatively mi, you can use the PREFIX.pwi and PREFIX.pwo in the command name to refer to the input and output file of the espresso calculation

[Prutthi]

Hello,

Thank you once again. I will give it a try. By the way, I attempted to close the tag with the following command: "command = 'mpprun -np 16 pw.x -npool 16 < espresso.pwi > espresso.pwo.'" It appears that the process successfully located pseudopotentials. However, after this initial calculation, the subsequent process, the SSCHA process, encountered an issue. The slurm-file indicates the following error:

Traceback (most recent call last):
File "/SSCHA/ScH-test6/sscha_gold.py", line 54, in
dyn = CC.Phonons.Phonons('harmonic_dyn', nqirr = 3)
File "/miniconda2/envs/sscha/lib/python3.10/site-packages/cellconstructor/Phonons.py", line 126, in init
self.LoadFromQE(structure, nqirr, full_name = full_name, use_format = use_format)
File "/miniconda2/envs/sscha/lib/python3.10/site-packages/cellconstructor/Phonons.py", line 200, in LoadFromQE
raise ValueError("Error, file %s does not exist." % filepath)
ValueError: Error, file harmonic_dyn1 does not exist.

As above mentioned, I am actively seeking a solution to this issue. I have a suspicion that there could be a problem with my script after completing the supercell calculations, obtained from the QE code. If you could provide assistance in resolving it, I would greatly appreciate your help. I would like to extend my apologies for reaching out to you multiple times.

[Prutthi]

Hello,

I have effectively accomplished the task of saving the dynamical matrix in the Quantum Espresso format. Although I have encountered a minor issue, I believe I can resolve it. If I have any questions, I will promptly keep you informed.

[Prutthi]

Hello, I need to follow the example provided in this link: https://wiki.fysik.dtu.dk/ase/ase/calculators/espresso.html. However, it appears that I'm unable to execute the calculation with the input file, as it's displaying a minimum input like this:

# Specify the command to call Quantum Espresso
espresso_profile = EspressoProfile(("mpprun -np 8 pw.x -npool 1 -i PREFIX.pwi > PREFIX.pwo"))
# command = 'mpirun -np 8 pw.x -npool 1 -i PREFIX.pwi > PREFIX.pwo'

structure = CC.Structure.Structure()
structure.read_generic_file("ScH.cif")

# We override (or add) the key for the calculation type
input_params["calculation"] = "vc-relax"
# Generate the Espresso calculator again
calculators = Espresso(input_data=input_params, pseudopotentials=pseudos, profile=espresso_profile, kspacing=k_spacing)
However, I'm encountering an issue resulting in the following traceback:

Traceback (most recent call last):
  File "/SSCHA/ScH-test7/sscha_gold.py", line 10, in <module>
    from ase.calculators.espresso import Espresso, EspressoProfile
ImportError: cannot import name 'EspressoProfile' from 'ase.calculators.espresso' (/miniconda2/envs/sscha/lib/python3.10/site-packages/ase/calculators/espresso.py)
Program finished with exit code 1 at: Fri Nov 3 16:04:15 CET 2023

When I use a single node, the process runs successfully, but after the harmonic_dyn step, it fails during the sscha_T300_dyn. Can you please provide some guidance on how to resolve this issue?

[mesonepigreco]

Hi,
This is an ASE error. You may need a newer version of ASE installed to use the EspressoProfile, but for further indications, contact ASE developers.

In general, when writing on this forum, use the '```' syntax to report code blocks as it makes the message more readable ('#' is used as a heading as in the markdown syntax and messes the display completely).

I know that executing Python on a cluster could be very hard.
I suggest employing the Cluster module to set remote calculations and run the SSCHA code locally; see the tutorial:
https://sscha.eu/Tutorials/tutorial_02_advanced_submission/

In particular, this mode was meant explicitly to deal with all these issues on the cluster.

In general, however, try to think of a solution for more minor problems not directly related to the sscha (like missing files or library versions)
Also, consider opening new discussions for new problems, as the forum is meant to guide everyone with common problems and how to solve them, so each issue should be searchable from the title separately.