Blocking MPI calls

[msimberg]

There are some places in the code base that still use blocking MPI calls. These have been used for simplicity and/or performance, but come with some risks since the underlying pika worker thread is blocked, and may lead to deadlocks. This issue lists the remaining calls (in miniapps and the main library, not the tests) so that we are aware of what's left and if anything needs to be prioritized to be changed to asynchronous communication.

• max_norm: call to reduce (

  DLA-Future/include/dlaf/auxiliary/norm/mc.h

  Lines 91 to 92 in 0be8cc9

  	dlaf::comm::sync::reduce(comm_grid.rankFullCommunicator(rank), comm_grid.fullCommunicator(), MPI_MAX,
  	make_data(&local_max_value, 1), make_data(&max_value, 1));

  )
  • called by (gen) eigensolver, cholesky miniapps for results checking
• reduce and broadcast calls in the cholesky miniapp
• allReduceInPlace in reduction to band algorithm (

  DLA-Future/include/dlaf/eigensolver/reduction_to_band/impl.h

  Lines 608 to 610 in 0be8cc9

  	comm::sync::allReduceInPlace(communicator, MPI_SUM,
  	common::make_data(x0_and_squares.data(),
  	to_SizeType(x0_and_squares.size())));

  and

  DLA-Future/include/dlaf/eigensolver/reduction_to_band/impl.h

  Line 680 in 0be8cc9

  comm::sync::allReduceInPlace(pcomm.get(), MPI_SUM, common::make_data(w[0].data(), pt_cols));

  )
• MPI_Barrier calls in miniapps (many, but not all removed in Add miniapps as tests #1112)

Please edit or comment if you find calls that I've missed.

[albestro]

#1221 addressed max_norm