Use MPI_Dist_graph_create[_adjacent] + MPI_Neighbor_allgather for PCU rounds. This will not account for the buffering that PCU does (PCU_COMM_PACK and friends), but handles the dynamic sparse data exchange.
See: https://github.com/ibaned/omega_h/blob/develop/src/Omega_h_comm.cpp
and the slides here for a summary:
https://www.archer.ac.uk/training/course-material/2016/09/160929_AdvMPI_EPCC/S2-L05-MPI-Neighbourhood%20collectives.pdf
Use
MPI_Dist_graph_create[_adjacent]+MPI_Neighbor_allgatherfor PCU rounds. This will not account for the buffering that PCU does (PCU_COMM_PACKand friends), but handles the dynamic sparse data exchange.See: https://github.com/ibaned/omega_h/blob/develop/src/Omega_h_comm.cpp
and the slides here for a summary:
https://www.archer.ac.uk/training/course-material/2016/09/160929_AdvMPI_EPCC/S2-L05-MPI-Neighbourhood%20collectives.pdf