Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion doc/src/physics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
.. license in this material to reproduce, prepare derivative works, distribute copies to
.. the public, perform publicly and display publicly, and to permit others to do so.
.. =======================================================================================

.. This file was created in part or in whole by one of OpenAI's generative AI models
.. _physics:

Physics Modules
Expand All @@ -28,6 +28,7 @@ In general, to activate a physics capability, it must be set in the ``<physics>`
conduction = false
cooling = false
gravity = false
self_gravity = false
nbody = false
rotating_frame = false
radiation = false
Expand Down Expand Up @@ -750,6 +751,46 @@ Available options are:

See `N-Body Dynamics`_ for a description of how to set up the N-body system.

Self Gravity
------------

The ``<self_gravity>`` node enables self-consistent gravitational forces for fluid components by solving the Poisson equation

.. math::

\nabla^2 \phi = 4 \pi G \rho_{\mathrm{total}},

where :math:`\rho_{\mathrm{total}}` is the sum of the mass densities of all fluids and :math:`G` is the gravitational constant in code units. The right-hand side is assembled from all registered fluid densities, and the gravitational potential :math:`\phi` is then used to compute gravitational forces in the hydrodynamic update.

The self-gravity module currently supports Cartesian geometries only.

The parameter stored internally is ``four_pi_G = 4 \pi G``. By default, :math:`G` is taken from the code constants and converted to code units. This behavior can be overridden by setting

.. code-block:: ini

<self_gravity>
units_override = true
four_pi_G = 1.0

When ``units_override`` is enabled, the supplied value of ``four_pi_G`` is used instead of the value derived from the code constants.

The gravitational potential currently supports periodic and zero (Dirichlet) boundary conditions. For fully periodic domains, the Poisson equation is solvable only if the mean density is removed. This is handled using the Jeans swindle, which subtracts the domain-averaged density before solving. If all mesh boundaries are periodic, ``use_swindle`` is automatically required and the code will abort if it is disabled. If ``use_swindle`` is enabled in a configuration that is not fully periodic, a warning is issued.

Zero Dirichlet boundary conditions for :math:`\phi` may be specified per face in the ``<self_gravity>`` block, e.g.,

.. code-block:: ini

<self_gravity>
ix1_bc = zero
ox1_bc = zero

Only faces explicitly set to ``zero`` enroll a zero-potential boundary function; otherwise, the mesh-level boundary configuration (e.g., periodic) applies.

The self-gravity package provides two primary fields: the gravitational potential ``grav::phi`` and the Poisson right-hand side ``grav::rhs``. ``grav::rhs`` corresponds to :math:`4 \pi G \sum_i \rho_i` and is constructed in the package’s ``FillDerivedMesh`` routine, including application of the Jeans swindle when enabled.

The Poisson equation is registered through Parthenon’s linear solver framework. The solver consists of a BiCGSTAB Krylov method preconditioned by a multigrid (MG) solver.


Rotating Frame
--------------

Expand Down
2 changes: 1 addition & 1 deletion external/parthenon
Submodule parthenon updated 95 files
+2 −2 .github/workflows/apply-formatting.yml
+2 −2 .github/workflows/check-formatting.yml
+2 −2 .github/workflows/ci-extended.yml
+10 −1 .github/workflows/ci-macos.yml
+55 −13 CHANGELOG.md
+39 −3 CMakeLists.txt
+2 −0 cmake/Format.cmake
+101 −1 doc/sphinx/src/boundary_communication.rst
+1 −0 doc/sphinx/src/building.rst
+3 −0 doc/sphinx/src/generated/diffusion-parth-table.csv
+38 −50 doc/sphinx/src/interface/sparse.rst
+7 −0 doc/sphinx/src/particles.rst
+44 −0 doc/sphinx/src/sparse_packs.rst
+8 −0 example/diffusion/CMakeLists.txt
+6 −0 example/poisson/CMakeLists.txt
+7 −0 example/poisson_gmg/CMakeLists.txt
+2 −2 example/poisson_gmg/parthenon_app_inputs.cpp
+8 −18 example/poisson_gmg/poisson_equation.hpp
+7 −4 example/poisson_gmg/poisson_package.cpp
+2 −0 example/sparse_advection/parthinput.sparse_advection
+1 −1 scripts/docker/Dockerfile.nvcc
+31 −7 scripts/python/packages/parthenon_tools/parthenon_tools/movie2d.py
+6 −2 src/CMakeLists.txt
+13 −11 src/amr_criteria/amr_criteria.cpp
+12 −0 src/basic_types.hpp
+4 −0 src/bvals/boundary_conditions.cpp
+9 −7 src/bvals/bvals.cpp
+2 −2 src/bvals/bvals.hpp
+4 −2 src/bvals/comms/bnd_id.hpp
+17 −11 src/bvals/comms/bnd_info.cpp
+6 −2 src/bvals/comms/bnd_info.hpp
+7 −6 src/bvals/comms/boundary_communication.cpp
+85 −62 src/bvals/comms/build_boundary_buffers.cpp
+17 −2 src/bvals/comms/bvals_in_one.hpp
+32 −25 src/bvals/comms/bvals_utils.hpp
+12 −12 src/bvals/comms/tag_map.cpp
+0 −1 src/bvals/neighbor_block.cpp
+2 −2 src/defs.hpp
+4 −1 src/driver/driver.cpp
+8 −0 src/driver/driver.hpp
+6 −2 src/interface/mesh_data.hpp
+1 −0 src/interface/meshblock_data.hpp
+23 −1 src/interface/metadata.hpp
+29 −0 src/interface/state_descriptor.hpp
+2 −1 src/interface/swarm.cpp
+6 −3 src/interface/swarm.hpp
+27 −20 src/interface/swarm_comms.cpp
+3 −77 src/interface/update.cpp
+4 −67 src/interface/update.hpp
+6 −10 src/interface/variable.cpp
+7 −1 src/interface/variable.hpp
+19 −5 src/kokkos_abstraction.hpp
+5 −4 src/mesh/mesh-amr_loadbalance.cpp
+29 −5 src/mesh/mesh-gmg.cpp
+87 −37 src/mesh/mesh.cpp
+55 −16 src/mesh/mesh.hpp
+45 −0 src/mesh/mesh_neighbors.hpp
+1 −1 src/mesh/mesh_refinement.cpp
+1 −2 src/mesh/meshblock.cpp
+45 −13 src/mesh/meshblock.hpp
+1 −0 src/outputs/output_parameters.hpp
+5 −1 src/outputs/outputs.cpp
+4 −4 src/outputs/parthenon_hdf5.cpp
+2 −2 src/pack/block_selector.hpp
+113 −0 src/pack/scratch_variables.hpp
+10 −0 src/pack/sparse_pack.hpp
+11 −9 src/pack/sparse_pack_base.cpp
+1 −1 src/parameter_input.cpp
+1 −0 src/parthenon/package.hpp
+93 −56 src/solvers/bicgstab_solver.hpp
+86 −20 src/solvers/internal_prolongation.hpp
+150 −41 src/solvers/mg_solver.hpp
+18 −0 src/solvers/solver_base.hpp
+62 −10 src/solvers/solver_utils.hpp
+178 −0 src/sparse/sparse_management.cpp
+53 −0 src/sparse/sparse_management.hpp
+109 −0 src/tasks/task_timing.cpp
+113 −0 src/tasks/task_timing.hpp
+12 −0 src/tasks/tasks.cpp
+20 −0 src/tasks/tasks.hpp
+25 −2 src/tasks/thread_pool.hpp
+0 −160 src/utils/buffer_utils.cpp
+0 −52 src/utils/buffer_utils.hpp
+6 −6 src/utils/index_split.cpp
+29 −19 src/utils/loop_utils.hpp
+106 −2 src/utils/object_pool.hpp
+72 −0 src/utils/summable_array.hpp
+3 −3 tst/performance/test_meshblock_data_iterator.cpp
+1 −0 tst/regression/test_suites/sparse_advection/parthinput.sparse_advection
+2 −0 tst/unit/CMakeLists.txt
+17 −0 tst/unit/test_kokkos_abstraction.cpp
+75 −0 tst/unit/test_object_pool.cpp
+139 −0 tst/unit/test_scratch_variables.cpp
+6 −4 tst/unit/test_swarm.cpp
+258 −0 tst/unit/test_tasklist.cpp
102 changes: 102 additions & 0 deletions inputs/self-gravity/poly.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# ========================================================================================
# (C) (or copyright) 2026. Triad National Security, LLC. All rights reserved.
#
# This program was produced under U.S. Government contract 89233218CNA000001 for Los
# Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
# for the U.S. Department of Energy/National Nuclear Security Administration. All rights
# in the program are reserved by Triad National Security, LLC, and the U.S. Department
# of Energy/National Nuclear Security Administration. The Government is granted for
# itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide
# license in this material to reproduce, prepare derivative works, distribute copies to
# the public, perform publicly and display publicly, and to permit others to do so.
# ========================================================================================

<artemis>
problem = polytrope # name of the pgen
coordinates = cartesian # coordinate system

<parthenon/job>
problem_id = polytrope # problem ID: basename of output filenames

<parthenon/output1>
file_type = hdf5 # HDF5 data dump
variables = gas.prim.density, &
gas.prim.velocity, &
gas.prim.pressure, &
grav.phi, &
grav.rhs
dt = 1.0 # time increment between outputs

<parthenon/output2>
file_type = hst # HDF5 data dump
dt = 1.0 # time increment between outputs

<parthenon/time>
nlim = -1 # cycle limit
tlim = 100.0 # time limit (to be reset by problem generator)
integrator = rk2 # time integration algorithm
ncycle_out = 1 # interval for stdout summary info

<parthenon/mesh>
# do_coalesced_comms = true
refinement = adaptive
numlevel = 3
multigrid = true

nx1 = 64
x1min = -32.0
x1max = 32.0
ix1_bc = outflow
ox1_bc = outflow

nx2 = 64
x2min = -32.0
x2max = 32.0
ix2_bc = outflow
ox2_bc = outflow

nx3 = 64
x3min = -32.0
x3max = 32.0
ix3_bc = outflow
ox3_bc = outflow

<parthenon/meshblock>
nx1 = 16
nx2 = 16
nx3 = 16

<parthenon/refinement0>
method = magnitude
comparator = greater_than
field = gas.prim.density_0
refine_tol = 0.1
derefine_tol = 0.1

<physics>
gas = true
self_gravity = true

<gas>
cfl = 0.9
reconstruct = plm
riemann = hllc-gamma

<gas/eos/ideal>
gamma = 2.0

<self_gravity>
print_per_step = true
max_iterations = 100
residual_tolerance = 1.0e-6
units_override = true
use_swindle = false
ix1_bc = zero
ox1_bc = zero
ix2_bc = zero
ox2_bc = zero
ix3_bc = zero
ox3_bc = zero

<problem>
iprob = 2
11 changes: 9 additions & 2 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# ========================================================================================
# (C) (or copyright) 2023-2025. Triad National Security, LLC. All rights reserved.
# (C) (or copyright) 2023-2026. Triad National Security, LLC. All rights reserved.
#
# This program was produced under U.S. Government contract 89233218CNA000001 for Los
# Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
Expand Down Expand Up @@ -43,10 +43,10 @@ set (SRC_LIST

gravity/binary_mass.cpp
gravity/gravity.cpp
gravity/gravity.hpp
gravity/nbody_gravity.hpp
gravity/point_mass.cpp
gravity/uniform.cpp
gravity/gravity.hpp

nbody/nbody.cpp
nbody/nbody.hpp
Expand All @@ -68,6 +68,7 @@ set (SRC_LIST
pgen/kh.hpp
pgen/linear_wave.hpp
pgen/lw.hpp
pgen/polytrope.hpp
pgen/pgen.hpp
pgen/problem_modifier.hpp
pgen/rt.hpp
Expand All @@ -91,6 +92,12 @@ set (SRC_LIST
rotating_frame/rotating_frame_impl.hpp
rotating_frame/rotating_frame.hpp

self_gravity/self_gravity.cpp
self_gravity/self_gravity.hpp
self_gravity/poisson_driver.cpp
self_gravity/poisson_equation.hpp
self_gravity/self_gravity.cpp

utils/artemis_utils.cpp
utils/artemis_utils.hpp
utils/history.hpp
Expand Down
7 changes: 6 additions & 1 deletion src/artemis.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//========================================================================================
// (C) (or copyright) 2023-2025. Triad National Security, LLC. All rights reserved.
// (C) (or copyright) 2023-2026. Triad National Security, LLC. All rights reserved.
//
// This program was produced under U.S. Government contract 89233218CNA000001 for Los
// Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
Expand All @@ -26,6 +26,7 @@
#include "radiation/radiation.hpp"
#include "radiation/raytrace/raytrace.hpp"
#include "rotating_frame/rotating_frame.hpp"
#include "self_gravity/self_gravity.hpp"
#include "utils/artemis_utils.hpp"
#include "utils/history.hpp"
#include "utils/units.hpp"
Expand Down Expand Up @@ -91,6 +92,7 @@ Packages_t ProcessPackages(std::unique_ptr<ParameterInput> &pin) {
const bool do_gas = pin->GetOrAddBoolean("physics", "gas", true);
const bool do_dust = pin->GetOrAddBoolean("physics", "dust", false);
const bool do_gravity = pin->GetOrAddBoolean("physics", "gravity", false);
const bool do_self_gravity = pin->GetOrAddBoolean("physics", "self_gravity", false);
const bool do_nbody = pin->GetOrAddBoolean("physics", "nbody", false);
const bool do_rotating_frame = pin->GetOrAddBoolean("physics", "rotating_frame", false);
const bool do_cooling = pin->GetOrAddBoolean("physics", "cooling", false);
Expand Down Expand Up @@ -128,6 +130,7 @@ Packages_t ProcessPackages(std::unique_ptr<ParameterInput> &pin) {
artemis->AddParam("do_gas", do_gas);
artemis->AddParam("do_dust", do_dust);
artemis->AddParam("do_gravity", do_gravity);
artemis->AddParam("do_self_gravity", do_self_gravity);
artemis->AddParam("do_nbody", do_nbody);
artemis->AddParam("do_rotating_frame", do_rotating_frame);
artemis->AddParam("do_cooling", do_cooling);
Expand Down Expand Up @@ -162,6 +165,8 @@ Packages_t ProcessPackages(std::unique_ptr<ParameterInput> &pin) {
packages.Add(geometry::Initialize(pin.get()));
if (do_nbody) packages.Add(NBody::Initialize(pin.get(), constants));
if (do_gravity) packages.Add(Gravity::Initialize(pin.get(), constants, packages));
if (do_self_gravity)
packages.Add(SelfGravity::Initialize(pin.get(), constants, packages));
if (do_gas) packages.Add(Gas::Initialize(pin.get(), units, constants, packages));
if (do_dust) packages.Add(Dust::Initialize(pin.get(), units));
if (do_rotating_frame) packages.Add(RotatingFrame::Initialize(pin.get()));
Expand Down
7 changes: 6 additions & 1 deletion src/artemis.hpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//========================================================================================
// (C) (or copyright) 2023-2025. Triad National Security, LLC. All rights reserved.
// (C) (or copyright) 2023-2026. Triad National Security, LLC. All rights reserved.
//
// This program was produced under U.S. Government contract 89233218CNA000001 for Los
// Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
Expand Down Expand Up @@ -97,6 +97,11 @@ SWARM_VARIABLE(int, rad.star, ijk);
} // namespace star
} // namespace rad

namespace grav {
ARTEMIS_VARIABLE(grav, phi);
ARTEMIS_VARIABLE(grav, rhs);
} // namespace grav

namespace geom {
ARTEMIS_VARIABLE(geom, x1v);
ARTEMIS_VARIABLE(geom, x2v);
Expand Down
18 changes: 15 additions & 3 deletions src/artemis_driver.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//========================================================================================
// (C) (or copyright) 2023-2025. Triad National Security, LLC. All rights reserved.
// (C) (or copyright) 2023-2026. Triad National Security, LLC. All rights reserved.
//
// This program was produced under U.S. Government contract 89233218CNA000001 for Los
// Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
Expand Down Expand Up @@ -32,6 +32,7 @@
#include "radiation/moments/moments.hpp"
#include "radiation/raytrace/raytrace.hpp"
#include "rotating_frame/rotating_frame.hpp"
#include "self_gravity/self_gravity.hpp"
#include "utils/integrators/artemis_integrator.hpp"

using namespace parthenon::driver::prelude;
Expand Down Expand Up @@ -61,6 +62,7 @@ ArtemisDriver<GEOM>::ArtemisDriver(ParameterInput *pin, ApplicationInput *app_in
do_gas = artemis_pkg->template Param<bool>("do_gas");
do_dust = artemis_pkg->template Param<bool>("do_dust");
do_gravity = artemis_pkg->template Param<bool>("do_gravity");
do_self_gravity = artemis_pkg->template Param<bool>("do_self_gravity");
do_rotating_frame = artemis_pkg->template Param<bool>("do_rotating_frame");
do_shear = artemis_pkg->template Param<bool>("do_shear");
do_cooling = artemis_pkg->template Param<bool>("do_cooling");
Expand Down Expand Up @@ -235,6 +237,9 @@ TaskCollection ArtemisDriver<GEOM>::StepTasks() {
const Real g1 = integrator->gam1[stage - 1];
const Real bdt = integrator->beta[stage - 1] * integrator->dt;

// Compute gravitational potential
if (do_self_gravity) SelfGravity::SolvePoisson(tc, pmesh);

TaskRegion &tr = tc.AddRegion(num_partitions);
for (int i = 0; i < num_partitions; i++) {
auto &tl = tr[i];
Expand Down Expand Up @@ -294,10 +299,17 @@ TaskCollection ArtemisDriver<GEOM>::StepTasks() {
Gravity::ExternalGravity<GEOM>, u0.get(), time, bdt);
}

TaskID rt_src = gravity_src;
// Apply self-gravity source term
TaskID self_gravity_src = gravity_src;
if (do_self_gravity) {
self_gravity_src =
tl.AddTask(gravity_src, SelfGravity::SelfGravity<GEOM>, u0.get(), time, bdt);
}

TaskID rt_src = self_gravity_src;
// Note that radiation moments will handle this source term if active
if (do_raytrace && !do_moment) {
rt_src = tl.AddTask(gravity_src, Gas::DepositEnergy, u0.get(), bdt);
rt_src = tl.AddTask(self_gravity_src, Gas::DepositEnergy, u0.get(), bdt);
}

// Apply rotating frame source term
Expand Down
4 changes: 2 additions & 2 deletions src/artemis_driver.hpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//========================================================================================
// (C) (or copyright) 2023-2025. Triad National Security, LLC. All rights reserved.
// (C) (or copyright) 2023-2026. Triad National Security, LLC. All rights reserved.
//
// This program was produced under U.S. Government contract 89233218CNA000001 for Los
// Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC
Expand Down Expand Up @@ -53,7 +53,7 @@ class ArtemisDriver : public EvolutionDriver {
IntegratorPtr_t integrator, nbody_integrator, rad_integrator;
StateDescriptor *artemis_pkg;
bool do_gas, do_dust, do_moment, do_imc;
bool do_gravity, do_nbody, do_rotating_frame, do_shear;
bool do_gravity, do_self_gravity, do_nbody, do_rotating_frame, do_shear;
bool do_cooling, do_drag, do_viscosity, do_conduction, do_diffusion;
bool do_coagulation, do_raytrace;
bool update_fluxes;
Expand Down
4 changes: 4 additions & 0 deletions src/artemis_params.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ physics:
_type: "bool"
_default: "false"
_description: "Turn on gravity"
self_gravity:
_type: "bool"
_default: "false"
_description: "Turn on self-gravity"
rotating_frame:
_type: "bool"
_default: "false"
Expand Down
2 changes: 0 additions & 2 deletions src/pgen/beam.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,6 @@ inline void BeamInnerX2(std::shared_ptr<MeshBlockData<Real>> &mbd, bool coarse)
using parthenon::MakePackDescriptor;
using TE = parthenon::TopologicalElement;
auto pmb = mbd->GetBlockPointer();
if (coarse && !ArtemisUtils::CoarseNeighbor(pmb)) return;

// Extract artemis package and params
auto artemis_pkg = pmb->packages.Get("artemis");
Expand Down Expand Up @@ -204,7 +203,6 @@ inline void BeamInnerX1(std::shared_ptr<MeshBlockData<Real>> &mbd, bool coarse)
using parthenon::MakePackDescriptor;
using TE = parthenon::TopologicalElement;
auto pmb = mbd->GetBlockPointer();
if (coarse && !ArtemisUtils::CoarseNeighbor(pmb)) return;

// Extract artemis package and params
auto artemis_pkg = pmb->packages.Get("artemis");
Expand Down
2 changes: 0 additions & 2 deletions src/pgen/conduction.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,6 @@ template <Coordinates GEOM, IndexDomain BDY, Diffusion::DiffType DTYP>
void CondBoundaryImpl(std::shared_ptr<MeshBlockData<Real>> &mbd, bool coarse) {
PARTHENON_INSTRUMENT
auto pmb = mbd->GetBlockPointer();
if (coarse && !ArtemisUtils::CoarseNeighbor(pmb)) return;

// Artemis package and params
auto artemis_pkg = pmb->packages.Get("artemis");
Expand Down Expand Up @@ -267,7 +266,6 @@ template <Coordinates GEOM, IndexDomain BDY>
inline void CondBoundary(std::shared_ptr<MeshBlockData<Real>> &mbd, bool coarse) {
PARTHENON_INSTRUMENT
auto pmb = mbd->GetBlockPointer();
if (coarse && !ArtemisUtils::CoarseNeighbor(pmb)) return;

auto artemis_pkg = pmb->packages.Get("artemis");
auto &pkg = pmb->packages.Get("gas");
Expand Down
Loading