Skip to content

Port build scripts from Perl to Python#1053

Open
abensonca wants to merge 13 commits intomasterfrom
claude/review-perl-scripts-eGdsv
Open

Port build scripts from Perl to Python#1053
abensonca wants to merge 13 commits intomasterfrom
claude/review-perl-scripts-eGdsv

Conversation

@abensonca
Copy link
Copy Markdown
Collaborator

This PR ports the Galacticus build system scripts from Perl to Python 3, modernizing the build infrastructure while maintaining functional equivalence.

Summary

All build-related Perl scripts have been converted to Python 3 equivalents. This includes compiler output post-processing, library dependency resolution, static relinking on macOS, include file dependency tracking, OpenMP critical section enumeration, and executable discovery. A new shared Fortran parsing utility module has been created to support these scripts.

Key Changes

  • postprocess.py: Replaces postprocess.pl - remaps line numbers in preprocessed files to original sources and filters compiler warnings
  • libraryDependencies.py: Replaces libraryDependencies.pl - determines required libraries for linking executables using topological sorting
  • staticRelinker.py: Replaces staticRelinker.pl - converts dynamic library links to static equivalents on macOS
  • includeDependencies.py: Replaces includeDependencies.pl - generates Makefile rules for include file dependencies
  • enumerateOpenMPCriticalSections.py: Replaces enumerateOpenMPCriticalSections.pl - enumerates OpenMP critical sections for instrumentation
  • findExecutables.py: Replaces findExecutables.pl - discovers Fortran program files and generates executable build rules
  • libraryInterfacesDependencies.py: Replaces libraryInterfacesDependencies.pl - builds Makefile dependencies for libgalacticus
  • fortran_utils.py: New shared utility module providing Fortran source parsing functions (logical line reading with continuation handling)

Implementation Details

  • Python scripts use standard library modules (subprocess, xml.etree, graphlib) instead of Perl dependencies
  • ANSI color output detection uses sys.stdout.isatty() instead of Perl's Term::ANSIColor
  • Regex patterns preserved from original Perl implementations for compatibility
  • Makefile and CI/CD workflow updated to invoke .py scripts instead of .pl equivalents
  • Error handling and environment variable validation maintained from original implementations

https://claude.ai/code/session_01WA7SHaciLZJ1S7Nof1KU1f

claude and others added 3 commits March 28, 2026 15:20
Port the 7 scripts/build/*.pl scripts that have no dependency on
perl/Galacticus/Build/* modules to Python equivalents:

  - includeDependencies.py
  - findExecutables.py
  - staticRelinker.py
  - libraryDependencies.py
  - libraryInterfacesDependencies.py
  - enumerateOpenMPCriticalSections.py
  - postprocess.py

Also introduce python/build/ as a new subpackage on PYTHONPATH, with
fortran_utils.py providing get_fortran_line() (replacing the Perl
Fortran::Utils::Get_Fortran_Line function) for use by the two scripts
that need Fortran source parsing.

https://claude.ai/code/session_01WA7SHaciLZJ1S7Nof1KU1f
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ports Galacticus build scripts from Perl to Python 3 to modernize the build pipeline while preserving existing behavior and outputs.

Changes:

  • Added Python 3 replacements for multiple build helper scripts (postprocess, dependency generation, static relinking, OpenMP critical section enumeration).
  • Updated Makefile, CI workflow, and developer docs to invoke the new .py scripts.
  • Introduced a shared Python Fortran parsing utility module (fortran_utils.py).

Reviewed changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
scripts/build/staticRelinker.py Python replacement for macOS static relinking helper.
scripts/build/staticRelinker.pl Removed Perl implementation.
scripts/build/postprocess.py Python replacement for compiler output post-processing and line remapping.
scripts/build/postprocess.pl Removed Perl implementation.
scripts/build/libraryInterfacesDependencies.py Python generator for libgalacticus Makefile dependency rules.
scripts/build/libraryInterfacesDependencies.pl Removed Perl implementation.
scripts/build/libraryDependencies.py Python replacement for link library resolution and ordering.
scripts/build/libraryDependencies.pl Removed Perl implementation.
scripts/build/includeDependencies.py Python generator for include-file dependency Makefile rules.
scripts/build/includeDependencies.pl Removed Perl implementation.
scripts/build/findExecutables.py Python executable discovery + Makefile rule generation.
scripts/build/findExecutables.pl Removed Perl implementation.
scripts/build/enumerateOpenMPCriticalSections.py Python OpenMP critical section enumerator + outputs.
scripts/build/enumerateOpenMPCriticalSections.pl Removed Perl implementation.
python/build/fortran_utils.py New shared Fortran logical-line parsing utility.
doc/Development.tex Documentation updates to reference .py scripts (plus one index entry issue).
Makefile Switched build rules from Perl to Python scripts.
.github/workflows/cicd.yml CI updated to run Python static relinker.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

abensonca and others added 4 commits April 2, 2026 09:34
fix: Add error handling as suggested by Copilot.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…void problems with file names containing spaces or other problematic characters
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2026

Spell check warnings (click to expand)

⚠️ Possible misspelled word 'buildpath' (32 instances) in file 'doc/Development.tex'
⚠️ Possible misspelled word 'compileprof' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'exe' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pr58175' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pre' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'py' (7 instances) in file 'doc/Development.tex'

Ensures that `$(BUILDPATH)/libgalacticus` is built before it needs to be scanned for module import dependencies.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 3, 2026

Spell check warnings (click to expand)

⚠️ Possible misspelled word 'buildpath' (32 instances) in file 'doc/Development.tex'
⚠️ Possible misspelled word 'compileprof' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'exe' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pr58175' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pre' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'py' (7 instances) in file 'doc/Development.tex'

This is important as double slash, `//`, can lead to file name/dependency mismatches, breaking the build.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

Spell check warnings (click to expand)

⚠️ Possible misspelled word 'buildpath' (32 instances) in file 'doc/Development.tex'
⚠️ Possible misspelled word 'compileprof' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'exe' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pr58175' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pre' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'py' (7 instances) in file 'doc/Development.tex'

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 6, 2026

Spell check warnings (click to expand)

⚠️ Possible misspelled word 'buildpath' (32 instances) in file 'doc/Development.tex'
⚠️ Possible misspelled word 'compileprof' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'exe' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pr58175' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pre' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'py' (7 instances) in file 'doc/Development.tex'

This avoids converting, for example, `-lgslcblas` to `/usr/local/lib/libgsl.acblas`.
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 6, 2026

Spell check warnings (click to expand)

⚠️ Possible misspelled word 'buildpath' (32 instances) in file 'doc/Development.tex'
⚠️ Possible misspelled word 'compileprof' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'exe' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pr58175' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'pre' in file 'doc/Development.tex'
⚠️ Possible misspelled word 'py' (7 instances) in file 'doc/Development.tex'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants