Research Notebook

A batteries-included scientific research project template for reproducible experiments.

Features

• 🐍 Python 3.12+ with modern type annotations
• 📦 Pixi for reproducible conda-forge environments across platforms
• 📊 DVC for data versioning and ML pipeline management
• ⚙️ Hydra + hydra-zen for type-safe configuration management
• 📓 JupyterLab with LSP, Git integration, and MyST rendering
• 📓 Jupyter notebooks — executed .ipynb files committed with outputs, rendered directly by MyST
• 🌊 Marimo reactive notebook environment
• 📚 MyST-MD for publication-quality documentation
• 🔍 Ruff for fast linting and formatting
• 🔒 Pre-commit hooks for code quality enforcement
• 🚀 14 GitHub Actions workflows for CI/CD, reproducibility, and automation
• 🖥️ Cross-platform support (Linux x86-64, macOS ARM64)
• 📝 CITATION.cff for academic citations
• 🔬 Hydra multirun sweeps for hyperparameter optimization

Project Structure

research_template/
├── .github/workflows/    # 14 GitHub Actions workflows
├── .devcontainer/        # VS Code / Codespaces dev container
├── configs/              # Hydra configuration hierarchy
│   ├── train.yaml        # Main config with defaults
│   ├── model/            # Model configs (baseline, transformer)
│   └── data/             # Data configs (small, full)
├── data/                 # Data directories (DVC-managed)
│   ├── raw/
│   ├── processed/
│   └── external/
├── docs/                 # MyST documentation
├── marimo_notebooks/     # Marimo reactive notebooks
├── notebooks/            # Jupyter notebooks (.ipynb, MyST-rendered)
├── results/              # Experiment results (DVC-managed)
├── scripts/              # Entry point scripts
├── src/research_notebook/        # Source package
│   ├── data/             # Data loading utilities
│   ├── models/           # Model implementations
│   ├── trainers/         # Training loops
│   └── utils/            # Utility functions
└── tests/                # Test suite

Quick Start

Prerequisites

Install Pixi:

curl -fsSL https://pixi.sh/install.sh | bash

Installation

git clone https://github.com/jejjohnson/research_notebook
cd research_notebook
pixi install

Run an Experiment

# Preprocess data
pixi run preprocess

# Train with default config
pixi run train

# Train with parameter overrides
pixi run train training.lr=0.001 model=transformer

# Evaluate
pixi run evaluate

Environments

Environment	Features	Use Case
default	dev	Testing, linting, training
docs	docs	Building MyST documentation
jupyterlab	dev + jupyterlab	Interactive notebooks
marimo	dev + marimo	Reactive notebooks

# Activate specific environment
pixi run -e jupyterlab lab
pixi run -e marimo marimo-edit
pixi run -e docs docs-build

Configuration Management

This template uses both Hydra and hydra-zen for configuration management.

Classic Hydra (YAML-based)

# Use default config
pixi run train

# Override parameters
pixi run train training.lr=0.001 training.epochs=50

# Use different model config
pixi run train model=transformer

# Hyperparameter sweep
pixi run train -m training.lr=0.001,0.01,0.1

hydra-zen (Python-based, type-safe)

pixi run train-zen

hydra-zen eliminates YAML boilerplate by defining configs as Python dataclasses:

from hydra_zen import builds, make_config, launch

ModelConfig = builds(BaselineModel, hidden_size=64, num_layers=2)
ExperimentConfig = make_config(model=ModelConfig, seed=42)

Experiment Tracking

This template uses DVC for data and experiment tracking.

# Add data to DVC
dvc add data/raw/dataset.csv

# Run the full pipeline
dvc repro

# Check pipeline status
dvc status

# View pipeline DAG
dvc dag

# Compare metrics across experiments
dvc metrics diff

The DVC pipeline is defined in dvc.yaml:

1. preprocess: Processes raw data → data/processed/
2. train: Trains model → results/metrics/train_metrics.json

Documentation

Documentation is built with MyST-MD using the book theme.

# Serve locally with live reload
pixi run -e docs docs-serve

# Build static HTML
pixi run -e docs docs-build

Docs are automatically deployed to GitHub Pages on every push to main.

Notebook Environments

Notebooks are committed as executed .ipynb files under notebooks/, with cell outputs embedded. MyST renders them directly in the docs site (no conversion step), so figures and prose stay together in a single source of truth. Jupytext is still available for optional .py ↔ .ipynb pairing if you prefer cleaner diffs during editing.

JupyterLab

Full-featured JupyterLab with LSP, Git integration, MyST rendering, and spell checking:

pixi run -e jupyterlab lab

Marimo

Reactive, reproducible notebooks in pure Python:

pixi run -e marimo marimo-edit

Marimo notebooks in marimo_notebooks/ are stored as .py files, making them diff-friendly and importable as regular Python modules.

CI/CD Workflows

Workflow	Trigger	Description
ci.yml	push/PR	pytest on ubuntu + macos
lint.yml	push/PR	Ruff linting
typecheck.yml	push/PR	ty type checking
pages.yml	push to main	Build + deploy MyST docs
dvc-check.yml	DVC file changes	Validate DVC pipeline
notebooks.yml	notebook changes	Validate .ipynb structure via nbformat
reproducibility.yml	weekly schedule	Full dvc repro
experiment-report.yml	PR	DVC metrics diff comment
citation.yml	CITATION.cff changes	Validate citation file
pixi-update.yml	monthly schedule	Update pixi lockfile
codeql.yml	push/PR/schedule	Security scanning
conventional-commits.yml	PR	Validate PR title
label-pr.yml	PR	Auto-label PRs
pre-commit-autoupdate.yml	weekly schedule	Update pre-commit hooks

Academic

Citation

If you use this template, please cite it using the metadata in CITATION.cff:

@software{johnson2026researchnotebook,
  author = {Johnson, Juan Emmanuel},
  title  = {Research Notebook},
  year   = {2024},
  url    = {https://github.com/jejjohnson/research_notebook},
}

References

Add BibTeX references to references.bib. They are automatically available in MyST documentation and Jupyter notebooks with the jupyterlab-myst extension.

Zenodo

Connect your GitHub repository to Zenodo for automatic DOI assignment on releases.

Acknowledgments

This template was inspired by:

• jejjohnson/pypackage_template — library-focused Python package template
• DrivenData Cookiecutter Data Science
• Hydra documentation
• DVC documentation