Skip to content

Advanced Features

Jump to bottom
Abdulkadir Özcan edited this page Mar 23, 2026 · 1 revision

Advanced Features

This page covers the optional features that sit on top of the core optimisation loop: dynamic bandwidth narrowing, crash recovery via checkpoints, evaluation caching, CSV logging, callback functions, and early stopping. All of these are configured through keyword arguments to .optimize() and have sensible defaults, so you can ignore them until you need them.

Bandwidth narrowing

The pitch adjustment step in Harmony Search perturbs a continuous value by a small amount — the bandwidth. harmonix decays this bandwidth exponentially from bw_max at iteration 0 to bw_min at the final iteration, following:

bw(t) = bw_max × exp(−ln(bw_max / bw_min) × t / T)

where t is the current iteration and T is max_iter. Early iterations explore broadly; late iterations converge precisely. Only Continuous variables are affected — Discrete, Integer, and Categorical variables always step by their natural unit regardless of bandwidth.

Parameters

Parameter Default Meaning
bw_max 0.05 Initial bandwidth as a fraction of domain width. 0.05 = 5% of hi − lo.
bw_min 0.001 Final bandwidth. Must be ≤ bw_max and > 0. 
result = Minimization(space, objective).optimize(
    max_iter = 10_000,
    bw_max   = 0.10,    # start wide — 10% of domain
    bw_min   = 0.0005,  # finish tight — 0.05% of domain
)

Guidance

  • Set bw_max == bw_min to disable decay and use a constant bandwidth (original HS behaviour).
  • For large, rugged landscapes increase bw_max to 0.10–0.20 so early iterations can escape local optima.
  • For fine-grained convergence on smooth problems decrease bw_min to 0.0001 or lower.
  • Both values must be strictly positive. Passing zero raises a ValueError.

Resume and checkpoints

Long runs can be interrupted by crashes, time limits, or manual stops. harmonix saves the full harmony memory to a JSON file at regular intervals so a run can be continued exactly where it left off.

Parameters

Parameter Default Meaning
checkpoint_path None Path to the JSON checkpoint file. Set this to enable checkpointing.
checkpoint_every 500 Save a checkpoint every this many iterations.
resume "auto" One of "auto", "new", or "resume" — see below.

The three resume modes

Mode Behaviour When to use
"auto" Resumes if the checkpoint file exists and is non-empty; starts fresh otherwise. Safe default for most situations — run the same script repeatedly without thinking about it.
"new" Always starts fresh. Overwrites any existing checkpoint file. When you want to guarantee a clean run regardless of previous state.
"resume" Always resumes. Raises FileNotFoundError if the checkpoint does not exist. When continuation is mandatory and a missing checkpoint should be an error, not a silent restart. 
result = Minimization(space, objective).optimize(
    max_iter         = 50_000,
    checkpoint_path  = "run.json",
    checkpoint_every = 1000,
    resume           = "auto",
)

How it works

The initial harmony memory is saved immediately at startup — before the first improvisation step. Even a run interrupted in the first few seconds can be resumed cleanly. Each subsequent checkpoint overwrites the previous one; only the latest state is kept.

The checkpoint file stores the complete harmony memory (all harmonies, fitness values, and penalties) and the last completed iteration index. It does not store the random state, so a resumed run will follow a different random trajectory — but from the same starting memory.

For MultiObjective, the Pareto archive is also serialised and restored alongside the harmony memory.

Evaluation cache

When use_cache=True, harmonix wraps the objective function in an LRU cache. Identical harmonies — same variable values, same order — are never evaluated twice. The cached result is returned immediately.

Parameters

Parameter Default Meaning
use_cache False Enable the evaluation cache.
cache_maxsize 4096 Maximum number of entries. Oldest entries are evicted when full (LRU policy). 
result = Minimization(space, objective).optimize(
    max_iter     = 30_000,
    use_cache    = True,
    cache_maxsize = 8192,
)

# Inspect cache performance after the run
print(optimizer._cache.stats())
# EvaluationCache: 4821 hits / 30000 total (16.1% hit rate)  size=4096/8192

When the cache helps

  • Expensive objectives — FEM simulations, CFD calls, database queries. Even a 5% hit rate can save significant wall time.
  • Late-stage convergence — when the harmony memory has largely converged, the same few harmonies are repeatedly considered. Hit rates of 30–50% are common in this phase.
  • Discrete-heavy spaces — variables with small finite domains produce repeated harmonies more often than purely continuous spaces.

When the cache does not help

  • Purely continuous spaces with wide domains — the probability of sampling exactly the same float twice is negligible. The cache overhead (hashing the harmony dict on every call) costs more than it saves.
  • Very cheap objectives — if evaluation takes microseconds, the cache adds more overhead than it removes.

CSV logging

harmonix can write three separate CSV log files during a run. All are readable directly in Excel or with pandas.read_csv(). Log file paths are derived automatically from checkpoint_path unless you specify them explicitly.

The three log files

Flag Auto path Columns Written
log_init=True run_init.csv harmony_index, all variable names, fitness, penalty, feasible Once, after memory initialisation
log_history=True run_history.csv iteration, best_fitness, best_penalty, feasible, all variable names prefixed best_ Every history_every iterations
log_evaluations=True run_evals.csv wall_time_s, iteration, all variable names, fitness, penalty, feasible Every evaluation 
result = Minimization(space, objective).optimize(
    max_iter         = 10_000,
    checkpoint_path  = "run.json",   # log paths derived from this
    log_init         = True,         # → run_init.csv
    log_history      = True,         # → run_history.csv
    log_evaluations  = True,         # → run_evals.csv
    history_every    = 50,           # write history every 50 iterations
)

You can also specify log paths explicitly:

result = optimizer.optimize(
    log_history      = True,
    history_log_path = "results/convergence.csv",
    history_every    = 10,
)

Guidance

  • log_init is useful for debugging — inspect the starting population to verify your space is sampling the right region.
  • log_history is the most useful for post-run analysis. Plot best_fitness vs iteration to see the convergence curve. Set history_every to a larger value (e.g. 100) to keep file size manageable on long runs.
  • log_evaluations records every harmony evaluated, not just the best. On a 30,000-iteration run this produces a 30,000-row file. Use it for detailed statistical analysis, not routine monitoring.

Callback and early stopping

A callback function is called at the end of every iteration and receives the current iteration index and a partial OptimizationResult (or ParetoResult for multi-objective runs). You can use it to monitor progress, log to an external system, or stop the run early.

def my_callback(iteration, partial_result):
    print(f"iter {iteration:>6d} | best = {partial_result.best_fitness:.6g}")

result = Minimization(space, objective).optimize(
    max_iter = 10_000,
    callback = my_callback,
)

Early stopping

Raise StopIteration inside the callback to terminate the run cleanly. The optimiser catches it and returns the best result found so far.

def early_stop(iteration, partial_result):
    if partial_result.best_fitness < 1e-6:
        raise StopIteration   # target reached — stop now

    if partial_result.best_penalty > 0 and iteration > 5000:
        raise StopIteration   # still infeasible after 5000 iters — give up

result = Minimization(space, objective).optimize(
    max_iter = 50_000,
    callback = early_stop,
)
print(f"Stopped at iteration {result.iterations}")

External progress bars

from tqdm import tqdm

bar = tqdm(total=10_000)

def update_bar(iteration, partial_result):
    bar.set_postfix(fitness=f"{partial_result.best_fitness:.4g}")
    bar.update(1)

result = Minimization(space, objective).optimize(
    max_iter = 10_000,
    callback = update_bar,
)
bar.close()

Things to avoid in callbacks

  • Do not modify the partial_result object — it is a live view of the optimiser state.
  • Avoid slow operations (file I/O, network calls) unless necessary — the callback is on the critical path of every iteration.
  • Do not raise any exception other than StopIteration for early stopping. Other exceptions will propagate and abort the run without returning a result.

Verbose mode

Set verbose=True to print the best fitness and penalty to stdout at every iteration. Useful for quick interactive monitoring without writing a callback.

result = Minimization(space, objective).optimize(
    max_iter = 1000,
    verbose  = True,
)
# [HS] iter      1 | fitness = 18.4321 | penalty = 0.0000
# [HS] iter      2 | fitness = 15.7842 | penalty = 0.0000
# ...

For production runs or Jupyter notebooks, prefer a callback with tqdm or a custom logger over verbose=True — the per-iteration print adds measurable overhead on fast objectives.

Combining features

All features compose freely. A typical long production run might look like:

from harmonix import Minimization

def on_iteration(it, res):
    if res.best_fitness < TARGET:
        raise StopIteration

result = Minimization(space, objective).optimize(
    max_iter         = 100_000,
    hmcr             = 0.90,
    par              = 0.40,
    bw_max           = 0.08,
    bw_min           = 0.0005,
    resume           = "auto",
    checkpoint_path  = "long_run.json",
    checkpoint_every = 2000,
    use_cache        = True,
    cache_maxsize    = 8192,
    log_history      = True,
    history_every    = 100,
    callback         = on_iteration,
)

See also

Clone this wiki locally