feat(memory): trend history as splayed tables (Layer 3, v0.7.0)

Materialize the .raysense/trends/history.json time-series as three
splayed baseline tables (trend_health, trend_hotspots,
trend_violations). Schema bumped 3 -> 4. Older v3 baselines need
re-save; v1 trend samples remain readable but contribute zero rows
to the long tables.

history.json gains schema v2: top_hotspots (capped at 20 by
risk_score) and rule_breakdown per sample. Old v1 samples
deserialize via #[serde(default)] and roll forward on the next
baseline save.

raysense_trend MCP tool generalized: window (7d|30d|90d|all),
dimension (health|hotspots|violations|all), format
(summary|table|json), limit (now actually wired).

raysense_drift MCP tool added: ranks regressions across the window,
returning worsened dimensions, hotspots new or risen, and rules
newly tripped or with increased counts.

claude-plugin gains drift skill alongside bootstrap/impact/verify/
audit/query. Wraps rescan -> drift -> remediations.

Author: singaraiona

.gitignore

@@ -1,5 +1,6 @@
 CLAUDE.md
 .claude/
+.raysense/
 /target/
 Cargo.lock
 /local/

Cargo.toml

@@ -21,7 +21,7 @@
 
 [package]
 name = "raysense"
-version = "0.6.2"
+version = "0.7.0"
 edition = "2021"
 license = "MIT"
 repository = "https://github.com/RayforceDB/raysense"

README.md

@@ -118,8 +118,9 @@ Rayfall expressions, drop in custom rules as `.rfl` files, and bring
 external CSVs into the same query substrate.
 
 ```bash
-# 1. Save a baseline with 18 splayed tables (files, functions, calls,
-#    call_edges, imports, types, hotspots, change_coupling, ...)
+# 1. Save a baseline with 21 splayed tables (files, functions, calls,
+#    call_edges, imports, types, hotspots, change_coupling,
+#    trend_health, trend_hotspots, trend_violations, ...)
 raysense baseline save .
 
 # 2. Ad-hoc query in Rayfall (LISP-like, prefix, arity-strict).
@@ -161,9 +162,10 @@ Raysense ships as a Claude Code plugin:
 /plugin install raysense
 ```
 
-Five skills: scan + baseline at session start, blast radius before
-edits, regression diff after, on-demand architecture audits, and a
-Rayfall query skill that lets agents run free-form expressions
+Six skills: scan + baseline at session start, blast radius before
+edits, regression diff after, on-demand architecture audits,
+time-window drift detection ("what got worse over the last 30 days"),
+and a Rayfall query skill that lets agents run free-form expressions
 against the saved baseline tables when the typed tools fall short.
 Multi-codebase isolation is cwd-driven, so per-project state stays in
 `<repo>/.raysense/`. Two sessions on two repos = two independent
@@ -196,9 +198,15 @@ baselines, zero cross-project bleed.
   agent edit is most likely to break. Composite of churn, max
   complexity, single-owner penalty, and missing-tests penalty,
   refreshed on every save
-- **Score drift per session** - every baseline save appends a sample;
-  verify diffs against the previous one and surfaces per-dimension
-  drift (Equality went B to D) instead of a single aggregate delta
+- **Drift over time** - every baseline save and `trend record`
+  appends a sample to `.raysense/trends/history.json`. The save
+  materializes three splayed tables (`trend_health`,
+  `trend_hotspots`, `trend_violations`) so agents query "what got
+  worse over the last 30 days" through Rayfall, the typed
+  `raysense_drift` MCP tool, or the `drift` skill, instead of
+  parsing history.json themselves. Verify still surfaces
+  per-dimension regressions against the saved baseline (Equality B
+  to D) for the no-time-window case
 - **Bug-density per file** - files where most of the churn is fix
   commits float to the top. Conventional Commits prefixes (fix,
   hotfix, revert) drive the classifier; absolute count and ratio

claude-plugin/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "raysense",
-  "version": "0.3.0",
-  "description": "Architectural telemetry for AI coding agents. Phase-scoped skills over the raysense MCP server: scan + baseline at session start, blast-radius before edits, regression diff after, on-demand architecture audits.",
+  "version": "0.4.0",
+  "description": "Architectural telemetry for AI coding agents. Phase-scoped skills over the raysense MCP server: scan + baseline at session start, blast-radius before edits, regression diff after, on-demand architecture audits, time-window drift detection.",
   "author": {
     "name": "Anton Kundenko",
     "email": "anton.kundenko@gmail.com"

claude-plugin/skills/drift/SKILL.md

@@ -0,0 +1,80 @@
+---
+name: drift
+description: Use after a rescan to surface structural regressions across a time window. Diffs the latest scan against the saved baseline AND the trend history, ranking dimensions that worsened, files newly hot, and rules newly tripped. Configurable window (7d, 30d, 90d). Use periodically (daily, weekly, or pre-PR).
+---
+
+# Drift
+
+`drift` answers one question: "what got worse since N days ago?"
+It is heavier than `verify` (which only diffs against the saved
+baseline) but lighter than `audit` (which sweeps the whole repo).
+Run it when the question is shaped like "are we slowly losing
+modularity?" or "did a bad pattern creep in over the last sprint?"
+
+All tools take a `path` argument; pass the current repo root as an
+absolute path.
+
+## Steps
+
+1. **Rescan**. Call `raysense_rescan` with `path: <cwd>` so the
+   active health is current.
+2. **Drift summary**. Call `raysense_drift` with `path: <cwd>` and
+   `window: 30d` (the default). Returns:
+   - `worsened_dimensions`: the dimensions whose scores dropped
+     (or rule count rose) across the window.
+   - `hotspots_new_or_risen`: files that newly entered the top
+     hotspots or whose `risk_score` climbed.
+   - `rules_new_or_increased`: rule codes that newly tripped or
+     whose violation count grew.
+3. **Trend context**. When `drift` reports `available: false`
+   (fewer than 2 samples in the window), call `raysense_trend` with
+   `window: all` so the user sees how short the history is. Suggest
+   they call `raysense_baseline_save` to seed a sample.
+4. **Remediations on regression**. For each entry in
+   `rules_new_or_increased`, call `raysense_remediations` and
+   surface the suggestion alongside the regression.
+
+## What to surface to the user
+
+A good drift report leads with the worst regression, not the full
+list. Three focused lines beat one wall of metrics:
+
+- "Modularity dropped 0.92 to 0.78 (worst dimension this window)."
+- "src/big.rs is the new top hotspot (risk_score 50 to 216)."
+- "Rule `max_function_complexity` newly tripped (0 to 2)."
+
+If `drift` returns nothing in any of the three categories, say so
+plainly: "No drift detected in the last 30d." Do not pad with empty
+sections.
+
+## Window choice
+
+- `7d`: catches regressions from the current week's edits.
+- `30d`: default. Sees a typical sprint's worth of structural
+  movement.
+- `90d`: quarterly review cadence. Often spans refactors that
+  haven't fully settled.
+- `all`: every recorded sample. Use when you want the full arc.
+
+Drift compares the *oldest* in-window sample to the *newest*
+in-window sample. Wider windows give bigger deltas but blur acute
+regressions.
+
+## When to skip
+
+- Fewer than 2 samples in the trend history. The skill will return
+  `available: false` and the report will be empty. Run
+  `raysense_baseline_save` first to seed history.json.
+- The user asked "what's broken right now?" That is a `verify` or
+  `audit` question, not a drift question.
+
+## See also
+
+- `verify`: snapshot diff against the session baseline (no time
+  axis). Use after a focused chunk of edits.
+- `audit`: whole-repo structural sweep, no time axis. Use when the
+  question is shape, not change.
+- `raysense_baseline_query`: the `query` skill covers Rayfall
+  directly. The splayed `trend_health`, `trend_hotspots`, and
+  `trend_violations` tables are queryable from there for custom
+  drift analyses (per-dimension regressions, file-specific arcs).

src/cli.rs

@@ -32,7 +32,7 @@ use crate::{
 use anyhow::{anyhow, Context, Result};
 use clap::{Parser, Subcommand};
 use serde_json::{json, Value};
-use std::collections::BTreeSet;
+use std::collections::{BTreeMap, BTreeSet};
 use std::fs;
 use std::path::{Path, PathBuf};
 use std::process;
@@ -2731,11 +2731,21 @@ fn record_trend(root: &Path, config_path: Option<&Path>) -> Result<()> {
     Ok(())
 }
 
+/// On-disk schema version for `history.json` samples. Bumped to 2 in
+/// v0.7 to add `top_hotspots` and `rule_breakdown`. Older samples
+/// without the `schema` field deserialize as `0` and are read by
+/// downstream consumers as v1.
+const TREND_SCHEMA_VERSION: u32 = 2;
+
+/// How many temporal hotspots to embed per trend sample. Capped to
+/// keep `history.json` from growing unbounded with churny repos.
+const TREND_TOP_HOTSPOTS_LIMIT: usize = 20;
+
 /// Append one row to `<root>/.raysense/trends/history.json` describing
 /// the score, quality signal, rule count, and per-dimension scores at
 /// `now`. Used both by `raysense trend record` and by `raysense
 /// baseline save`. Returns the path the row was written to.
-fn append_trend_sample(report: &ScanReport, health: &HealthSummary) -> Result<PathBuf> {
+pub(crate) fn append_trend_sample(report: &ScanReport, health: &HealthSummary) -> Result<PathBuf> {
     let dir = report.snapshot.root.join(".raysense/trends");
     let path = dir.join("history.json");
     fs::create_dir_all(&dir).with_context(|| format!("failed to create {}", dir.display()))?;
@@ -2744,7 +2754,27 @@ fn append_trend_sample(report: &ScanReport, health: &HealthSummary) -> Result<Pa
     } else {
         Vec::new()
     };
+    let top_hotspots: Vec<Value> = health
+        .metrics
+        .evolution
+        .temporal_hotspots
+        .iter()
+        .take(TREND_TOP_HOTSPOTS_LIMIT)
+        .map(|h| {
+            serde_json::json!({
+                "path": h.path,
+                "commits": h.commits,
+                "max_complexity": h.max_complexity,
+                "risk_score": h.risk_score,
+            })
+        })
+        .collect();
+    let mut rule_breakdown: BTreeMap<String, usize> = BTreeMap::new();
+    for finding in &health.rules {
+        *rule_breakdown.entry(finding.code.clone()).or_default() += 1;
+    }
     samples.push(serde_json::json!({
+        "schema": TREND_SCHEMA_VERSION,
         "timestamp": unix_time(),
         "snapshot_id": report.snapshot.snapshot_id,
         "score": health.score,
@@ -2754,6 +2784,8 @@ fn append_trend_sample(report: &ScanReport, health: &HealthSummary) -> Result<Pa
         "functions": report.functions.len(),
         "root_causes": health.root_causes,
         "overall_grade": health.grades.overall,
+        "top_hotspots": top_hotspots,
+        "rule_breakdown": rule_breakdown,
     }));
     fs::write(&path, serde_json::to_string_pretty(&samples)?)
         .with_context(|| format!("failed to write {}", path.display()))?;
@@ -2871,6 +2903,15 @@ fn save_baseline(root: &Path, output: &Path, config_path: Option<&Path>) -> Resu
     let report = scan_path_with_config(root, &config)?;
     let health = compute_health_with_config(&report, &config);
     let baseline = build_baseline(&report, &health);
+
+    // Record the trend sample first so this snapshot is part of the
+    // history that the splayed `trend_*` tables read from. Failures
+    // here are non-fatal: the trend log is best-effort, the baseline
+    // itself is what the user asked for.
+    if let Err(reason) = append_trend_sample(&report, &health) {
+        eprintln!("warning: failed to record trend sample: {reason}");
+    }
+
     let memory = crate::memory::RayMemory::from_report_with_config(&report, &config)?;
     let tables_dir = output.join("tables");
 
@@ -2889,13 +2930,6 @@ fn save_baseline(root: &Path, output: &Path, config_path: Option<&Path>) -> Resu
         .save_splayed(&tables_dir)
         .with_context(|| format!("failed to write baseline tables {}", tables_dir.display()))?;
 
-    // Record a trend sample so the score-drift series captures every
-    // baseline save automatically. Failures here are non-fatal: the
-    // baseline itself succeeded, the trend log is best-effort.
-    if let Err(reason) = append_trend_sample(&report, &health) {
-        eprintln!("warning: failed to record trend sample: {reason}");
-    }
-
     Ok(())
 }
 
@@ -3545,4 +3579,43 @@ mod tests {
         assert!(summary.skipped.is_empty());
         fs::remove_dir_all(&root).unwrap();
     }
+
+    #[test]
+    fn append_trend_sample_writes_v2_schema() {
+        // Run the full pipeline against this crate so we get a real
+        // ScanReport + HealthSummary. Redirect the trend file into a
+        // temp dir by overriding the snapshot root post-scan.
+        let root = temp_root("trend-v2");
+        fs::create_dir_all(&root).unwrap();
+        let mut report = crate::scan_path(env!("CARGO_MANIFEST_DIR")).unwrap();
+        // Point the writer at the temp dir, but keep the rest of the
+        // report intact so health computation has real data to chew on.
+        report.snapshot.root = root.clone();
+        let health = crate::compute_health(&report);
+
+        let path = append_trend_sample(&report, &health).unwrap();
+        let content = fs::read_to_string(&path).unwrap();
+        let samples: Vec<Value> = serde_json::from_str(&content).unwrap();
+        assert_eq!(samples.len(), 1);
+        let sample = &samples[0];
+        assert_eq!(sample["schema"].as_u64(), Some(TREND_SCHEMA_VERSION as u64));
+        assert!(
+            sample.get("top_hotspots").is_some(),
+            "v2 sample must carry top_hotspots field"
+        );
+        assert!(
+            sample.get("rule_breakdown").is_some(),
+            "v2 sample must carry rule_breakdown field"
+        );
+        // Cap is enforced.
+        let hotspots = sample["top_hotspots"].as_array().unwrap();
+        assert!(
+            hotspots.len() <= TREND_TOP_HOTSPOTS_LIMIT,
+            "expected <= {} hotspots, got {}",
+            TREND_TOP_HOTSPOTS_LIMIT,
+            hotspots.len()
+        );
+
+        fs::remove_dir_all(&root).unwrap();
+    }
 }