Skip to content

VAALE_EVENT_ANALYSIS.md

Latest commit

 

History

History
548 lines (401 loc) · 24.2 KB

VAALE_EVENT_ANALYSIS.md

File metadata and controls

548 lines (401 loc) · 24.2 KB
Raw

VAALE 2026 Comprehensive Event Analysis

Team 2363 | March 7-8, 2026

About This Report

This report was generated by prompting Claude (via Claude Code) to analyze WPILOG files using the wpilog-mcp MCP server. It serves as an example of how teams can use AI-assisted log analysis for post-event debriefs.

Prompt Style Used:

Analyze all WPILOG files from the event in the logs folder.
Generate a comprehensive markdown report covering:
- Executive summary with critical findings
- Log file health (complete, truncated, corrupted)
- Match-by-match analysis (scores, code versions, CAN errors, voltage)
- Code version timeline with git commits and dirty builds
- CAN bus health with error timelines
- Battery & power analysis with brownout risk
- Vision system performance
- Loop timing and GC impact
- Device connectivity issues
- Math/runtime errors with timestamps
- Inter-match trends
- Chronological narrative of the event
- Prioritized recommendations
- Fun facts section

Write like a lead engineer debriefing the team—direct, specific,
occasionally wry. Use tables for comparisons, weave data into narrative.

The MCP server provides tools for loading logs, searching entries, computing statistics, detecting anomalies, and generating reports. Claude iterates through the logs, cross-references data, and synthesizes findings into this narrative format.

Executive Summary

Team 2363 competed at VAALE over two days, finishing qualifications at 4-4 before advancing to eliminations (reaching at least E9). The robot—a turret-based launcher with swerve drive and pneumatic intake—performed well when systems cooperated, but several recurring issues hampered consistency.

Critical Issues:

  • REV Pneumatic Hub CAN instability caused 25 timeout errors across Q8, Q68, and Q74. The PH intermittently dropped off the bus during matches, affecting solenoid control and compressor monitoring.
  • Severe voltage sag in multiple matches: Q51 hit 6.62V, Q64 reached 6.81V, and both elimination matches (E6, E9) dipped below 6.9V. The robot repeatedly flirted with brownout territory.
  • 20+ second startup freezes occurred in every match—likely JVM class loading or GC initialization. The worst was Q8 at 24.7 seconds.
  • Rotation2d zero-vector errors appeared 8 times across Q8, Q56, and E9, indicating geometry calculations receiving invalid inputs from vision or odometry edge cases.
  • Log corruption affected 12 of 13 logs—only Q56 closed cleanly. Q15 was completely unreadable, and Q68/Q74 had corrupted timestamps showing impossible durations.

What Worked:

  • Vision system remained stable with consistent observation scores (mean ~0.06) across all matches
  • All four PhotonVision cameras maintained connectivity throughout
  • Code iteration was rapid—7 builds across 4 commits in two days
  • CAN issues were resolved between Q74 and eliminations (zero CAN errors in E6/E9)

Log File Health

Log Match Duration Status Concern
P6 Practice 28.9s Truncated Short session
Q8 Qual 8 506.1s Truncated Full match, incomplete close
Q15 Qual 15 — CORRUPTED Memory error on load—log unreadable
Q38 Qual 38 32.9s Truncated Only 33s captured—significant data loss
Practice Day 1 76.0s Truncated Short session
Q51 Qual 51 419.3s Truncated Full match, incomplete close
Q56 Qual 56 499.8s Complete Only fully-written log of the event
Q64 Qual 64 507.1s Truncated Full match, incomplete close
Q68 Qual 68 ~180s* Truncated Timestamp corruption (shows 38M sec)
Q74 Qual 74 ~370s* Truncated Timestamp corruption (shows 98M sec)
Practice Day 2 91.0s Truncated Short session
E6 Elim 6 332.3s Truncated Full match, incomplete close
E9 Elim 9 626.3s Truncated Full match, incomplete close

*Estimated from sample counts; actual timestamps corrupted.

The Q68/Q74 timestamp corruption—showing durations of 38 million and 98 million seconds respectively—suggests memory corruption or integer overflow in the logging system, possibly related to heavy CAN errors occurring simultaneously. The team captured 3,200+ seconds of robot operation across the event, but only 500 seconds closed cleanly.

Match-by-Match Analysis

Match Time Score Result Git SHA Dirty CAN Errors Min V Max Cycle Brownout
Q8 12:17 PM 62-157 Loss 47dbdac Yes 2 7.24V 24.7s MODERATE
Q15 1:25 PM 66-37 Win — — — — — —
Q38 5:01 PM 134-27 Win 6e7a2fe Yes 0 12.75V — LOW
Q51 7:02 PM 120-129 Loss 65f4a3f No 0 6.62V 20.4s HIGH
Q56 7:57 PM 164-111 Win 65f4a3f Yes 0 7.06V 19.4s MODERATE
Q64 10:13 AM 103-119 Loss d9fbb04 No 0 6.81V 24.4s HIGH
Q68 10:48 AM 79-209 Loss d9fbb04 No 12 7.54V 23.9s MODERATE
Q74 11:37 AM 149-98 Win d9fbb04 Yes 11 7.47V 21.6s MODERATE
E6 ~2:48 PM — — d9fbb04 Yes 0 6.88V 20.4s HIGH
E9 ~3:16 PM — — d9fbb04 Yes 0 6.89V 23.6s HIGH

Win/Loss Patterns:

  • Matches with minimum voltage above 7.0V: 3-1 record
  • Matches with minimum voltage below 7.0V: 1-3 record (only Q56 at 7.06V was a win)
  • The team won Q74 despite 11 CAN errors—when the robot held together, it performed

Code Version Timeline

The team ran 4 distinct commits across 7 builds during the event:

Day 1 (March 7) — EST

Build Time Git SHA Branch Dirty Matches
08:05:59 47dbdac main No P6
11:16:35 47dbdac main Yes Q8
16:36:48 6e7a2fe vaale-day1 Yes Q38, Practice
18:38:21 65f4a3f vaale-day1 No Q51
19:12:57 65f4a3f vaale-day1 Yes Q56

Day 2 (March 8) — EDT

Build Time Git SHA Branch Dirty Matches
08:38:04 d9fbb04 vaale-day1 No Q64, Q68
11:24:49 d9fbb04 vaale-day1 Yes Q74
14:22:16 d9fbb04 vaale-day1 Yes Practice, E6, E9

The team branched to vaale-day1 early on Day 1 and never merged back during the event. Five of seven builds were dirty—uncommitted changes deployed to the robot. Build d9fbb04 ran six consecutive logs from Q64 through E9.

CAN Bus Health Analysis

Error Distribution

Match CAN Errors Primary Device Assessment
Q8 2 REV PH (solenoids) Intermittent
Q68 12 REV PH (compressor, solenoids) Cascading failure
Q74 11 REV PH + Canandgyro Multiple devices
All others 0 — Clean

Q68 Error Timeline (12 errors over 73 seconds)

The REV Pneumatic Hub began dropping packets at 202.1s and continued through 275.7s:

202.1s - LoggedRobot: CAN Timeout
206.7s - IntakeArmIOReal.java:19 - getSolenoids() timeout
214.9s - IntakeArmIOReal.java:19 - CAN Timeout
217.5s - LoggedRobot - getCompressorCurrent() timeout
222.1s - LoggedRobot - getCompressor() timeout
226.9s - LoggedRobot - getCompressorCurrent() timeout
229.4s - LoggedRobot - CAN Timeout
242.1s - LoggedRobot - CAN Timeout
253.3s - LoggedRobot - CAN Timeout
258.7s - IntakeArmIOReal.java:19 - getSolenoids() timeout
269.7s - LoggedRobot - getCompressorCurrent() timeout
275.7s - LoggedRobot - getCompressorCurrent() timeout

This is a cascading failure: the PH oscillates between responsive and unresponsive states with ~5 second timeout/retry cycles.

Q74 Startup Issue

At 10.7s, Q74 logged:

Canandgyro[device_id=0] did not respond to firmware version check
-- is the device powered and connected to the robot?

The Redux Canandgyro failed its initialization handshake but the robot continued operating. This suggests either a loose connection or slow initialization timing.

Root Cause Hypothesis

The CAN errors cluster in Day 2 morning (Q68, Q74) and disappear entirely for eliminations. Possible causes:

  1. Loose CAN termination near the PH—fixed in the pits before eliminations
  2. Temperature-related issues that resolved as the venue warmed
  3. Vibration-induced connector problems from match impacts

Battery & Power Analysis

Voltage Statistics by Match

Match Min V Max V Mean V Std Dev Risk Level
Q8 7.24 13.25 11.57 1.36 MODERATE
Q38 12.75 12.80 12.78 — LOW
Q51 6.62 13.22 11.13 1.72 HIGH
Q56 7.06 13.48 12.13 1.27 MODERATE
Q64 6.81 13.03 11.43 1.43 HIGH
Q68 7.54 13.10 10.85 1.20 MODERATE
Q74 7.47 13.47 11.38 1.31 MODERATE
E6 6.88 13.14 11.39 1.50 HIGH
E9 6.89 12.84 11.91 1.33 HIGH

Q51's 6.62V minimum is the most severe—only 0.12V above roboRIO brownout protection threshold. Four matches (Q51, Q64, E6, E9) crossed into dangerous territory below 7V.

Voltage Trends

Day 1 showed voltage management becoming an issue late in the day—Q51's crisis occurred at 7:02 PM after a full day of matches. Day 2 morning started with problems (Q64: 6.81V at 10:13 AM) and eliminations continued the pattern despite presumably fresh batteries.

The voltage variance (std dev 1.2-1.7V) indicates significant load swings—likely aggressive driving combined with launcher operation.

Vision System Analysis

Observation Score Statistics

Match Samples Min Max Mean Median Std Dev
Q8 11,174 0.020 0.094 0.065 0.064 0.015
Q51 9,715 0.020 0.094 0.060 0.068 0.018
Q56 11,012 0.020 0.094 0.057 0.065 0.019
Q64 9,445 0.020 0.094 0.066 0.067 0.016
Q68 6,040 0.020 0.093 0.062 0.068 0.020
Q74 7,329 0.020 0.094 0.064 0.068 0.023
E6 7,652 0.020 0.094 0.054 0.056 0.021
E9 13,580 0.020 0.094 0.058 0.063 0.019

Vision scores remained remarkably consistent (mean 0.054-0.066) across all matches. E6 had the lowest mean score, potentially indicating more chaotic gameplay or partially obscured cameras during elimination intensity.

Camera Connectivity

All four cameras (Camera0-3) remained connected throughout logged matches. PhotonVision coprocessors at 10.23.63.201 and 10.23.63.202 established stable NT4 connections. No camera disconnection errors were recorded.

Loop Timing & Performance

Cycle Time Statistics

Match Cycles Min (ms) Max (ms) Mean (ms) Median (ms)
Q8 21,828 7.1 24,724 18.5 15.0
Q51 18,212 7.7 20,407 18.2 14.8
Q56 21,777 7.4 19,409 18.1 14.9
Q64 18,844 8.4 24,438 23.6 18.6
Q68 10,727 7.6 23,934 21.2 15.5
Q74 15,468 8.4 21,598 19.9 15.8
E6 13,866 7.7 20,359 19.6 15.3
E9 26,944 7.6 23,581 18.9 15.6

The Startup Freeze Problem

Every match shows a 20-25 second freeze early in robot code execution. Q8's 24.7s freeze is the longest recorded. This appears to be JVM class loading, static initialization, or first-time JIT compilation.

The freezes occur consistently regardless of code version, suggesting the issue is structural rather than a regression. A 20+ second freeze during autonomous would be catastrophic—the team must be starting robot code well before matches.

Garbage Collection Impact

Match GC Events Max GC (ms) Mean GC (ms)
Q56 384 538 11.4
E9 474 682 11.3

E9 recorded the worst GC pause at 682ms—34 missed cycles. These pauses likely coincide with heap pressure from logging or vision frame processing.

Device Connectivity Issues

Subsystem Health Summary

Subsystem Status Notes
Drive (4 Swerve Modules) Healthy All modules connected throughout
Gyro (Canandgyro) Intermittent Failed firmware check in Q74
Vision (4 Cameras) Healthy Stable throughout event
Flywheel Healthy Connected, peak 21.3 m/s
Hood Healthy Connected
IntakeArm (Pneumatic) Affected PH CAN issues impacted solenoid reads
IntakeRoller Healthy Connected
Kicker Healthy Connected
Spindexer Healthy Connected
Turret Healthy Connected

Pneumatic Hub Timeline

Match PH Status Errors
Q8 Intermittent 2 solenoid timeouts
Q68 Failing 12 errors (solenoids + compressor)
Q74 Failing 10 errors (solenoids + compressor)
E6 Healthy 0 errors
E9 Healthy 0 errors

The PH was clearly fixed between Q74 and eliminations—12 and 11 errors dropped to zero.

Math & Runtime Errors

Rotation2d Zero-Vector Errors

Match Count Timestamps
Q8 1 283.5s
Q56 3 474.7s, 475.7s, 477.2s
E9 1 414.8s

Error:

x and y components of Rotation2d are zero
    at edu.wpi.first.math.geometry.Rotation2d.<init>(Rotation2d.java:126)
    at edu.wpi.first.math.geometry.Translation2d.getAngle(Translation2d.java:200)

This occurs when code calls Translation2d.getAngle() on a zero vector—likely when computing angles from identical sequential poses or when vision tracking has no valid target.

Inter-Match Trends & Degradation

Day 1 Progression

Time Match Min Voltage CAN Errors Outcome
12:17 PM Q8 7.24V 2 Loss
5:01 PM Q38 12.75V 0 Win
7:02 PM Q51 6.62V 0 Loss
7:57 PM Q56 7.06V 0 Win

Late-day voltage crisis: Q51's 6.62V came after 7 hours of competition. Q38's excellent 12.75V minimum suggests a fresh battery or low-intensity match.

Day 2 Progression

Time Match Min Voltage CAN Errors Outcome
10:13 AM Q64 6.81V 0 Loss
10:48 AM Q68 7.54V 12 Loss
11:37 AM Q74 7.47V 11 Win
~2:48 PM E6 6.88V 0 —
~3:16 PM E9 6.89V 0 —

Morning CAN crisis, afternoon resolution. The pit crew clearly found and fixed the PH connection issue between Q74 (11:37 AM) and the 2:22 PM elimination build.

Chronological Narrative

Day 1 Morning

The team arrived at VAALE with commit 47dbdac on the main branch, built fresh at 8:05 AM. Practice Match 6 ran briefly—29 seconds of data before the robot was disabled, suggesting a quick test or immediate issue.

By 11:16 AM, uncommitted changes were deployed for Q8. The match went poorly: a 62-157 loss. Two CAN timeouts from the Pneumatic Hub appeared around 354-355 seconds, plus a Rotation2d zero-vector error at 283 seconds. The 24.7-second startup freeze—the event's longest—occurred during pre-match warmup. Battery voltage dipped to 7.24V but stayed above brownout territory.

Day 1 Afternoon

Between Q8 and Q38, the team created the vaale-day1 branch (commit 6e7a2fe) and deployed new code at 4:36 PM.

Q38 was a blowout: 134-27. Unfortunately, the log captured only 33 seconds of data—almost no telemetry from what was clearly the robot's best performance. What little data exists shows pristine conditions: voltage above 12.75V, zero errors.

Day 1 Evening

At 6:38 PM, changes were committed (now 65f4a3f) and a clean build deployed for Q51. This match was a close loss: 120-129. The telemetry tells a story of electrical collapse—voltage crashed to 6.62V, the event's lowest point. The robot danced on the edge of brownout for significant portions of the match.

Thirty-four minutes later, another dirty build ran Q56: a commanding 164-111 win. Voltage still dipped to 7.06V, and three Rotation2d errors appeared in the final seconds. Q56 is the only complete log file of the entire event.

Day 2 Morning

Daylight Saving Time started overnight, shifting timestamps from EST to EDT.

The team deployed commit d9fbb04 at 8:38 AM and ran it through both Q64 and Q68. Q64 was a loss (103-119) with severe voltage issues (6.81V minimum) but zero CAN errors.

Then came Q68: disaster. The 79-209 score barely hints at the chaos—12 CAN timeout errors from the REV Pneumatic Hub between 202-276 seconds. The compressor and solenoids intermittently dropped off the bus for over a minute of match time.

Q74 showed the problem persisted: 11 more CAN errors, plus a Canandgyro that failed its firmware check at startup. Despite this, the team won 149-98—the robot was competitive even when fighting its own electrical demons.

Day 2 Afternoon

Something changed between Q74 and eliminations. The team deployed another dirty build at 11:24 AM, then again at 2:22 PM for playoff rounds.

E6 and E9 both ran with zero CAN errors. Someone found and fixed the Pneumatic Hub connection. However, voltage stress remained severe—both elimination matches saw minimums around 6.88V. The robot was being pushed hard in high-stakes gameplay.

E9 logged one final Rotation2d error at 414.8 seconds. The event logs end with E9—we don't know the final result, but the team made it deep into eliminations.

Recommendations

Critical (Fix Before Next Event)

  1. Investigate REV Pneumatic Hub CAN connection

    • Inspect all CAN wiring at and around the PH
    • Check CAN termination resistor placement
    • Consider replacing cables to the PH
    • Verify PH firmware is current

  2. Resolve the 20+ second startup freeze

    • Profile with VisualVM to identify the blocking operation
    • Consider lazy initialization for non-critical subsystems
    • May require restructuring robot initialization
    • This freeze during autonomous would be catastrophic

  3. Implement stricter battery management

    • Voltage dropped below 7V in 4 of 9 matches with data
    • Track battery serial numbers for health monitoring
    • Consider reducing drive motor current limits during endgame
    • Rotate batteries more aggressively

Important (Should Address Soon)

  1. Fix Rotation2d zero-vector handling

    • Add guards before Translation2d.getAngle() calls
    • Check for zero-magnitude vectors before computing angles
    • Likely in vision pose processing code

  2. Investigate log file corruption

    • Only 1 of 13 logs closed cleanly
    • Q68/Q74 timestamp corruption suggests deeper issues
    • Review AdvantageKit shutdown hooks
    • Consider periodic explicit flush() calls

  3. Address Canandgyro startup reliability

    • Add retry logic or longer timeout for gyro initialization
    • Verify CAN ID 0 isn't conflicting

Nice to Have

  1. Commit changes before matches

    • 5 of 7 builds were dirty
    • Makes reproducing issues difficult
    • Add git status to pre-match checklist

  2. Investigate Q38 log truncation

    • Best match of the event has only 33 seconds of data
    • Could reveal roboRIO resource issues

Fun Facts & Wacky Statistics

By The Numbers

  • Total logged runtime: 3,200+ seconds (~53 minutes) of robot operation
  • Longest startup freeze: 24.7 seconds (Q8)—time enough for a commercial break
  • Lowest voltage: 6.62V (Q51)—0.12V from the roboRIO pulling the plug
  • Worst GC pause: 682ms (E9)—34 missed control loops
  • CAN errors: 25 total, all from REV PH and Canandgyro

The "If The Robot Could Talk" Awards

  • Most Stressed: Q51—"My voltage hit 6.62V and you wanted me to play DEFENSE?!"
  • Most Mysterious: Q38—"I scored 134 points and you captured 33 seconds of it. I demand a recount."
  • Most Resilient: Q74—"My gyro ghosted me, my pneumatics are having a meltdown, and I STILL won by 51 points."
  • Most Reformed: E6—"Zero errors. I finally have my life together."

Controller Corner

  • Driver: Radiomaster Zorro Joystick
  • Operator: Xbox One Controller for Windows
  • The driver chose precision (flight stick), the operator chose comfort (Xbox)

Temperature Check

  • Coldest CPU: 43.2°C (Q56)
  • Hottest CPU: 51.9°C (E9)
  • Temperature delta: 8.7°C across the event

Vision System Glory

  • Observation score range: 0.020 - 0.094
  • Most consistent camera: Camera0 with 11,174 samples in Q8
  • PhotonVision coprocessors: 10.23.63.201 and 10.23.63.202

The Inexplicable

  • Q15's log triggered a Java memory error: newLimit < 0: (-368229605 < 0). Negative 368 million. The bits have opinions.
  • Q68 claimed to run for 38,472,579 seconds (1.2 years). The timestamp clearly took a detour through a time vortex.
  • The team deployed 7 builds over 2 days—one new build every 4 hours. Sleep deprivation is a feature, not a bug.

Flywheel Achievement Unlocked

  • Peak velocity: 21.3 m/s (76.7 km/h)
  • Mean velocity: 12.2 m/s when active
  • That's faster than Usain Bolt

Appendix: Raw Data Tables

A1: Complete Voltage Statistics

Match Samples Min (V) Max (V) Mean (V) Median (V) Std Dev
Q8 18,109 7.24 13.25 11.57 12.40 1.36
Q51 15,646 6.62 13.22 11.13 12.18 1.72
Q56 17,486 7.06 13.48 12.13 12.81 1.27
Q64 15,629 6.81 13.03 11.43 12.22 1.43
Q68 9,773 7.54 13.10 10.85 10.63 1.20
Q74 13,237 7.47 13.47 11.38 11.80 1.31
E6 12,104 6.88 13.14 11.39 11.78 1.50
E9 21,952 6.89 12.84 11.91 12.79 1.33

A2: Complete Cycle Time Statistics

Match Cycles Min (ms) Max (ms) Mean (ms) Median (ms) Std Dev
Q8 21,828 7.07 24,724 18.48 15.03 167.8
Q51 18,212 7.66 20,407 18.18 14.79 151.7
Q56 21,777 7.40 19,409 18.06 14.91 132.0
Q64 18,844 8.38 24,438 23.63 18.59 178.7
Q68 10,727 7.64 23,934 21.17 15.45 231.4
Q74 15,468 8.39 21,598 19.85 15.80 174.3
E6 13,866 7.74 20,359 19.59 15.32 173.5
E9 26,944 7.62 23,581 18.86 15.62 144.1

A3: Vision Observation Scores

Match Samples Min Max Mean Median Std Dev
Q8 11,174 0.020 0.094 0.065 0.064 0.015
Q51 9,715 0.020 0.094 0.060 0.068 0.018
Q56 11,012 0.020 0.094 0.057 0.065 0.019
Q64 9,445 0.020 0.094 0.066 0.067 0.016
Q68 6,040 0.020 0.093 0.062 0.068 0.020
Q74 7,329 0.020 0.094 0.064 0.068 0.023
E6 7,652 0.020 0.094 0.054 0.056 0.021
E9 13,580 0.020 0.094 0.058 0.063 0.019

A4: Code Deployment History

Build Time SHA Branch Dirty Matches
2026-03-07 08:05:59 EST 47dbdacf main No P6
2026-03-07 11:16:35 EST 47dbdacf main Yes Q8
2026-03-07 16:36:48 EST 6e7a2fe1 vaale-day1 Yes Q38, Practice
2026-03-07 18:38:21 EST 65f4a3f2 vaale-day1 No Q51
2026-03-07 19:12:57 EST 65f4a3f2 vaale-day1 Yes Q56
2026-03-08 08:38:04 EDT d9fbb049 vaale-day1 No Q64, Q68
2026-03-08 11:24:49 EDT d9fbb049 vaale-day1 Yes Q74
2026-03-08 14:22:16 EDT d9fbb049 vaale-day1 Yes Practice, E6, E9

A5: CPU Temperature Statistics

Match Min (°C) Max (°C) Mean (°C)
Q8 44.8 51.4 49.0
Q56 43.2 51.2 48.1
E6 46.5 51.8 49.9
E9 44.0 51.9 49.2

Report generated: March 14, 2026 Analysis performed on 13 log files totaling 4.5 GB

Generated using: Claude Code + wpilog-mcp MCP server