Multi-Equipment CBM (v2.1) ✅

Fully Compliant with equipment-cbm-mvp - Multi-Equipment Coordinated Maintenance System with Advanced Probability Distribution Analysis

High-performance DQN learning system for coordinated maintenance decision-making across multiple equipment units.
Extends the single-equipment implementation from equipment-cbm-mvp to 4-equipment coordinated control, achieving realistic anomaly rates, high-performance algorithms, and comprehensive QR-DQN-style probability distribution analysis.

✅ Implementation Status

Component	Status	Description
Environment	✅	4-equipment support based on equipment-cbm-mvp/cbm_environment.py
Preprocessor	✅	Realistic anomaly rates (1.9-2.2%) & inter-equipment correlation analysis
Training	✅	Full implementation of QR-DQN + Noisy Networks + PER
Configuration	✅	3 scenario support & comprehensive settings
Integration Test	✅	All-module integration test successful
Training Test	✅	50-episode training successful (avg reward 374.85)
Visualization v2.1	✅	Advanced probability distribution analysis & QR-DQN visualization

📊 System Specifications

Target Equipment (4-Equipment Coordinated Control)

Steam Turbine (43124) - Anomaly Rate: 2.2%
Desulfurization Unit (43114) - Anomaly Rate: 1.9%
Compressor_1 (275490) - Anomaly Rate: 2.0%
Compressor_2 (275491) - Anomaly Rate: 2.0%

Algorithm Performance

QR-DQN: 51-quantile distribution learning
Noisy Networks: Parameter space exploration
PER: Prioritized Experience Replay (α=0.6, β=0.4)
N-step Learning: 3-step look-ahead learning
Mixed Precision: GPU-optimized training

Inter-Equipment Correlation

Inter-equipment correlation coefficient: 0.92-1.00 (high correlation)
Coordinated maintenance bonus configured
Cascade failure prevention function

State Transition Model (Transition Matrices)

Data-Driven: Estimates equipment-specific failure/recovery patterns from actual equipment data
Equipment-Specific: Individual learning of each equipment's characteristics (reliability, natural recovery rate)
High-Precision Modeling: Achieves high reliability with 98-99% Normal→Normal transitions

🚀 Quick Start

1. Prerequisites Check

# Verify Python environment and PyTorch
python --version  # 3.12.10
python -c "import torch; print(f'PyTorch: {torch.__version__}')"

2. System Integration Test

# Verify all component operations
python test_system_integration.py

# Check transition matrix data
python -c "
import json
with open('preprocessed_data/equipment_43124_stats.json', 'r') as f:
    data = json.load(f)
    print('Steam Turbine Transition Matrix:')
    print(data['transition_matrix'])
"

3. Training Execution

# 1000 episodes, balanced scenario
python train_multi_equipment_dqn.py --episodes 1000 --scenario balanced --output_dir outputs_balanced

# Individual scenario execution
python train_multi_equipment_dqn.py --episodes 1000 --scenario safety_first --output_dir outputs_safety_first
python train_multi_equipment_dqn.py --episodes 1000 --scenario cost_efficient --output_dir outputs_cost_efficient

🔧 Implementation Features

Full equipment-cbm-mvp Compliance

Base Architecture: Complete foundation on equipment-cbm-mvp/{cbm_environment.py, data_preprocessor.py, train_cbm_dqn_v2.py}
Algorithm Inheritance: Full functionality inheritance of QR-DQN + Noisy Networks + PER
Quality Assurance: Extension of proven high-performance single-equipment implementation to multi-equipment

Realistic Anomaly Rates

Statistical Threshold Calculation: Appropriate normal/anomalous state classification using k-sigma=2.0
Validated Anomaly Rates: 1.9-2.2% (realistic industrial range)
Data Quality: Automatic selection of 4 high-quality equipment units (excluding high anomaly rate equipment)

High-Performance Multi-Equipment Implementation

State Dimension: 8 dimensions (4 equipment × 2 features)
Action Space: MultiDiscrete [3,3,3,3] = 81 action combinations
Network: Large-scale network with 2.5M parameters
Parallelization: AsyncVectorEnv (4-8 parallel environments)

📁 File Structure

multi-equipments-cbm/
├── multi_equipment_environment.py     # 🏭 Multi-equipment environment (equipment-cbm-mvp based)
├── multi_equipment_preprocessor.py    # 🔧 Data preprocessing (equipment-cbm-mvp based) 
├── train_multi_equipment_dqn.py      # 🧠 QR-DQN training (equipment-cbm-mvp based)
├── visualize_multi_equipment.py      # 📊 Advanced visualization system (v2.1) - QR-DQN probability distribution analysis
├── config.yaml                       # ⚙️ Configuration file (equipment-cbm-mvp based)
├── test_system_integration.py        # 🧪 Integration test script
├── README.md                         # 📖 This file (English)
├── README_JP.md                      # 📖 Japanese documentation
├── requirements.txt                  # 📦 Dependencies
└── preprocessed_data/                # 📊 Preprocessed data
    ├── equipment_43124_stats.json   #     Steam turbine statistics
    ├── equipment_43114_stats.json   #     Desulfurization unit statistics  
    ├── equipment_275490_stats.json  #     Compressor_1 statistics
    ├── equipment_275491_stats.json  #     Compressor_2 statistics
    ├── equipment_43124_timeseries.csv    #     Steam turbine time series
    ├── equipment_43114_timeseries.csv    #     Desulfurization unit time series
    ├── equipment_275490_timeseries.csv   #     Compressor_1 time series
    ├── equipment_275491_timeseries.csv   #     Compressor_2 time series
    └── multi_equipment_correlations.json #     Inter-equipment correlation info

🎯 Three Maintenance Scenarios

Safety First

Strategy: Early anomaly detection & immediate response
Features: Anomaly threshold ×0.8, repair preference ×1.5, coordination emphasis ×1.2
Application: Equipment/plants where safety is paramount

Cost Efficient

Strategy: Minimum necessary maintenance for cost reduction
Features: Anomaly threshold ×1.2, repair preference ×0.8, coordination emphasis ×0.8
Application: Manufacturing lines where cost optimization is critical

Balanced

Strategy: Optimal balance of safety and cost
Features: Standard settings (threshold ×1.0, preference ×1.0, coordination ×1.0)
Application: General manufacturing environments & standard operations

📊 Visualization System (v2.1) - New Features Added ✅

Advanced Probability Distribution Analysis

1. Basic Learning Visualization

Learning Curves: Time-series analysis of episode rewards, losses, and statistics
Inter-Equipment Correlation: Visualization of coordinated maintenance effects
Action Patterns: Frequency and efficiency analysis of maintenance actions
Cost Efficiency: ROI and efficiency metric evaluation

2. QR-DQN Probability Distribution Analysis NEW

Distribution Statistics: Comprehensive analysis of mean, median, standard deviation, skewness, kurtosis
Quantile Analysis: Detailed quantile statistics Q5-Q95 (51-quantile support)
VaR/CVaR Analysis: Value at Risk & Conditional VaR risk indicators
Uncertainty Evaluation: Learning stability analysis using uncertainty ratio (σ/|μ|)

3. Advanced Statistical Visualization NEW

Quantile Functions: Detailed visualization of 51 quantiles in QR-DQN style
Distribution Evolution: Probability distribution changes across learning periods
Quantile Stability: IQR-based stability indicators
Risk Profile: Comprehensive risk statistics summary

Visualization Execution

Basic Visualization

# Standard visualization (learning curves, correlations, action patterns, efficiency)
python visualize_multi_equipment.py --results outputs_2000eps_16envs_balanced --output visualization_standard

Extended Probability Distribution Analysis NEW

# Comprehensive visualization including QR-DQN probability distribution analysis
python visualize_multi_equipment.py --results outputs_2000eps_16envs_balanced --output visualization_enhanced_distributions

Generated Visualization Files

training_curves.png - Basic learning curves
equipment_correlations.png - Inter-equipment correlation analysis
maintenance_action_patterns.png - Maintenance action patterns
cost_efficiency_analysis.png - Cost efficiency analysis
reward_distribution_analysis.png - NEW Comprehensive probability distribution analysis
advanced_quantile_analysis.png - NEW QR-DQN quantile analysis

Probability Distribution Analysis Features

VaR/CVaR Risk Analysis

VaR(5%): -152.3  # Worst loss occurring with 5% probability
CVaR(5%): -198.7  # Average loss of worst 5% cases

Quantile Statistics (51 quantiles)

Q05: 245.2    Q25: 289.7    Q50: 325.4
Q75: 361.8    Q95: 398.3
IQR: 72.1     90% Range: 153.1

Uncertainty Evaluation

Stability Index: 0.089  # σ/|μ| < 0.1 = High stability
Risk Assessment: ✓ Low Risk

📈 Validated Performance Metrics (v2.1 Updated)

Visualization System Performance ✅

2000 Episode Analysis: Comprehensive distribution analysis completed
6 Types of Visualization: Basic 4 types + probability distribution 2 types
Statistical Rigor: VaR, CVaR, quantiles, skewness & kurtosis support
QR-DQN Compatible: Detailed distribution analysis using 51 quantiles

Integration Test Results ✅

Modules: All imports successful
Environment Creation: 4-equipment coordinated environment operating normally
Network: 2.5M parameter normal initialization
Replay Buffer: PER normal operation confirmed

Training Test Results ✅

Episodes: 50 episodes completed
Average Reward: 374.85
Execution Time: 31.39 seconds (0.628 seconds/episode)
Convergence: Stable learning curve

Data Quality ✅

Equipment Count: 4 equipment selection completed
Anomaly Rates: 1.9-2.2% (realistic range)
Correlation Coefficients: 0.92-1.00 (high correlation confirmed)
Transition Matrices: Real Data Estimation - State transition probabilities estimated from actual equipment data rather than default values

🔄 Usage

Step 1: Data Preprocessing ✅

# Already completed - 4 equipment data generated in preprocessed_data/
python multi_equipment_preprocessor.py

Step 2: System Integration Test ✅

# All component operation verification - already successfully confirmed
python test_system_integration.py

Step 3: Full Training Execution 🚀

# Balanced scenario (recommended)
python train_multi_equipment_dqn.py --episodes 1000 --scenario balanced --output_dir outputs_balanced

# Safety first scenario  
python train_multi_equipment_dqn.py --episodes 1000 --scenario safety_first --output_dir outputs_safety_first

# Cost efficient scenario
python train_multi_equipment_dqn.py --episodes 1000 --scenario cost_efficient --output_dir outputs_cost_efficient

📈 Expected Learning Results

Learning Progress Pattern

Initial Stage (0-200 episodes): Exploratory behavior, reward 300-400
Mid Stage (200-600 episodes): Strategy learning, reward 400-500
Convergence Stage (600-1000 episodes): Optimization, reward 500+

Expected Performance by Scenario

Safety First: High normal maintenance rate, acceptable maintenance cost increase
Cost Efficient: Low maintenance cost, minimum necessary intervention
Balanced: Optimal risk-cost balance

⚙️ Advanced Configuration

Main Parameters (config.yaml)

# Multi-equipment settings
multi_equipment:
  equipment_list: ["Steam Turbine", "Desulfurization Unit", "Compressor_1", "Compressor_2"]
  correlation_threshold: 0.7
  enable_coordination: true

# Training settings  
training:
  n_episodes: 1000
  n_envs: 8
  learning_rate: 1.0e-3
  batch_size: 64
  buffer_size: 10000
  n_quantiles: 51
  n_steps: 3

# Scenario settings
scenarios:
  balanced:
    anomaly_threshold_multiplier: 1.0
    repair_preference: 1.0
    coordination_weight: 1.0

System Requirements

Python: 3.12.10 (recommended)
PyTorch: 2.9.1+
RAM: 8GB+ recommended
GPU: Optional (CUDA support for training acceleration)

🔧 Troubleshooting

Common Issues and Solutions

1. Import Errors

# Check module path
python -c "import sys; print(sys.path)"
python -c "import multi_equipment_environment; print('OK')"

2. GPU/CUDA Issues

# Check GPU
python -c "import torch; print(f'CUDA: {torch.cuda.is_available()}')"
# Fallback to CPU training
python train_multi_equipment_dqn.py --episodes 100 --device cpu

3. Memory Shortage

# Adjust settings in config.yaml
training:
  n_envs: 4        # Reduce from 8→4
  batch_size: 32   # Reduce from 64→32  
  buffer_size: 5000 # Reduce from 10000→5000

📋 Update History

v2.1.0 (2025-12-21) ✅ - Probability Distribution Analysis Enhanced Version

✅ Full equipment-cbm-mvp compliance: Extension of single-equipment high-performance implementation to 4-equipment
✅ Multi-equipment coordinated control: Simultaneous control of 4 equipment (Steam Turbine, Desulfurization Unit, Compressor×2)
✅ Realistic anomaly rates: Appropriate 1.9-2.2% anomaly rates using k-sigma=2.0
✅ High-performance algorithms: Full implementation of QR-DQN + Noisy Networks + PER
✅ 3 scenario support: Safety First, Cost Efficient, Balanced
✅ Integration test completed: All module operation verified
✅ Training test completed: 50 episodes successful (average reward 374.85)
✅ Inter-equipment correlation analysis: High correlation equipment group confirmed (0.92-1.00)
✅ Advanced visualization system: QR-DQN probability distribution analysis, VaR/CVaR, quantile statistics support
✅ Comprehensive statistical analysis: 51 quantiles, uncertainty evaluation, risk profile complete

🚀 Future Development Plans

Visualization System (v2.1 Complete): Comprehensive learning analysis & QR-DQN probability distribution visualization
Real-time Monitoring: Dashboard-type monitoring interface
Advanced Inter-equipment Modeling: Graph Neural Networks implementation
Automatic Hyperparameter Tuning: Optimization using Optuna etc.
Industry-specific Scenarios: Industry-specific maintenance strategy templates

📞 Technical Support

Implementation Base

equipment-cbm-mvp: High-performance implementation of single-equipment CBM
Algorithms: QR-DQN + Noisy Networks + PER
Validated: Realistic anomaly rates, integration test, training test successful

System Status

✅ Implementation Complete: All component operation verified
✅ Test Success: Integration and training tests all passed
✅ Visualization v2.1: QR-DQN probability distribution analysis & comprehensive statistics support
🚀 Operational: Ready for full training execution

Multi-Equipment CBM v2.1 is ready for production use! Detailed learning quality evaluation is now possible with probability distribution analysis features!

📄 Language Versions

English: README.md (this file)
Japanese: README_JP.md (詳細な日本語ドキュメント)

📜 License

This project is based on equipment-cbm-mvp and extends its capabilities for multi-equipment scenarios while maintaining full compatibility with the original high-performance algorithms.

Name		Name	Last commit message	Last commit date
visualization_2000eps_enhanced_distributions		visualization_2000eps_enhanced_distributions
.gitignore		.gitignore
GITHUB_DESCRIPTION.txt		GITHUB_DESCRIPTION.txt
README.md		README.md
README_JP.md		README_JP.md
base_environment.py		base_environment.py
config.yaml		config.yaml
multi_equipment_environment.py		multi_equipment_environment.py
multi_equipment_preprocessor.py		multi_equipment_preprocessor.py
requirements.txt		requirements.txt
test_system.py		test_system.py
test_system_integration.py		test_system_integration.py
train_multi_equipment_dqn.py		train_multi_equipment_dqn.py
visualize_multi_equipment.py		visualize_multi_equipment.py

Latest commit

History

Folders and files

Repository files navigation