|
| 1 | +# CLAUDE.md - AI Assistant Guide for Apache SkyWalking |
| 2 | + |
| 3 | +This file provides guidance for AI assistants working with the Apache SkyWalking codebase. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +Apache SkyWalking is an open-source APM (Application Performance Monitoring) system designed for microservices, cloud-native, and container-based architectures. It provides distributed tracing, service mesh telemetry analysis, metrics aggregation, alerting, and observability capabilities. |
| 8 | + |
| 9 | +## Repository Structure |
| 10 | + |
| 11 | +``` |
| 12 | +skywalking/ |
| 13 | +├── oap-server/ # OAP (Observability Analysis Platform) backend server |
| 14 | +│ ├── server-core/ # Core module with fundamental services |
| 15 | +│ ├── server-library/ # Shared libraries (module system, util, etc.) |
| 16 | +│ ├── server-receiver-plugin/ # Data receivers (gRPC, HTTP, Kafka, etc.) |
| 17 | +│ ├── server-storage-plugin/ # Storage implementations (BanyanDB, Elasticsearch, etc.) |
| 18 | +│ ├── server-cluster-plugin/ # Cluster coordination (Zookeeper, K8s, etc.) |
| 19 | +│ ├── server-query-plugin/ # Query interfaces (GraphQL) |
| 20 | +│ ├── server-alarm-plugin/ # Alerting system |
| 21 | +│ ├── server-fetcher-plugin/ # Data fetchers |
| 22 | +│ ├── server-configuration/ # Dynamic configuration providers |
| 23 | +│ ├── oal-grammar/ # OAL (Observability Analysis Language) grammar |
| 24 | +│ ├── oal-rt/ # OAL runtime |
| 25 | +│ ├── mqe-grammar/ # MQE (Metrics Query Engine) grammar |
| 26 | +│ ├── mqe-rt/ # MQE runtime |
| 27 | +│ ├── analyzer/ # Log and trace analyzers |
| 28 | +│ ├── ai-pipeline/ # AI/ML pipeline components |
| 29 | +│ └── exporter/ # Data export functionality |
| 30 | +├── apm-protocol/ # Protocol definitions (submodule) |
| 31 | +│ └── apm-network/ # gRPC/Protobuf network protocols |
| 32 | +├── skywalking-ui/ # Web UI (submodule - skywalking-booster-ui) |
| 33 | +├── apm-webapp/ # Web application packaging |
| 34 | +├── apm-dist/ # Distribution packaging |
| 35 | +├── docs/ # Documentation |
| 36 | +├── docker/ # Docker build files |
| 37 | +├── test/ # E2E and integration tests |
| 38 | +└── tools/ # Development tools |
| 39 | +``` |
| 40 | + |
| 41 | +## Build System |
| 42 | + |
| 43 | +### Prerequisites |
| 44 | +- JDK 11, 17, or 21 (LTS versions) |
| 45 | +- Maven 3.6+ |
| 46 | +- Git (with submodule support) |
| 47 | + |
| 48 | +### Common Build Commands |
| 49 | + |
| 50 | +```bash |
| 51 | +# Clone with submodules |
| 52 | +git clone --recurse-submodules https://github.com/apache/skywalking.git |
| 53 | + |
| 54 | +# Or initialize submodules after clone |
| 55 | +git submodule init && git submodule update |
| 56 | + |
| 57 | +# Full build (skip tests) |
| 58 | +./mvnw clean package -Dmaven.test.skip |
| 59 | + |
| 60 | +# Build backend only |
| 61 | +./mvnw package -Pbackend,dist |
| 62 | +# or: make build.backend |
| 63 | + |
| 64 | +# Build UI only |
| 65 | +./mvnw package -Pui,dist |
| 66 | +# or: make build.ui |
| 67 | + |
| 68 | +# Run tests |
| 69 | +./mvnw test |
| 70 | + |
| 71 | +# Run integration tests |
| 72 | +./mvnw integration-test |
| 73 | + |
| 74 | +# Build with all profiles |
| 75 | +./mvnw clean package -Pall -Dmaven.test.skip |
| 76 | +``` |
| 77 | + |
| 78 | +### Maven Profiles |
| 79 | +- `backend` (default): Builds OAP server modules |
| 80 | +- `ui` (default): Builds web application |
| 81 | +- `dist` (default): Creates distribution packages |
| 82 | +- `all`: Builds everything including submodule initialization |
| 83 | + |
| 84 | +## Architecture & Key Concepts |
| 85 | + |
| 86 | +### Module System |
| 87 | +SkyWalking uses a custom module/provider architecture based on Java SPI: |
| 88 | + |
| 89 | +- **ModuleDefine**: Declares a module and its required services |
| 90 | +- **ModuleProvider**: Implements a module with specific technology/approach |
| 91 | +- **Service**: Interface that modules expose to other modules |
| 92 | + |
| 93 | +Key pattern: |
| 94 | +```java |
| 95 | +public class XxxModule extends ModuleDefine { |
| 96 | + public Class[] services() { |
| 97 | + return new Class[] { XxxService.class }; |
| 98 | + } |
| 99 | +} |
| 100 | + |
| 101 | +public class XxxModuleProvider extends ModuleProvider { |
| 102 | + public void prepare() { /* initialize */ } |
| 103 | + public void start() { /* start services */ } |
| 104 | +} |
| 105 | +``` |
| 106 | + |
| 107 | +### Core Concepts |
| 108 | +- **OAL (Observability Analysis Language)**: DSL for defining metrics aggregation rules |
| 109 | +- **MQE (Metrics Query Engine)**: DSL for querying metrics |
| 110 | +- **LAL (Log Analysis Language)**: DSL for log processing |
| 111 | +- **MAL (Meter Analysis Language)**: DSL for meter data processing |
| 112 | +- **Source/Scope**: Data model definitions for telemetry data |
| 113 | +- **Stream Processing**: Metrics, Records, and TopN processing pipelines |
| 114 | + |
| 115 | +### Data Flow |
| 116 | +1. Agents/Collectors send data via gRPC/HTTP/Kafka |
| 117 | +2. Receiver plugins parse and validate data |
| 118 | +3. Analysis engine processes data using OAL/LAL/MAL |
| 119 | +4. Storage plugins persist aggregated data |
| 120 | +5. Query plugins serve data to UI/API |
| 121 | + |
| 122 | +## Code Style & Conventions |
| 123 | + |
| 124 | +### Checkstyle Rules (enforced via `apm-checkstyle/checkStyle.xml`) |
| 125 | + |
| 126 | +**Prohibited patterns:** |
| 127 | +- No `System.out.println` - use proper logging (SLF4J) |
| 128 | +- No `@author` tags - ASF projects don't use author annotations |
| 129 | +- No Chinese characters in source files |
| 130 | +- No tab characters (use 4 spaces) |
| 131 | +- No star imports (`import xxx.*`) |
| 132 | +- No unused or redundant imports |
| 133 | +- No empty statements (standalone `;`) |
| 134 | + |
| 135 | +**Required patterns:** |
| 136 | +- `@Override` annotation required for overridden methods |
| 137 | +- `equals()` and `hashCode()` must be overridden together |
| 138 | +- Javadoc `@param`, `@return`, `@throws` must have descriptions |
| 139 | +- Long constants must use uppercase `L` (e.g., `100L` not `100l`) |
| 140 | +- `default` case must come last in switch statements |
| 141 | +- One statement per line |
| 142 | + |
| 143 | +**Naming conventions:** |
| 144 | +- Constants/static variables: `UPPER_CASE_WITH_UNDERSCORES` |
| 145 | +- Type parameters: `UPPER_CASE` (e.g., `TYPE`, `KEY`, `VALUE`) |
| 146 | +- Package names: `org.apache.skywalking.*` or `test.apache.skywalking.*` |
| 147 | +- Type names: `PascalCase` or `UPPER_CASE_WITH_UNDERSCORES` |
| 148 | +- Local variables/parameters/members: `camelCase` |
| 149 | + |
| 150 | +**File limits:** |
| 151 | +- Max file length: 3000 lines |
| 152 | + |
| 153 | +**Whitespace:** |
| 154 | +- Whitespace required after commas, semicolons, type casts |
| 155 | +- Whitespace required around operators |
| 156 | +- No multiple consecutive blank lines |
| 157 | +- Empty line separators between class members (fields can be grouped) |
| 158 | + |
| 159 | +### Code Style (via `codeStyle.xml` for IntelliJ IDEA) |
| 160 | + |
| 161 | +**Indentation:** |
| 162 | +- 4-space indentation |
| 163 | +- 4-space continuation indent |
| 164 | + |
| 165 | +**Imports:** |
| 166 | +- No star imports (threshold set to 999) |
| 167 | +- Import order: regular imports, blank line, static imports |
| 168 | + |
| 169 | +**Formatting:** |
| 170 | +- `while` in do-while on new line |
| 171 | +- Align multiline chained method calls |
| 172 | +- Align multiline parameters in calls |
| 173 | +- Array initializer braces on new lines |
| 174 | +- Wrap long method call chains |
| 175 | + |
| 176 | +**General:** |
| 177 | +- Use `final` for local variables and parameters |
| 178 | +- Use Lombok annotations (`@Getter`, `@Setter`, `@Builder`, `@Data`, `@Slf4j`, etc.) |
| 179 | +- Follow existing patterns in similar files |
| 180 | + |
| 181 | +### License Header |
| 182 | +Java, XML, and YAML/YML files must include the Apache 2.0 license header (see `HEADER` file). |
| 183 | +JSON and Markdown files are excluded (JSON doesn't support comments, see `.licenserc.yaml`). |
| 184 | + |
| 185 | +## Testing |
| 186 | + |
| 187 | +### Test Frameworks |
| 188 | +- JUnit 5 (`org.junit.jupiter`) |
| 189 | +- Mockito for mocking |
| 190 | +- AssertJ for assertions |
| 191 | +- PowerMock for reflection utilities |
| 192 | + |
| 193 | +### Test Naming |
| 194 | +- Unit tests: `*Test.java` |
| 195 | +- Integration tests: `IT*.java` or `*IT.java` |
| 196 | + |
| 197 | +### Running Tests |
| 198 | +```bash |
| 199 | +# Unit tests only |
| 200 | +./mvnw test |
| 201 | + |
| 202 | +# Integration tests |
| 203 | +./mvnw integration-test |
| 204 | + |
| 205 | +# Skip tests during build |
| 206 | +./mvnw package -Dmaven.test.skip |
| 207 | +``` |
| 208 | + |
| 209 | +## Git Submodules |
| 210 | + |
| 211 | +The project uses submodules for protocol definitions and UI: |
| 212 | +- `apm-protocol/apm-network/src/main/proto` - skywalking-data-collect-protocol |
| 213 | +- `oap-server/server-query-plugin/.../query-protocol` - skywalking-query-protocol |
| 214 | +- `skywalking-ui` - skywalking-booster-ui |
| 215 | +- `oap-server/server-library/library-banyandb-client/src/main/proto` - banyandb-client-proto |
| 216 | + |
| 217 | +Always use `--recurse-submodules` when cloning or update submodules manually. |
| 218 | + |
| 219 | +## IDE Setup (IntelliJ IDEA) |
| 220 | + |
| 221 | +1. Import as Maven project |
| 222 | +2. Run `./mvnw compile -Dmaven.test.skip=true` to generate protobuf sources |
| 223 | +3. Mark generated source folders: |
| 224 | + - `*/target/generated-sources/protobuf/java` |
| 225 | + - `*/target/generated-sources/protobuf/grpc-java` |
| 226 | + - `*/target/generated-sources/antlr4` |
| 227 | +4. Import `codeStyle.xml` for consistent formatting |
| 228 | + |
| 229 | +## Key Files for Understanding the Codebase |
| 230 | + |
| 231 | +- `oap-server/server-core/src/main/java/.../CoreModule.java` - Core module definition |
| 232 | +- `oap-server/server-library/library-module/src/main/java/.../ModuleDefine.java` - Module system base |
| 233 | +- `oap-server/oal-grammar/src/main/antlr4/.../OALParser.g4` - OAL grammar definition |
| 234 | +- `oap-server/server-starter/` - Application entry point |
| 235 | +- `docs/en/concepts-and-designs/` - Architecture documentation |
| 236 | + |
| 237 | +## Common Development Tasks |
| 238 | + |
| 239 | +### Adding a New Receiver Plugin |
| 240 | +1. Create module in `server-receiver-plugin/` |
| 241 | +2. Implement `ModuleDefine` and `ModuleProvider` |
| 242 | +3. Register via SPI in `META-INF/services/` |
| 243 | +4. Add configuration to `application.yml` |
| 244 | + |
| 245 | +### Adding a New Storage Plugin |
| 246 | +1. Create module in `server-storage-plugin/` |
| 247 | +2. Implement storage DAOs for each data type |
| 248 | +3. Follow existing plugin patterns (e.g., BanyanDB, elasticsearch) |
| 249 | + |
| 250 | +### Modifying OAL Metrics |
| 251 | +1. Edit `.oal` files in `oap-server/server-starter/src/main/resources/oal/` |
| 252 | +2. Regenerate by building the project |
| 253 | +3. Update storage schema if needed |
| 254 | + |
| 255 | +### MAL Scripts (in `oap-server/server-starter/src/main/resources/`) |
| 256 | +- `otel-rules/` - OpenTelemetry metrics (Prometheus, etc.) |
| 257 | +- `meter-analyzer-config/` - SkyWalking native meter protocol |
| 258 | + |
| 259 | +### LAL Scripts (in `oap-server/server-starter/src/main/resources/`) |
| 260 | +- `lal/` - Log processing rules |
| 261 | +- `log-mal-rules/` - Metrics extracted from logs |
| 262 | + |
| 263 | +## Important Links |
| 264 | + |
| 265 | +- Documentation: `docs/en/` folder (source for all published docs, structure defined in `docs/menu.yml`) |
| 266 | +- Change Logs: `docs/en/changes/changes.md` (update this file when making changes) |
| 267 | +- GitHub Issues: https://github.com/apache/skywalking/issues |
| 268 | +- Mailing List: dev@skywalking.apache.org |
| 269 | +- Slack: #skywalking channel at Apache Slack |
| 270 | + |
| 271 | +## Tips for AI Assistants |
| 272 | + |
| 273 | +1. **Always check submodules**: Protocol changes may require submodule updates |
| 274 | +2. **Generate sources first**: Run `mvnw compile` before analyzing generated code |
| 275 | +3. **Respect checkstyle**: No System.out, no @author, no Chinese characters |
| 276 | +4. **Follow module patterns**: Use existing modules as templates |
| 277 | +5. **Check multiple storage implementations**: Logic may vary by storage type |
| 278 | +6. **OAL generates code**: Don't manually edit generated metrics classes |
| 279 | +7. **Use Lombok**: Prefer annotations over boilerplate code |
| 280 | +8. **Test both unit and integration**: Different test patterns for different scopes |
0 commit comments