🚀 Spring Boot Microservices Series V2

A comprehensive microservices architecture showcase using modern cloud-native technologies

Building resilient, scalable, and observable distributed systems with Spring Boot ecosystem

---

📖 Table of Contents

• 🎯 Overview
• ✨ Key Features
• 🏗️ Architecture
• 🔧 Tech Stack
• 🚀 Quick Start
• 🎛️ Service Discovery & API Access
• 📊 Observability
• 🗃️ Database Management
• 📦 Services Overview
• 💡 Development Tips
• 🤝 Contributing
• 📄 License

🎯 Overview

This project demonstrates a complete microservices architecture built with Spring Boot, Spring Cloud, and modern cloud-native technologies. It serves as a practical guide for developers looking to understand and implement production-ready distributed systems.

✨ Key Features

🏢 Microservice Architecture Decoupled services for different business domains Independent deployment and scaling Domain-driven design principles 🌱 Spring Boot & Spring Cloud Service Registry (Eureka) API Gateway (Spring Cloud Gateway) Configuration Server Circuit Breakers & Load Balancing	🐳 Containerization Dockerized services Docker Compose orchestration Consistent development environments 📡 Event-Driven Communication Apache Kafka message streaming Asynchronous service communication Event sourcing patterns
📊 Comprehensive Observability Prometheus metrics collection Grafana dashboards Distributed tracing with Micrometer Centralized logging 🗄️ Database Diversity PostgreSQL Liquibase schema management Multiple changelog formats (XML, YAML, JSON)	🔍 Service Discovery Eureka service registry Dynamic service registration Health check monitoring 🛡️ Production Ready Security with OAuth2/OIDC Performance testing with Gatling CI/CD with GitHub Actions

🏗️ Architecture

This project implements a microservices pattern where different functionalities are broken down into independent services. These services communicate asynchronously via Apache Kafka and are managed through Spring Cloud Gateway and Eureka Service Discovery.

Architecture Highlights:

• 🌐 API Gateway as the single entry point
• 📋 Service Registry for dynamic service discovery
• ⚙️ Configuration Server for centralized config management
• 📨 Event-driven communication via Kafka
• 📊 Distributed monitoring and tracing

📦 Services Overview

Service	Port	Description	Tech Stack
🌐 API Gateway	8765	Single entry point, routing & load balancing	Spring Cloud Gateway
📁 Config Server	8888	Centralized configuration management	Spring Cloud Config
🏢 Service Registry	8761	Service discovery with Eureka	Spring Cloud Netflix
📚 Catalog Service	18080	Product catalog management	PostgreSQL + Liquibase (YAML)
📦 Inventory Service	18181	Stock level management	PostgreSQL + Liquibase (JSON)
🛍️ Order Service	18282	Order processing & orchestration	PostgreSQL + Liquibase (XML)
💳 Payment Service	18085	Payment processing	PostgreSQL + Liquibase (XML)
🛒 Retail Store Web	8080	Customer-facing web application	Thymeleaf + Alpine.js

🔗 Service Interactions

graph TB
    UI[🛒 Retail Store WebApp] --> GW[🌐 API Gateway]
    GW --> CS[📚 Catalog Service]
    GW --> IS[📦 Inventory Service]
    GW --> OS[🛍️ Order Service]
    GW --> PS[💳 Payment Service]
    
    OS -.->|Events| Kafka[📡 Kafka]
    PS -.->|Events| Kafka
    IS -.->|Events| Kafka
    
    SR[🏢 Service Registry] -.->|Discovery| GW
    SR -.->|Discovery| CS
    SR -.->|Discovery| IS
    SR -.->|Discovery| OS
    SR -.->|Discovery| PS
    
    CC[📁 Config Server] -.->|Configuration| GW
    CC -.->|Configuration| CS
    CC -.->|Configuration| IS
    CC -.->|Configuration| OS
    CC -.->|Configuration| PS
    
    style UI fill:#e1f5fe
    style GW fill:#f3e5f5
    style Kafka fill:#fff3e0
    style SR fill:#e8f5e8
    style CC fill:#fce4ec

Loading

🔧 Tech Stack

🖥️ Core Technologies

Languages & Frameworks

• Java 21+
• Spring Framework
• Spring Cloud
• Project Reactor

📡 Communication & Messaging

Message Brokers & Streaming

• Apache Kafka
• Kafka Streams
• Kafka REST

🗄️ Data Storage & Management

Databases

• PostgreSQL
• MongoDB
• Redis

Database Tools

• Liquibase
• Spring Data
• jOOQ

🐳 DevOps & Deployment

Containerization & CI/CD

• Docker
• Docker Compose
• Kubernetes deployment guide
• Kubernetes E2E workflow
• GitHub Actions

📊 Monitoring & Observability

Metrics & Tracing

• Prometheus
• Grafana
• Micrometer

Logging

• Logback

🧪 Testing

Testing Tools

• JUnit 5
• Gatling (Performance Testing)
• Testcontainers

📋 For complete tech stack details, see techstack.md

🚀 Quick Start

📋 Prerequisites

Before you begin, ensure you have the following installed:

Tool	Version	Purpose
☕ Java	25+	Runtime environment
📦 Maven	3.9.x+	Build tool
🐳 Docker	Latest	Containerization
🔧 Docker Compose	Latest	Orchestration
Kind	Latest	Local Kubernetes cluster
kubectl	Latest	Kubernetes CLI
📚 Git	Latest	Version control

💡 Tip: Ensure JAVA_HOME environment variable is properly set

🔨 Build the Project

1. Clone the repository:

   git clone https://github.com/rajadilipkolli/spring-boot-microservices-series-v2.git
   cd spring-boot-microservices-series-v2

2. Build all modules:

   🐧 Linux/macOS:

   ./mvnw clean install

   🪟 Windows:

   .\mvnw.cmd clean install

🚀 Running the Application

Choose one of the following deployment options:

Option 1: Core Services Only

Perfect for development and basic functionality testing:

docker-compose -f deployment/docker-compose.yml up -d --remove-orphans

Option 2: Full Stack with Monitoring

Includes Prometheus, Grafana, and other observability tools:

docker-compose -f deployment/docker-compose-tools.yml up -d --remove-orphans

Option 3: Using Helper Scripts

🐧 Linux/macOS:

bash run.sh

🪟 Windows:

.\start-services.ps1

Option 4: Local Kubernetes with Kind

For a local Kubernetes deployment, use Kind with the manifests under deployment/k8s/. See the Kubernetes Deployment Guide for the complete setup and validation flow.

./deployment/scripts/run-local-kubernetes-e2e.sh

The same Kubernetes end-to-end gate runs in k8s-e2e.yml.

🛑 Stopping Services

# Stop core services
docker-compose -f deployment/docker-compose.yml down

# Stop services with monitoring tools
docker-compose -f deployment/docker-compose-tools.yml down

✅ Verify Installation

Once all services are running, verify the setup:

• 🌐 Service Registry: http://localhost:8761
• 📚 API Documentation: http://localhost:8765/swagger-ui.html
• 🛒 Retail Store App: http://localhost:8080
• 📊 Grafana Dashboard: http://localhost:3000 (user/password)
• 🔍 Loki Tracing: http://localhost:3000

🎛️ Service Discovery & API Access

🔍 Service Registry (Eureka)

Monitor all registered microservices and their health status:

• Dashboard: http://localhost:8761/
• Features: Real-time service health, load balancing, failover

📚 API Gateway & Documentation

Access all microservice APIs through a unified interface:

• Swagger UI: http://localhost:8765/swagger-ui.html
• Features: API aggregation, rate limiting, authentication

💡 Pro Tip: Use the dropdown menu in Swagger UI to switch between different service APIs

🔐 Keycloak Security Configuration

To fully access all features of the application (such as the Inventory UI), you must assign your user the ADMIN role. Follow these steps to correctly configure the role and ensure it is included in the OIDC tokens so Spring Security can map it:

1. Create the ADMIN Client Role

1. Open the Keycloak Admin Console (http://localhost:9191).
2. Select the retailstore realm.
3. Go to Clients -> click on retailstore-webapp.
4. Go to the Roles tab and click Create Role.
5. Name the role ADMIN and click Save.

2. Map the Client Role to the ID Token

By default, Keycloak only puts client roles inside the Access Token. To ensure the retailstore-webapp can see the role during login, we need to map it to the ID Token.

1. On the left menu, go to Client Scopes.
2. Click on the roles scope in the list.
3. Go to the Mappers tab.
4. Click on the client roles mapper.
5. Toggle ON both Add to ID token and Add to userinfo.
6. Click Save.

3. Assign the Role to Your User

1. On the left menu, go to Users.
2. Search for your user (e.g., raja) and click on the username.
3. Go to the Role mapping tab.
4. Click Assign role.
5. From the dropdown at the top, select Filter by clients.
6. Find retailstore-webapp ADMIN, check the box, and click Assign.

📊 Observability

📈 Metrics & Monitoring with Prometheus & Grafana

🔍 Prometheus URL: http://localhost:9090 Purpose: Metrics collection and alerting Features: Custom queries, alert rules, service discovery 📊 Grafana URL: http://localhost:3000 Credentials: user/password Features: Beautiful dashboards, alerting, data visualization

🚨 Alertmanager URL: http://localhost:9093 Purpose: Alert routing and management Features: Email, Slack, webhook notifications 📋 Key Metrics Monitored: ⚡ Response times & throughput 💾 Memory & CPU usage 🔄 Service health & availability 📊 Business metrics & KPIs

🔍 Distributed Tracing

LGTM Integration

• URL: http://localhost:3000/
• Features: Request flow visualization, latency analysis, dependency mapping
• Integration: Micrometer Tracing with Spring Boot

📊 Alerting Architecture

---
title: 🚨 Monitoring & Alerting Flow
---
flowchart TD
    subgraph "📊 Microservices"
        MS1[📚 Catalog Service]
        MS2[📦 Inventory Service]
        MS3[🛍️ Order Service]
        MS4[💳 Payment Service]
    end

    PROM[🔍 Prometheus Server]
    AM[🚨 AlertManager]
    GRAF[📊 Grafana]
    RULES[📋 Alert Rules]
    
    MS1 --> PROM
    MS2 --> PROM
    MS3 --> PROM
    MS4 --> PROM
    
    PROM --> RULES
    RULES --> AM
    AM --> |📧 Email| EMAIL[📧 Notifications]
    AM --> |💬 Slack| SLACK[💬 Chat Alerts]
    
    PROM --> GRAF
    
    style PROM fill:#ff6b6b
    style AM fill:#4ecdc4
    style GRAF fill:#45b7d1
    style RULES fill:#f9ca24

Loading

🗃️ Database Management

📊 Liquibase Schema Management

This project demonstrates flexible database schema management using Liquibase with various changelog formats:

🏷️ Format	📁 Service Examples	📝 Use Case
XML	order-service, payment-service	Complex migrations, detailed documentation
YAML	catalog-service	Human-readable, simple structure
JSON	inventory-service	API-friendly, structured data
SQL	Custom implementations	Direct SQL control, legacy migrations

📂 Changelog Location

src/main/resources/db/changelog/
├── db.changelog-master.xml     # Master changelog file
├── migrations/
│   ├── 001-initial-schema.xml
│   ├── 002-add-indexes.yaml
│   └── 003-seed-data.json

🗄️ Database Technologies

🐘 PostgreSQL Services 📚 Catalog Service 📦 Inventory Service 🛍️ Order Service 💳 Payment Service Features: ACID compliance, complex queries, relational integrity

💡 Best Practice: Each service manages its own database schema independently, following the database-per-service pattern

💡 Development Tips

🐳 Essential Docker Commands

🧹 Cleanup Commands

# 🗑️ Clean up entire Docker system (nuclear option)
docker system prune -a -f --volumes

# 📦 Remove unused volumes only
docker volume prune -f

# 🔍 List all running containers
docker ps

# 📋 List all containers (including stopped)
docker ps -a

📋 Logging Commands

# 📊 View logs for all services (with monitoring tools)
docker-compose -f deployment/docker-compose-tools.yml logs -f

# 🎯 View logs for specific service
docker-compose -f deployment/docker-compose.yml logs -f order-service

# 📜 Follow logs from last 100 lines
docker-compose -f deployment/docker-compose-tools.yml logs --tail=100 -f

# 🔍 Search logs for specific patterns
docker-compose logs | grep ERROR

📋 Kubernetes Commands Cheat‑Sheet

This guide provides a concise reference of useful kubectl commands for everyday debugging, maintenance, and inspection of the retailstore namespace used by this project. Each command includes a short description of when to use it.

---

1. Viewing Logs

Command	Purpose
kubectl logs deployment/order-service -n retailstore --previous | Select-Object -Last 30	Show the last 30 lines of the previous container instance (useful after a crash/restart).
kubectl logs deployment/order-service -n retailstore --previous	Get the full logs of the previous container run (full crash dump).
kubectl logs deployment/inventory-service -n retailstore --tail=40 2>&1	Show the most recent 40 lines of the inventory‑service deployment logs (including errors).
kubectl logs deployment/keycloak -n retailstore | Select-Object -Last 50	Retrieve the last 50 lines of the keycloak deployment logs.

---

2. Cleaning Up & Restarting Pods

Command	Purpose
kubectl delete pods -l app=keycloak -n retailstore	Delete all pods with label app=keycloak normally; they will be recreated by the Deployment controller.
kubectl rollout restart deployment config-server -n retailstore	Trigger a rolling restart of the config‑server deployment (e.g., after config changes).

---

3. Inspecting Resources

Command	Purpose
kubectl get deployment keycloak -n retailstore -o yaml	Dump the full YAML definition of the keycloak Deployment for inspection or debugging.
kubectl get pods -n retailstore -l app=keycloak; kubectl describe pod -l app=keycloak -n retailstore	List all Keycloak pods and then show a detailed description (events, conditions, env vars).
kubectl get pods -l app=keycloak -n retailstore -w	Watch the Keycloak pods in real‑time; useful while waiting for them to become Ready.
kubectl wait --for=condition=ready pod -l app=keycloak -n retailstore --timeout=60s	Block until all Keycloak pods reach the Ready condition (or timeout).
kubectl get all -n retailstore	Overview of every resource (pods, services, deployments, etc.) in the retailstore namespace.

---

4. Port‑Forwarding (local testing)

Command	Purpose
kubectl port-forward service/order-service 28282:28282 -n retailstore	Expose the order-service port locally on 28282 to interact with the service from your workstation.

---

5. General Tips

• Namespace: All commands target the retailstore namespace (-n retailstore). Adjust if you work in a different namespace.
• Label selectors (-l) are handy for batch operations on a set of pods.
• Use --previous to see logs from a terminated container (useful after a CrashLoopBackOff).
• Combine commands with PowerShell pipelines (| Select-Object -Last N) to trim output.
• For scripted CI pipelines, prefer kubectl wait to ensure resources are ready before proceeding.

🚫 Kill Processes by Port

🪟 Windows (PowerShell/CMD)

# 🔍 Find process using port 18080
netstat -ano | findstr :18080

# ⚡ Kill process by PID
taskkill /PID <PID_FROM_ABOVE> /F

# 🎯 One-liner to kill process on port
$process = Get-NetTCPConnection -LocalPort 18080 -ErrorAction SilentlyContinue
if ($process) { Stop-Process -Id $process.OwningProcess -Force }

🐧 Linux/macOS (Terminal)

# 🔍 Find process using port 18080
sudo lsof -i :18080

# ⚡ Kill process by PID
kill -9 <PID_FROM_ABOVE>

# 🎯 One-liner to kill process on port
sudo kill -9 $(sudo lsof -t -i:18080)

# 🔥 Alternative using fuser
sudo fuser -k 18080/tcp

⚠️ Important Notes & Best Practices

🔄 Spring Boot 3.x Migration Notes

• 📦 Jakarta EE Namespace: Migration from javax.* to jakarta.*
• 📊 Observability: Spring Cloud Sleuth → Micrometer Tracing
• 🔧 Configuration: Updated property names and patterns
• 🛡️ Security: Enhanced OAuth2/OIDC integration

🧪 Testing Best Practices

// 💰 BigDecimal testing with precision handling
import static org.hamcrest.Matchers.closeTo;

// ❌ Don't do this:
// .andExpected(jsonPath("$.totalPrice").value(100.00))

// ✅ Do this instead:
.andExpect(jsonPath("$.totalPrice").value(closeTo(new BigDecimal("100.00"), new BigDecimal("0.01"))))

🏗️ Architecture Considerations

• 💾 Transaction Management: Use @Transactional directly on jOOQ repository methods
• 📊 Event Sourcing: Kafka integration for reliable message delivery
• 🚀 Native Images: Some services may need additional GraalVM configuration. Note: For native builds, we cannot use @RefreshScope, OpenTelemetry Java agents, or Spring Boot DevTools.
• 🔍 Service Discovery: Health checks are crucial for proper load balancing
• 🔑 TSID Generation: If a deployment does not set a unique tsid.node or TSID_NODE for each generator, replicas can share a node and produce colliding IDs during an overlapping timestamp and counter window. Configure unique node IDs, or inject an explicitly configured TSID.Factory, for this service and the other TSID generators. The Hypersistence documentation states that exclusive node IDs avoid collisions and that the node is random when no setting is provided. This project's TSID implementation will fallback to this random node library default when withNode is omitted.

🎯 Performance Testing

# 🚀 Run Gatling performance tests
cd gatling-tests

# 📊 Basic performance test
./mvnw gatling:test

# 🔥 Stress test with custom parameters
./mvnw gatling:test -P stress -DmaxUsers=100 -DrampDurationMinutes=5

🤝 Contributing

We welcome contributions from the community! Here's how you can help:

🚀 How to Contribute

1. 🍴 Fork the repository
2. 🌟 Create a feature branch: git checkout -b feature/amazing-feature
3. 💻 Make your changes with clear, tested code
4. 📝 Commit your changes: git commit -m 'Add amazing feature'
5. 📤 Push to the branch: git push origin feature/amazing-feature
6. 🔄 Open a Pull Request

📋 Contribution Guidelines

• 📖 Documentation: Update documentation for any new features
• 🧪 Testing: Add tests for new functionality
• 🎨 Code Style: Follow existing code conventions
• 💬 Discussion: Open an issue first for significant changes
• 🏗️ Kafka Event Contract: Kafka event DTOs are resolved by explicit consumer-side typing (JsonSerde<T> / manual readValue(..., Dto.class)), not by the type/__TypeId__ header. Producers must not add type headers.

🤗 Community Standards

Please read our Code of Conduct to understand our community standards and expectations.

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

MIT License - Feel free to use, modify, and distribute! 🎉

---

🙏 Acknowledgements & References

📚 Inspiration & Resources

• 🌟 Spring Ecosystem: Built on the robust Spring Boot and Spring Cloud frameworks
• 📖 Microservices Patterns: Implementing industry best practices for distributed systems
• 📝 Community Insights: Distributed Transactions in Microservices with Kafka Streams and Spring Boot

🎯 Special Thanks

• 🏗️ Architecture Patterns: Following domain-driven design principles
• 🔧 DevOps Practices: Docker, monitoring, and observability best practices
• 👥 Community: All contributors and users who make this project better

---

⭐ If this project helped you, please give it a star! ⭐

Made with ❤️ by the Spring Boot Microservices Community

🐛 Report Bug • ✨ Request Feature • 💬 Discussions