Skip to content

USAGE.md

Latest commit

 

History

History
276 lines (197 loc) · 5.24 KB

USAGE.md

File metadata and controls

276 lines (197 loc) · 5.24 KB
Raw

Usage

Architecture

The following diagram illustrates how the Kafka Auth Handler integrates with Kafka clients and cloud providers:

flowchart LR
    subgraph Client["Kafka Client"]
        KC[Kafka Consumer/Producer]
    end

    subgraph Handler["kafka-auth-handler"]
        HTTP[HTTP Server :14293]
        GCP[GCP Handler]
        AWS[AWS Handler]
    end

    subgraph CloudProviders["Cloud Providers"]
        GCPIAM[GCP IAM]
        AWSIAM[AWS IAM]
    end

    subgraph Kafka["Kafka Cluster"]
        MSK[AWS MSK]
        GMK[GCP Managed Kafka]
    end

    KC -->|OAUTHBEARER token request| HTTP
    HTTP -->|PROVIDER=gcp| GCP
    HTTP -->|PROVIDER=aws| AWS
    GCP -->|Application Default Credentials| GCPIAM
    AWS -->|IRSA / IAM Credentials| AWSIAM
    GCPIAM -->|OAuth2 Token| GCP
    AWSIAM -->|MSK IAM Token| AWS
    GCP -->|JWT-like token| HTTP
    AWS -->|JWT-like token| HTTP
    HTTP -->|access_token| KC
    KC -->|SASL/OAUTHBEARER| MSK
    KC -->|SASL/OAUTHBEARER| GMK
Loading

Using as a Library

The primary purpose of this module is to be used as a library in any Go HTTP server. The handler package provides an http.Handler implementation that can be mounted on any route.

Installation

go get github.com/martoc/kafka-auth-handler

Basic Integration

package main

import (
    "log"
    "net/http"
    "os"

    "github.com/martoc/kafka-auth-handler/handler"
)

func main() {
    // Create the auth handler based on provider
    provider := os.Getenv("PROVIDER") // "gcp" or "aws"
    region := os.Getenv("REGION")     // Required for AWS
    authHandler := handler.NewAuthHandler(provider, region)

    // Mount on your preferred route
    http.Handle("/oauth/token", authHandler)

    // Or use with your preferred router (e.g., gorilla/mux, chi, gin)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Provider-Specific Handlers

GCP Handler

authHandler := handler.NewGCPAuthHandlerBuilder().Build()

With custom Google service for testing:

authHandler := handler.NewGCPAuthHandlerBuilder().
    WithGoogleService(myCustomGoogleService).
    Build()

AWS Handler

authHandler := handler.NewAWSAuthHandlerBuilder().
    WithRegion("eu-central-1").
    Build()

With custom token generator for testing:

authHandler := handler.NewAWSAuthHandlerBuilder().
    WithRegion("eu-central-1").
    WithTokenGenerator(myMockTokenGenerator).
    Build()

Integration with Popular Routers

gorilla/mux

import "github.com/gorilla/mux"

r := mux.NewRouter()
r.Handle("/oauth/token", handler.NewAuthHandler("gcp", ""))

chi

import "github.com/go-chi/chi/v5"

r := chi.NewRouter()
r.Handle("/oauth/token", handler.NewAuthHandler("aws", "eu-central-1"))

gin

import "github.com/gin-gonic/gin"

r := gin.Default()
authHandler := handler.NewAuthHandler("gcp", "")
r.GET("/oauth/token", gin.WrapH(authHandler))

Response Format

Both GCP and AWS handlers return a JSON response with the following structure:

{
  "access_token": "<header>.<claims>.<token>",
  "token_type": "Bearer",
  "expires_in": 3600
}

The access_token is a JWT-like token composed of:

  • Base64-encoded header with type and algorithm
  • Base64-encoded claims (exp, iss, iat, sub)
  • Base64-encoded cloud provider access token

GCP Token Structure

Field Value
alg GOOG_OAUTH2_TOKEN
iss Google
sub GCP service account email

AWS Token Structure

Field Value
alg AWS_MSK_IAM
iss AWS

Standalone Server

The module also includes a standalone server for quick deployment.

Environment Variables

Variable Required Default Description
PROVIDER No gcp Cloud provider: gcp or aws
REGION Yes (AWS) - AWS region for MSK IAM token generation

From Source

make build

# GCP (default)
./target/builds/kafka-auth-handler-darwin-arm64 serve

# AWS
PROVIDER=aws REGION=eu-central-1 ./target/builds/kafka-auth-handler-darwin-arm64 serve

Binaries are built for darwin/linux on amd64/arm64 in ./target/builds/.

Docker

# GCP
docker pull martoc/kafka-auth-handler:latest
docker run -p 14293:14293 martoc/kafka-auth-handler:latest

# AWS
docker run -p 14293:14293 \
  -e PROVIDER=aws \
  -e REGION=eu-central-1 \
  martoc/kafka-auth-handler:latest

The standalone server listens on port 14293.

API

Get OAuth2 Token

curl http://localhost:14293/

AWS MSK Configuration

When using AWS MSK with IAM authentication:

  1. Use port 9098 for IAM authentication on your MSK bootstrap servers
  2. Set KAFKA_SECURITY_PROTOCOL=SASL_SSL
  3. Set KAFKA_SASL_MECHANISM=OAUTHBEARER
  4. Configure your Kafka client to use http://localhost:14293/ as the token endpoint
  5. Ensure your pod/service has proper IAM permissions (via IRSA on EKS)

Development

Build

make build

Run Tests

make test

Run Linter

make lint

Run Integration Tests

make run-integration-tests

Install Dependencies

make install

Regenerate Mocks

make generate

Format Code

make format