Test Tool Platform - Modular Testing Framework

A comprehensive modular platform for test automation tools, featuring XML report comparison, FIX protocol messaging, audio file comparison, and OUCH/ITCH protocol testing, built with Python, FastAPI, and SQLAlchemy.

Features

XML Comparison Module

XML Deep Comparison: Compare two XML documents with XPath-based difference tracking
Web UI: User-friendly interface for uploading and comparing XML reports
REST API: Programmatic access for automation and integration
Database Storage: All comparison results stored with full history
Filtering & Search: Query jobs by status, suite name, environment, and date range
Detailed Diff Reports: Track added, removed, and changed elements with full context

FIX Protocol Messaging Module

FIX Message Creation: Build and send FIX 4.4 protocol messages
Server Endpoint Integration: Send messages directly to FIX server via TCP socket
Configurable Endpoints: Specify host and port for each message or session
Order Management: Support for NewOrderSingle, OrderCancel, and other message types
Message History: Store and query all sent FIX messages with status tracking
Connection Monitoring: Track SENT, FAILED, ACKNOWLEDGED statuses
REST API: Send messages and track status programmatically
SimpleFIX Library: Pure Python implementation, no C++ dependencies
Test Server: Included test server for local development and testing

Audio Comparison Module

Audio File Analysis: Compare audio files for conformance and quality
Spectral Analysis: MFCC-based similarity scoring (0-100%)
Property Comparison: Duration, sample rate, channels, and format verification
Format Support: WAV, MP3, FLAC, OGG, M4A, and more
Web UI: Upload and compare audio files with detailed results
REST API: Programmatic audio comparison and analysis

OUCH/ITCH Protocol Testing Module

NASDAQ Protocols: Test OUCH (Order Entry) and ITCH (Market Data) protocols
SoupBinTCP Sessions: Manage protocol sessions with heartbeat monitoring
Message Testing: Create, send, and validate protocol messages
Conformance Testing: Protocol compliance verification
Message History: Track all sent and received messages
REST API: Programmatic protocol testing and validation

Tech Stack

Language: Python 3.11+
Web Framework: FastAPI
ORM: SQLAlchemy
Database: SQLite (default) / MySQL / MariaDB
Migrations: Alembic
Templating: Jinja2
HTTP Server: Uvicorn
Testing: pytest
FIX Protocol: SimpleFIX
Audio Processing: librosa, soundfile, numpy, scipy
NASDAQ Protocols: nasdaq-protocols

Modular Architecture

The platform uses a feature-based modular architecture where each major feature is a self-contained module:

app/modules/xml_compare/ - XML comparison functionality
app/modules/fix_messaging/ - FIX protocol messaging
app/modules/audio_compare/ - Audio file comparison
app/modules/protocol_testing/ - OUCH/ITCH protocol testing

See MODULAR_ARCHITECTURE.md for detailed architecture documentation.

Project Structure

multi/
├── app/
│   ├── modules/                    # Feature modules
│   │   ├── xml_compare/           # XML Comparison Module
│   │   │   ├── models.py          # Database models
│   │   │   ├── schemas.py         # Pydantic schemas
│   │   │   ├── service.py         # Business logic
│   │   │   └── job_service.py
│   │   ├── fix_messaging/         # FIX Protocol Module
│   │   │   ├── models.py
│   │   │   ├── schemas.py
│   │   │   └── service.py
│   │   └── audio_compare/         # Audio Comparison Module
│   │       ├── models.py
│   │       ├── schemas.py
│   │       └── service.py
│   ├── api/
│   │   ├── web.py                 # Web UI routes
│   │   ├── xml_compare.py         # XML REST API
│   │   ├── fix_messaging.py       # FIX REST API
│   │   └── audio_compare.py       # Audio REST API
│   ├── templates/
│   │   ├── base.html
│   │   ├── index.html
│   │   ├── new_comparison.html
│   │   ├── jobs_list.html
│   │   ├── job_detail.html
│   │   ├── fix_home.html
│   │   ├── fix_send.html
│   │   ├── fix_messages_list.html
│   │   ├── fix_message_detail.html
│   │   ├── audio_home.html
│   │   ├── audio_compare_form.html
│   │   ├── audio_comparisons_list.html
│   │   └── audio_comparison_detail.html
│   ├── config.py                  # Application configuration
│   ├── database.py                # Database setup
│   ├── models.py                  # Model exports
│   └── main.py                    # FastAPI app entry point
├── alembic/
│   ├── versions/
│   │   ├── 001_initial_migration.py
│   │   ├── 002_add_fix_messages.py
│   │   └── 003_add_audio_comparisons.py
│   └── env.py
├── tests/
│   ├── conftest.py
│   ├── test_xml_compare.py        # XML tests (9 tests)
│   ├── test_api.py                # XML API tests (11 tests)
│   ├── test_fix_messaging.py      # FIX tests (8 tests)
│   └── test_audio_compare.py      # Audio tests (12 tests)
├── requirements.txt
├── alembic.ini
├── .env.example
├── README.md
├── docs/
│   ├── FIX_MESSAGING_GUIDE.md     # FIX Protocol documentation
│   ├── AUDIO_COMPARISON_GUIDE.md  # Audio Comparison documentation
│   ├── MODULAR_ARCHITECTURE.md    # Architecture guide
│   └── SQLITE_BACKEND.md          # SQLite setup guide

Installation

Prerequisites

Python 3.11 or higher
pip and virtualenv
(Optional) MySQL 8.0+ or MariaDB 10.5+ for production use

Note: SQLite is used by default for easy setup. No database server required!

Setup Steps

Clone or navigate to the project directory

cd c:\multi

Create and activate virtual environment

python -m venv venv
.\venv\Scripts\Activate.ps1

Install dependencies

pip install -r requirements.txt

Configure environment variables

Copy .env.example to .env and update with your settings:

cp .env.example .env

Edit .env (SQLite is default, no changes needed):

# SQLite (default - no database setup required)
DATABASE_URL=sqlite:///./testtool.db

# Or use MySQL for production
# DATABASE_URL=mysql+pymysql://root:password@localhost:3306/testtool_db

APP_NAME=Test Tool Platform
DEBUG=True
HOST=0.0.0.0
PORT=8000

Run database migrations

alembic upgrade head

That's it! SQLite database will be created automatically.

Optional: MySQL Setup

If you prefer MySQL for production:

Install pymysql: pip install pymysql cryptography
Create database:

# Connect to MySQL and create database
mysql -u root -p

CREATE DATABASE testtool_db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
EXIT;

Update .env with MySQL URL
Run migrations: alembic upgrade head

Quick Start (SQLite)

Use the provided start script for instant setup:

.\start_sqlite.ps1

This will:

Create virtual environment (if needed)
Install dependencies
Set up SQLite database
Run migrations
Start the server

Running the Application

Development Server

# Using uvicorn directly
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Or using the start script (SQLite)
.\start_sqlite.ps1

# Or using the original start script (MySQL)
.\start.ps1

The application will be available at:

Web UI: http://localhost:8000
API Documentation: http://localhost:8000/api/docs
ReDoc: http://localhost:8000/api/redoc

Usage

Web Interface

Home Page: Navigate to http://localhost:8000
Create Comparison: Click "New Comparison" or visit http://localhost:8000/compare/new
Upload XML: Paste or upload baseline and current XML files
Add Metadata (optional): Enter suite name, environment, and run ID
Compare: Click "Compare XMLs" to run the comparison
View Results: See detailed diff report with XPath-based changes
Browse Jobs: Visit http://localhost:8000/jobs to see all comparison jobs

REST API

Create a Comparison Job

curl -X POST "http://localhost:8000/api/xml-compare/jobs" \
  -H "Content-Type: application/json" \
  -d '{
    "baseline_xml": "<testsuite><testcase name=\"TC1\" status=\"pass\"/></testsuite>",
    "current_xml": "<testsuite><testcase name=\"TC1\" status=\"fail\"/></testsuite>",
    "suite_name": "Regression Suite",
    "environment": "staging",
    "run_id": "run-123"
  }'

Response:

{
  "id": 1,
  "status": "COMPLETED",
  "suite_name": "Regression Suite",
  "environment": "staging",
  "run_id": "run-123",
  "summary": {
    "added": 0,
    "removed": 0,
    "changed": 1,
    "total_differences": 1,
    "status": "FAILED"
  },
  "differences": [
    {
      "type": "CHANGED",
      "path": "/testsuite/testcase[@name='TC1']/@status",
      "baseline_value": "pass",
      "current_value": "fail",
      "description": "Attribute 'status' changed"
    }
  ],
  "created_at": "2025-11-15T10:30:00",
  "started_at": "2025-11-15T10:30:00",
  "completed_at": "2025-11-15T10:30:01",
  "error_message": null
}

Get Job Details

curl -X GET "http://localhost:8000/api/xml-compare/jobs/1"

List Jobs with Filtering

# List all jobs
curl -X GET "http://localhost:8000/api/xml-compare/jobs"

# Filter by status
curl -X GET "http://localhost:8000/api/xml-compare/jobs?status=COMPLETED"

# Filter by suite and environment
curl -X GET "http://localhost:8000/api/xml-compare/jobs?suite_name=Regression%20Suite&environment=staging"

# Pagination
curl -X GET "http://localhost:8000/api/xml-compare/jobs?page=2&page_size=10"

Delete a Job

curl -X DELETE "http://localhost:8000/api/xml-compare/jobs/1"

XML Comparison Logic

The comparison engine provides:

Features

Deep Comparison: Recursively compares all elements, attributes, and text content
XPath Identification: Uses XPath expressions to identify changed elements
Normalization: Ignores insignificant whitespace and attribute order
Change Types:
- ADDED: Elements present only in current XML
- REMOVED: Elements present only in baseline XML
- CHANGED: Elements with different attributes or text content

Example Comparison

Baseline XML:

<testsuite name="Suite" tests="2">
    <testcase name="TC1" time="1.23" status="pass">
        <result>PASSED</result>
    </testcase>
    <testcase name="TC2" time="0.56" status="pass">
        <result>PASSED</result>
    </testcase>
</testsuite>

Current XML:

<testsuite name="Suite" tests="3">
    <testcase name="TC1" time="1.50" status="pass">
        <result>PASSED</result>
    </testcase>
    <testcase name="TC2" time="0.56" status="fail">
        <result>FAILED</result>
    </testcase>
    <testcase name="TC3" time="2.00" status="pass">
        <result>PASSED</result>
    </testcase>
</testsuite>

Detected Differences:

CHANGED: /testsuite/@tests (2 → 3)
CHANGED: /testsuite/testcase[@name='TC1']/@time (1.23 → 1.50)
CHANGED: /testsuite/testcase[@name='TC2']/@status (pass → fail)
CHANGED: /testsuite/testcase[@name='TC2']/result/text() (PASSED → FAILED)
ADDED: /testsuite/testcase[@name='TC3']

FIX Protocol Messaging

The platform includes FIX protocol support for financial message sending.

Send a FIX Message

curl -X POST "http://localhost:8000/api/fix/messages" \
  -H "Content-Type: application/json" \
  -d '{
    "msg_type": "D",
    "sender_comp_id": "SENDER",
    "target_comp_id": "TARGET",
    "cl_ord_id": "ORDER123",
    "symbol": "AAPL",
    "side": "1",
    "order_qty": "100",
    "ord_type": "2",
    "price": "150.50",
    "time_in_force": "0"
  }'

List FIX Messages

curl -X GET "http://localhost:8000/api/fix/messages"

FIX Message Types Supported

D: NewOrderSingle - Submit new order
F: OrderCancelRequest - Cancel order
0: Heartbeat - Connection heartbeat
A: Logon - Session logon
And more...

See FIX_MESSAGING_GUIDE.md for complete FIX protocol documentation.

Testing

Run All Tests (28 tests)

pytest

Run Module-Specific Tests

# XML comparison tests (9 tests)
pytest tests/test_xml_compare.py

# XML API tests (11 tests)
pytest tests/test_api.py

# FIX messaging tests (8 tests)
pytest tests/test_fix_messaging.py

Run with Coverage

pytest --cov=app --cov-report=html

Test Coverage

The test suite includes:

XML Module: Comparison engine, API endpoints, edge cases (20 tests)
FIX Module: Message creation, listing, filtering, deletion (8 tests)
Total: 28 passing tests

Database Migrations

Create a New Migration

alembic revision --autogenerate -m "Description of changes"

Apply Migrations

# Upgrade to latest
alembic upgrade head

# Upgrade to specific revision
alembic upgrade <revision_id>

Rollback Migrations

# Downgrade one revision
alembic downgrade -1

# Downgrade to specific revision
alembic downgrade <revision_id>

View Migration History

alembic history
alembic current

API Documentation

FastAPI provides interactive API documentation:

Swagger UI: http://localhost:8000/api/docs
ReDoc: http://localhost:8000/api/redoc
OpenAPI Schema: http://localhost:8000/api/openapi.json

Configuration

Environment Variables

Variable	Description	Default
`DATABASE_URL`	MySQL connection string	`mysql+pymysql://root:password@localhost:3306/testtool_db`
`APP_NAME`	Application name	`Test Tool Platform`
`DEBUG`	Debug mode	`True`
`HOST`	Server host	`0.0.0.0`
`PORT`	Server port	`8000`

Database Configuration

The platform supports MySQL and MariaDB. To switch databases, update the DATABASE_URL in .env:

MySQL:

DATABASE_URL=mysql+pymysql://user:password@host:3306/database

MariaDB:

DATABASE_URL=mysql+pymysql://user:password@host:3306/database

SQLite (for testing):

DATABASE_URL=sqlite:///./testtool.db

Troubleshooting

Common Issues

1. Database Connection Error

sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server...")

Verify MySQL is running
Check DATABASE_URL in .env
Ensure database exists

2. Import Errors

ModuleNotFoundError: No module named 'fastapi'

Activate virtual environment: .\venv\Scripts\Activate.ps1
Install dependencies: pip install -r requirements.txt

3. Migration Errors

alembic.util.exc.CommandError: Target database is not up to date.

Run migrations: alembic upgrade head

4. Template Not Found

jinja2.exceptions.TemplateNotFound: base.html

Ensure you're running from project root: cd c:\multi
Verify app/templates/ directory exists

Production Deployment

Recommended Setup

Use Production Database: Configure MySQL with appropriate credentials
Disable Debug Mode: Set DEBUG=False in .env
Use Production Server: Deploy with Gunicorn + Uvicorn workers
Add Reverse Proxy: Use Nginx or Apache as reverse proxy
Enable HTTPS: Configure SSL/TLS certificates
Set Up Monitoring: Use tools like Prometheus, Grafana
Configure Backups: Regular database backups

Production Server Command

gunicorn app.main:app -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000

Future Enhancements

File upload support (instead of paste)
Background job processing with Celery
Export diff reports (PDF, CSV)
Email notifications for job completion
User authentication and authorization
Comparison history and trending
Custom diff rules and configurations
Support for other file formats (JSON, YAML)

License

This project is for educational and internal use.

Support

For issues and questions, please check the API documentation at /api/docs or review the test files for usage examples.

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
alembic		alembic
app		app
docs		docs
examples		examples
migrations/versions		migrations/versions
tests		tests
.env.example		.env.example
.gitignore		.gitignore
ARCHITECTURE.txt		ARCHITECTURE.txt
FIX_MESSAGING_GUIDE.md		FIX_MESSAGING_GUIDE.md
IMPLEMENTATION_COMPLETE.txt		IMPLEMENTATION_COMPLETE.txt
MODULAR_APP_COMPLETE.md		MODULAR_APP_COMPLETE.md
MODULAR_ARCHITECTURE.md		MODULAR_ARCHITECTURE.md
PROJECT_SUMMARY.md		PROJECT_SUMMARY.md
QUICK_REFERENCE.md		QUICK_REFERENCE.md
README.md		README.md
SQLITE_BACKEND.md		SQLITE_BACKEND.md
TEST_RESULTS.md		TEST_RESULTS.md
TEST_SUMMARY.md		TEST_SUMMARY.md
alembic.ini		alembic.ini
check_db.py		check_db.py
pytest.ini		pytest.ini
requirements.txt		requirements.txt
setup.py		setup.py
start.ps1		start.ps1
start_sqlite.ps1		start_sqlite.ps1
test_app.py		test_app.py
test_fix_server.py		test_fix_server.py
test_quick.py		test_quick.py
test_sqlite.py		test_sqlite.py
verify_build.py		verify_build.py

Folders and files

Latest commit

History

Repository files navigation

Test Tool Platform - Modular Testing Framework

Features

XML Comparison Module

FIX Protocol Messaging Module

Audio Comparison Module

OUCH/ITCH Protocol Testing Module

Tech Stack

Modular Architecture

Project Structure

Installation

Prerequisites

Setup Steps

Optional: MySQL Setup

Quick Start (SQLite)

Running the Application

Development Server

Usage

Web Interface

REST API

Create a Comparison Job

Get Job Details

List Jobs with Filtering

Delete a Job

XML Comparison Logic

Features

Example Comparison

FIX Protocol Messaging

Send a FIX Message

List FIX Messages

FIX Message Types Supported

Testing

Run All Tests (28 tests)

Run Module-Specific Tests

Run with Coverage

Test Coverage

Database Migrations

Create a New Migration

Apply Migrations

Rollback Migrations

View Migration History

API Documentation

Configuration

Environment Variables

Database Configuration

Troubleshooting

Common Issues

Production Deployment

Recommended Setup

Production Server Command

Future Enhancements

License

Support

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages