Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions GEMINI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Gemini CLI Context: MRI Viewer (med3web)

This file provides context and automation instructions for Gemini CLI when working on the `med3web` project.

## Project Information
- **Repository:** [epam/med3web](https://github.com/epam/med3web)
- **Main Stack:** React 17, Three.js, Vite, Vitest.
- **Domain:** Medical 2D/3D volumetric data visualization (DICOM, NIfTI, KTX).

## AI Automation & GitHub Integration
Gemini CLI should use the `gh` CLI to interact with the repository.

### Issue Management
- **List Open Issues:** `gh issue list --repo epam/med3web`
- **View Issue Details:** `gh issue view <id> --repo epam/med3web`
- **Create a New Issue:** `gh issue create --repo epam/med3web --title "Title" --body "Body"`
- **Add Comments:** `gh issue comment <id> --repo epam/med3web --body "Message"`

### Workflows
- **Bug Fixes:**
1. Fetch issue details using `gh issue view`.
2. Reproduce the bug with a test case in `src/`.
3. Apply the fix.
4. Run `npm run test` to verify.
- **Feature Development:**
1. Research existing implementation in `src/engine/` or `src/ui/`.
2. Implement and add tests.
3. Verify with `npm run build`.

## Common Commands
- `npm install`: Install dependencies.
- `npm run dev`: Start development server.
- `npm run build`: Build for production.
- `npm run test`: Run Vitest suites.
- `npm run lint`: Run ESLint.
142 changes: 142 additions & 0 deletions docs/RENDERING_CONTROLS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# 3D Rendering Controls & Settings

A detailed guide to the visualization tools available in the 3D viewing mode, explaining their purpose, medical application, and underlying technology.

---

## 1. Isosurface (Thresholding)

**Description:**
Renders a 3D surface representing a specific density value. Only tissues matching exactly this density are visible, creating a "skin" extraction effect.

**Medical Discovery:**
Crucial for segmentation. It allows clinicians to isolate specific structures with distinct densities, such as bone surfaces, contrast-filled vessels, or skin boundaries, hiding everything else.

**Under the Hood:**
The ray marching shader checks each sampled voxel's intensity. If it crosses the user-defined threshold, a surface point is calculated using gradient-based lighting (Phong shading) to simulate depth and texture.

---

## 2. Ambient Occlusion

**Description:**
Adds soft shadows to crevices and deep areas of the 3D model, making the shape and depth much easier to perceive.

**Medical Discovery:**
Enhances spatial relationships between complex anatomical structures (e.g., sulci in the brain, folding of the colon). Without it, 3D structures can look flat and featureless.

**Under the Hood:**
Implemented as a screen-space effect (SSAO) or pre-calculated volume shadowing. It estimates how exposed each surface point is to "ambient" light; recessed areas receive less light and appear darker.

---

## 3. Maximum Intensity Projection (MIP)

**Description:**
Instead of accumulating color/opacity along a ray, this mode displays only the *brightest* voxel encountered along each viewing ray.

**Medical Discovery:**
The gold standard for vascular imaging (angiography). It allows blood vessels (which are bright with contrast agent) to be clearly visualized through surrounding soft tissue, even if they are deep inside the volume.

**Under the Hood:**
The ray caster traverses the volume and keeps track of the maximum intensity value found. The final pixel color corresponds directly to this maximum value, effectively "flattening" the brightest structures onto the screen.

---

## 4. Opacity (Transfer Function Alpha)

**Description:**
Controls the transparency of the volume. Lower opacity makes the volume more "ghost-like," allowing you to see internal structures through outer layers.

**Medical Discovery:**
Useful for visualizing overlapping structures, such as a tumor inside an organ or the position of an implant relative to surrounding bone.

**Under the Hood:**
Modifies the Alpha channel in the Transfer Function. During ray compositing, this value determines how much each sample contributes to obstructing the background.

---

## 5. Cut (Clipping Plane)

**Description:**
A virtual knife that slices through the 3D volume, removing one side to reveal the interior cross-section.

**Medical Discovery:**
Essential for virtually dissecting organs to inspect internal pathologies (e.g., inside heart chambers) without obstructions from the chest wall or ribs.

**Under the Hood:**
The shader discards any fragments or ray samples that lie on the "negative" side of a user-defined mathematical plane equation ($Ax + By + Cz + D = 0$).

---

## 6. Brightness & Contrast

**Description:**
**Brightness** shifts the overall image intensity.
**Contrast** expands or shrinks the range of displayed visible intensities (window width/level).

**Medical Discovery:**
Standard for all radiology reading. Adjusting these helps differentiate tissues with very subtle density differences (e.g., gray matter vs. white matter in the brain).

**Under the Hood:**
Applied as a post-processing step or directly during ray sample mapping. It linearly transforms the input voxel intensity $I$ before looking up the color: $I_{final} = (I - \text{Level}) \times \text{Width} + \text{Brightness}$.

---

## 7. Quality (Step Size)

**Description:**
Adjusts the fidelity of the rendering. High quality is sharper but may be slower; low quality is faster but may look "layered" or blurry.

**Medical Discovery:**
Allows the user to balance performance vs. detail. High quality is needed for final diagnosis; low quality is useful for smooth rotation and manipulation on slower computers.

**Under the Hood:**
Controls the **step size** of the ray marching algorithm. A smaller step size means more samples per ray (higher precision, higher GPU load). A larger step size misses fine details but renders much faster.

---

## 8. Volume Eraser

**Description:**
Manual tool to "paint" over parts of the volume to permanently hide them from the 3D view.

**Medical Discovery:**
Used to manually remove artifacts, skull stripping (removing bone to see brain), or removing table/headrest structures that clutter the view.

**Under the Hood:**
Uses a secondary 3D texture (Mask Volume). When the user "erases," the tool writes zeros into this mask texture at the corresponding 3D coordinates. The shader checks this mask; if the value is 0, the voxel is skipped during rendering.


---

## 9. Set (Transfer Function Histogram/Sliders)

**Description:**
A set of three sliders overlaying a density histogram (shown in the screenshot).
* **Left Slider (Min):** Hides everything below this density (noise, air, low-density tissue).
* **Middle Slider (Peak/Mid):** Defines the density range that should be most visible or opaque.
* **Right Slider (Max):** Hides everything above this density or defines the upper limit of the contrast window.

**Medical Discovery:**
This IS the primary tool for "windowing" in 3D. It allows the user to dynamically tune the visible tissue range.
* *Example:* Slide left to see skin, slide middle to see muscle/organs, slide right to see only bone and contrast dye.

**Under the Hood:**
These sliders define the **control points** of the Transfer Function.
* The system creates a piecewise linear function (a ramp or a curve) based on these three positions.
* Voxel intensities map to this function to determine their Opacity (Alpha) and Color.
* The histogram in the background shows the distribution of tissue densities in the dataset, helping the user know where to place the sliders (e.g., placing the middle slider on a tall peak to highlight that specific tissue).

---

## 10. ROI (Region of Interest) Selection

**Description:**
Allows highlighting or isolating specific segmented regions (e.g., "Lung", "Heart", "Tumor") if the dataset contains pre-calculated segmentation masks.

**Medical Discovery:**
Focuses the view on a specific organ or pathology while ghosting out the rest of the anatomy for context.

**Under the Hood:**
The dataset includes an additional byte-map where each voxel holds an ID. The renderer creates a specific color map assigned to each ID, allowing selective visibility rendering based on the voxel's integer ID.
61 changes: 61 additions & 0 deletions docs/RENDERING_PIPELINE_OVERVIEW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# 3D Medical Image Rendering Pipeline

A high-level overview of how volumetric medical images are processed and rendered in 3D.

---

## 1. File Ingestion

The application accepts standard medical imaging formats (DICOM, NIfTI). File headers are inspected to determine the format and route to the appropriate parser.

---

## 2. Metadata Extraction

Each image slice is parsed to extract critical metadata: pixel dimensions, physical spacing between pixels, slice position in 3D space, and patient orientation vectors.

---

## 3. Multi-Series Handling

Medical studies often contain multiple scans (e.g., different anatomical views or time points). Slices are grouped by shared geometric properties to separate distinct series for individual viewing.

---

## 4. Volume Assembly

Slices within a series are sorted by their physical position along the scan axis. The ordered 2D images are stacked to form a 3D voxel grid representing the patient's anatomy.

---

## 5. GPU Transfer

The 3D voxel data is uploaded to graphics memory as a volumetric texture. A lookup table (Transfer Function) is also uploaded, which maps intensity values to colors and transparency levels.

---

## 6. Volume Rendering (Ray Casting)

For each screen pixel, a virtual ray is cast through the 3D volume. The ray samples intensity values at regular intervals, converting each sample to a color/opacity via the Transfer Function. Samples are blended front-to-back until the ray exits the volume.

---

## 7. Final Display

The accumulated color and opacity values are composited into the final image and displayed on screen.

---

## Key Concepts

| Concept | Description |
|---------|-------------|
| **Transfer Function** | A mapping from scalar intensity to visual appearance. Enables selective visualization of tissues (e.g., bone vs. soft tissue). |
| **Ray Casting** | A direct volume rendering technique that samples the 3D data along viewing rays to produce an image without intermediate geometry. |
| **Voxel** | A 3D pixel; the fundamental unit of volumetric data. |

---

## Why Ray Casting for Medical Imaging?

Unlike surface-based rendering (meshes), ray casting preserves the full volumetric density field. This allows clinicians to interactively adjust visualization parameters—highlighting different tissue types—without reprocessing the underlying data. It is the standard approach for CT, MRI, and PET visualization.
62 changes: 62 additions & 0 deletions docs/RENDERING_PIPELINE_TECHNICAL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# MRI-Viewer 3D Rendering Pipeline

This document outlines the step-by-step process from file upload to 3D volume rendering.

---

## 1. File Upload & Format Detection

User selects DICOM files via the `OpenFromDeviceComponent`. The `MRIReaderFactory` inspects file headers to determine the format (DICOM, NIfTI, HDR, KTX) and instantiates the appropriate loader.

---

## 2. DICOM Parsing

`LoaderDicom` reads each file's binary data using the Daikon library. Key DICOM tags are extracted: pixel data, image dimensions, slice position, spacing, and orientation vectors.

---

## 3. Series Grouping

Slices are grouped into series based on a hash of their orientation and spacing (`DicomSlicesVolume`). This handles multi-series datasets where different scans (e.g., different anatomical views) are stored together.

---

## 4. Volume Reconstruction

Slices within a series are sorted by their 3D position (calculated from `ImagePositionPatient` and `ImageOrientationPatient` tags). The sorted 2D slices are stacked to form a contiguous 3D voxel array (`Uint8Array` or `Float32Array`).

---

## 5. Texture Upload to GPU

The 3D voxel array is uploaded to the GPU as a WebGL **3D texture** (`THREE.Data3DTexture`). A separate 1D **Transfer Function texture** (256 RGBA values) maps voxel intensity to color and opacity.

---

## 6. Ray Marching (Volume Rendering)

The `volumerender.frag` shader performs **ray marching** through the 3D texture. For each pixel, a ray is cast from the camera into the volume. The ray samples the texture at regular intervals, accumulating color and opacity based on the Transfer Function until it exits the volume or reaches full opacity.

---

## 7. Compositing & Display

Front-to-back alpha compositing blends samples along each ray. The final RGBA value is written to the framebuffer and displayed on a full-screen quad via `THREE.WebGLRenderer`.

---

## Key Rendering Parameters

| Parameter | Description |
|-----------|-------------|
| **Transfer Function** | Maps scalar intensity → RGBA. Controls tissue visibility (e.g., bone=white, soft tissue=red). |
| **Step Size** | Distance between ray samples. Smaller = higher quality, slower render. |
| **ISO Value** | Threshold for isosurface rendering mode. |
| **Brightness / Contrast** | Post-process adjustments applied in the shader. |

---

## Domain Relevance

Volumetric ray casting is the standard technique for medical imaging because it preserves the full 3D density field (unlike mesh-based rendering). This allows clinicians to adjust the Transfer Function in real-time to highlight specific tissues (bone, blood vessels, tumors) without re-processing the data.
86 changes: 86 additions & 0 deletions rules/react-ui-best-practices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
description: Common React UI best practices (repo-aligned)
---

# Repo-aligned conventions

- Prefer `*.jsx` for UI components under `src/ui/` (this repo mostly uses JSX, not TSX).
- Keep explicit `import React from 'react'` / `import React, { ... } from 'react'` in `*.jsx` files for consistency with the existing codebase.
- Formatting is enforced by Prettier:
- Single quotes
- Semicolons
- 2-space indentation
- `printWidth` ~ 140
- Styling patterns used in this repo:
- CSS Modules: `import css from './Component.module.css'`
- `classnames` helper: `import cx from 'classnames'`
- MUI (`@mui/material`) and Emotion (`@emotion/styled`) in some components

# Component structure

- Keep components small and focused.
- Prefer function components with named exports, e.g. `export const MyComponent = (...) => { ... }` or `export function MyComponent() { ... }`.
- Keep render logic readable:
- Extract complex conditions into variables.
- Extract repeated UI into small subcomponents.

# State management

- Prefer local component state (`useState`) for strictly-local UI state.
- Use the existing Redux patterns (`useDispatch`, `useSelector`) for shared/app state.
- Avoid copying Redux state into local state unless you need a derived/transient UI value.
- For derived data:
- Prefer deriving directly in render when cheap.
- Use memoization (`useMemo`) for expensive derivations.

# Event handlers and re-renders

- Avoid inline heavy computations inside JSX.
- Use `useCallback` when:
- Passing handlers deep into the tree
- Passing handlers to memoized children
- Use `React.memo` for leaf components that re-render often and are expensive to render.
- Avoid unnecessary re-renders from object/array literals:
- Hoist constants out of the component when possible.
- Memoize objects passed as props when needed.

# MUI / Emotion usage

- Prefer MUI components for consistent UI controls when already used in the surrounding area.
- Keep styling approach consistent within a component:
- For small one-offs: MUI `sx` prop is acceptable.
- For reusable styled variants: prefer Emotion `styled(...)`.
- For layout/structure and app-specific visuals: prefer CSS Modules.
- Avoid overriding too many internal MUI classes unless necessary (maintenance cost).

# CSS Modules conventions

- Use `css.someClass` and combine with `cx(...)`.
- Keep class naming consistent within the file (the repo uses both simple names and BEM-like `block__element`).
- Prefer layout utilities (e.g. existing `Layout/*` components) rather than duplicating layout CSS.

# Accessibility

- Ensure interactive elements are semantic:
- Use `<button>` for actions, `<a>` for navigation.
- Add accessible labels:
- Provide `aria-label` for icon-only controls.
- Ensure tooltips are not the only source of meaning.
- Ensure focus states are visible for keyboard navigation.

# Error handling and user feedback

- Prefer user-visible error feedback for recoverable UI errors (existing modal patterns are used in this repo).
- Do not swallow errors silently; log only when it helps debugging.

# Testing

- Use Testing Library patterns already present in the repo.
- Prefer selecting by role/label/text; use `data-testid` when there is no good semantic selector.
- Keep `testId` props stable when exposed by shared UI components (e.g. `UIButton`).

# Performance

- Avoid expensive work on every render (especially in panels that re-render with sliders).
- Debounce/throttle high-frequency updates when needed (sliders, mouse move, drag).
- Prefer rendering only what’s visible/needed; avoid mounting heavy panels when not active.
Loading