An open source computer vision library in Mojo

Overview • Installation • Examples

Overview

Description

Mosaic is a cross-platform (macOS and linux) computer vision library for prototyping and production.

By transparently leveraging hardware acceleration and using the latest MLIR compiler technology, Mosaic is the first computer vision library built specifically for heterogenous compute: CPUs, GPUs, XPUs.

It unifies the computer vision workflow into one language that runs on any hardware.

Principles

1. Easy to use
2. Type-safe, memory-safe
3. Performant

Features

• Image representation
  • Image file encoding and decoding
  • Data type specification (at compile time)
  • Color space specification (at compile time)
• Image processing
  • Filtering
  • Geometric transforms
  • Fourier transforms
• Video capture (currently only macOS)
  • Stream connected cameras
• Video processing (currently only macOS)
  • Filter live video using custom processors
• Visualization (currently only macOS)
  • Native image and video rendering
• Hardware acceleration
  • Parallelization and SIMD on CPUs
  • GPU acceleration when available

Installation

Prerequisites

• Pixi (the Mojo environment and package manager)

Using the pixi cli

pixi add mosaic

After installation

The pixi.toml file should include the Modular community channel and the Mosaic dependency:

[project]
channels = ["https://repo.prefix.dev/modular-community"]

[dependencies]
mosaic = "*"

Examples

Prerequisites

Clone the repo and build Mosaic:

git clone [email protected]:christianbator/mosaic.git
cd mosaic
pixi run build

Running an example

Run an example file from the examples/ directory like so:

cd examples
pixi run mojo load_image.mojo

Load image

• Loads an image file and displays it in the visualizer
• Source: examples/load_image.mojo

Save image

• Saves an image to a file
• Source: examples/save_image.mojo

Scale image

• Scales an image to half size using bilinear interpolation
• Source: examples/scale_image.mojo

Resize image

• Resizes an image to double its width, stretching it in the process
• Source: examples/resize_image.mojo

Slice image

• Slices the top half of an image
• Image slices are mutable views into the underlying image data
• Source: examples/slice_image.mojo

Rotate image

• Rotates an image by 90° clockwise
• Source: examples/rotate_image.mojo

Flip image

• Flips an image vertically
• Source: examples/flip_image.mojo

Pad image

• Pads an image with zeros
• Source: examples/pad_image.mojo

Convert type

• Starting with a uint8 image, explicitly converts it to float64
• Source: examples/convert_type.mojo

Convert color space

• Converts an RGB image to greyscale
• Source: examples/convert_color_space.mojo

Convert color space as type

• Converts a float64 RGB image to uint8 greyscale in one method
• Source: examples/convert_as_type.mojo

Blur image

• Blurs an image using the built-in Gaussian kernel and reflected border handling
• Source: examples/blur_image.mojo

Detect edges

• Detects edges in an image by smoothing and applying a custom Laplacian kernel
• Source: examples/detect_edges.mojo

Unsharp mask

• Applies unsharp masking and image stacking to visualize the results
• Source: examples/unsharp_mask.mojo

Picture in picture

• Captures a sub-rect of an image and copies it to the top left corner
• Source: examples/picture_in_picture.mojo

Extract channel

• Extracts the green channel of an RGB image
• Source: examples/extract_channel.mojo

Fourier transform

• Calculates the spectrum of an image using the Fourier transform
• Source: examples/fourier_transform.mojo

Inverse Fourier transform

• Recreates an image from a previously calculated spectrum
• Source: examples/inverse_fourier_transform.mojo

High-pass filter

• Applies a high-pass filter by discarding low frequencies in the spectrum
• Source: examples/high_pass_filter.mojo

Video capture

• Opens the first available camera and visualizes the frames
• Source: examples/video_capture.mojo

Video processing

• Detects edges in the camera stream
• Source: examples/video_processing.mojo