Image preprocessing suggestions

[fsuk]

I was having problems reading barcodes with poor print quality, eventually I have worked out how to preprocess the image to make it more readable by Quagga2.

This would be good as a documented example or even as a feature.

My method uses OpenCV.js however I'm sure there would be a way to do this without it.
In my final solution I try to decode first without preprocessing and then with.
It does have a small performance impact, in my testing I found it took between 150-250ms which is still fast enough for my needs.

import { imread, Mat, cvtColor, GaussianBlur, adaptiveThreshold, COLOR_RGBA2GRAY, THRESH_BINARY, ADAPTIVE_THRESH_MEAN_C, imshow, Size } from '@techstark/opencv-js'

async preProcessImage (htmlImage:HTMLImageElement) {

    // Assumptions:
    // The image is 800x800 px
    // The barcode is roughly of the correct orientation 
    // The barcode is located within the central 50% of the image vertically

    // Load the image as an OpenCV.js Mat
    const mat = imread(htmlImage)

    // Crop the image vertically to only use the center of the image
    // Similar to the 'area' property in the quagga input stream
    const readArea = {x: 0, y: 0.25, width: 1, height: 0.5}

    const cropped = mat.roi({
      x: Math.floor(mat.cols * readArea.x),
      y: Math.floor(mat.rows * readArea.y),
      width: Math.floor(mat.cols * readArea.width),
      height: Math.floor(mat.rows * readArea.height)
    })

    // Tidy up the orignal
    mat.delete()

    // Convert the image to greyscale
    // Quagga2 does this already but we need to do it before the rest of the preprocessing.
    const gray = new Mat()
    cvtColor(cropped, gray, COLOR_RGBA2GRAY)
    cropped.delete()

    // Smooth the image to reduce noice
    // This uses a patch size which is taller than it is wide as we are assuming the barcode is more or less the correct orientation, we therefore want to smooth the bars of the barcode vertically.
    // Patch size may need to be adjusted if the original image is not 800x800
    const smoothed = new Mat()
    GaussianBlur(gray, smoothed, new Size(3, 9), 0)
    gray.delete()

    // Convert the image to binary black and white
    // Block size may need to be adjusted if the original image is not 800x800
    const binary = new Mat()
    adaptiveThreshold(smoothed, binary, 255, ADAPTIVE_THRESH_MEAN_C, THRESH_BINARY, 17, 2)
    smoothed.delete()

    // Tidy up and return the image as a data url.
    const filteredMat = binary
    const canvas = document.createElement('canvas')
    canvas.width = filteredMat.cols
    canvas.height = filteredMat.rows
    imshow(canvas, filteredMat)
    const filteredImage = canvas.toDataURL()
    filteredMat.delete()
    return filteredImage
  }

[ericblade]

I like it. I've wanted to add some pre-processing capabilities to the chain, it's part of the reason I've been (excruciatingly slowly) separating the internal chain that Quagga sends things through into separate bits so that we can add or remove additional processes from it and chain them together in whatever way is necessary for any specific use case.

[apcqa]

Would be good if it was written so that we could add middleware or plugins as required (essentially just functions which accept an image and options and return an image)

[ericblade]

yeah, I have a vision of getting to a point where you can chain an image source (camera, file, etc) to a pre-process chain, then to the reader/decoder, then to a post process validation step, in a way that is a lot less hacky than trying to do it with v1. There's some of that work in the v2 branch, but I've been a lot more absent from the project than I wanted to be, since jumping back into the workforce a few years ago, and other things outside of work have been sucking my time. I'm hoping to get back to it "soon", especially since the app that I use it in has bit rotted and requires significant maintenance to get back to working condition.