How to Accurately Detect Grid Cell Boundaries in Python Image Processing Using AI

Detecting the borders of a regular grid—whether it’s a spreadsheet, a game board, or a microscopy slide—can be surprisingly tricky. Traditional computer‑vision techniques often falter when the grid is noisy, warped, or partially occluded. In recent years, AI‑based methods have become the go‑to solution for robust, automated grid detection. This post walks through a practical, Python‑centric workflow that combines classic image‑processing steps with modern deep‑learning models to accurately locate grid cell boundaries.

Why Use AI for Grid Detection?

Adaptability: Convolutional neural networks (CNNs) learn to recognize grid patterns under varying lighting, perspective distortion, and background clutter.
Precision: Pixel‑level segmentation models (e.g., U‑Net) can output sub‑pixel accurate masks, which is essential for downstream tasks like digit extraction or cell counting.
Speed: Once trained, inference runs in milliseconds on a CPU/GPU, enabling real‑time processing of video streams.

Overall Pipeline

Pre‑processing: Denoise, enhance contrast, and optionally correct perspective.
AI‑based segmentation: Use a trained CNN to produce a binary mask of the grid lines.
Post‑processing: Refine the mask, extract line coordinates, and compute cell boundaries.
Validation: Visualize and optionally apply a sanity check based on expected grid dimensions.

Step‑By‑Step Implementation

1. Install Required Packages

pip install opencv-python numpy matplotlib torch torchvision
pip install segmentation-models-pytorch  # for ready‑made U‑Net models

2. Load and Pre‑process the Image

import cv2
import numpy as np
import matplotlib.pyplot as plt

def preprocess(img_path, target_size=(512, 512)):
    img = cv2.imread(img_path, cv2.IMREAD_COLOR)
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    # Resize while preserving aspect ratio
    img = cv2.resize(img, target_size, interpolation=cv2.INTER_LINEAR)

    # Denoise (optional)
    img = cv2.fastNlMeansDenoisingColored(img, None, 10, 10, 7, 21)

    # Contrast Limited Adaptive Histogram Equalization (CLAHE)
    lab = cv2.cvtColor(img, cv2.COLOR_RGB2LAB)
    l, a, b = cv2.split(lab)
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
    l = clahe.apply(l)
    lab = cv2.merge((l,a,b))
    img = cv2.cvtColor(lab, cv2.COLOR_LAB2RGB)

    # Normalize for the network
    img_norm = img.astype('float32') / 255.0
    return img_norm

3. Load a Pre‑trained Segmentation Model

We’ll use a U‑Net backbone (ResNet‑34) that has been fine‑tuned on a synthetic grid dataset. You can replace it with your own model.

import torch
import segmentation_models_pytorch as smp

DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model = smp.Unet(
    encoder_name='resnet34',
    encoder_weights='imagenet',
    classes=1,
    activation=None,
)
model.load_state_dict(torch.load('grid_unet_resnet34.pth', map_location=DEVICE))
model.to(DEVICE)
model.eval()

4. Perform Inference to Get the Grid Mask

def predict_mask(img_norm):
    with torch.no_grad():
        tensor = torch.from_numpy(img_norm).permute(2,0,1).unsqueeze(0).to(DEVICE)
        logits = model(tensor)
        prob = torch.sigmoid(logits)
        mask = (prob > 0.5).cpu().numpy().squeeze().astype(np.uint8)
    return mask

5. Post‑process the Binary Mask

We’ll clean up small artifacts, thin the lines, and extract the dominant horizontal & vertical lines using morphological operations.

def postprocess(mask):
    # Remove small noise
    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
    clean = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, iterations=2)

    # Skeletonize to 1‑pixel width lines
    size = np.size(clean)
    skel = np.zeros(clean.shape, np.uint8)
    element = cv2.getStructuringElement(cv2.MORPH_CROSS, (3,3))
    while True:
        eroded = cv2.erode(clean, element)
        temp = cv2.dilate(eroded, element)
        temp = cv2.subtract(clean, temp)
        skel = cv2.bitwise_or(skel, temp)
        clean = eroded.copy()
        if cv2.countNonZero(clean) == 0:
            break

    # Detect lines with Hough Transform
    lines = cv2.HoughLinesP(skel, 1, np.pi/180, threshold=100,
                           minLineLength=30, maxLineGap=5)
    return skel, lines

6. Compute Cell Boundaries

def extract_grid_cells(lines, img_shape):
    # Separate horizontal and vertical lines
    horiz = []
    vert = []
    for x1,y1,x2,y2 in lines[:,0]:
        if abs(y2-y1) < abs(x2-x1):  # horizontal
            horiz.append((y1, (x1,x2)))
        else:
            vert.append((x1, (y1,y2)))

    # Sort and deduplicate (tolerance = 5 px)
    def dedup(arr):
        arr.sort()
        out = []
        for val,_ in arr:
            if not out or abs(val - out[-1]) > 5:
                out.append(val)
        return out

    rows = dedup(horiz)
    cols = dedup(vert)

    # Build cell rectangles
    cells = []
    for i in range(len(rows)-1):
        for j in range(len(cols)-1):
            top, bottom = rows[i], rows[i+1]
            left, right = cols[j], cols[j+1]
            cells.append((left, top, right, bottom))
    return cells

7. Visualize the Result

def visualize(img, cells):
    fig, ax = plt.subplots(figsize=(8,8))
    ax.imshow(img)
    for (l,t,r,b) in cells:
        rect = plt.Rectangle((l,t), r-l, b-t,
                             edgecolor='lime', facecolor='none', lw=2)
        ax.add_patch(rect)
    ax.axis('off')
    plt.show()

8. Full Workflow Example

img_path = 'sample_grid.jpg'
img = preprocess(img_path)
mask = predict_mask(img)
skel, lines = postprocess(mask)
cells = extract_grid_cells(lines, img.shape[:2])
visualize(cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB), cells)

Tips for Better Accuracy

Data Augmentation: When fine‑tuning your own model, include rotations, perspective warps, and synthetic occlusions.
Custom Loss: Combine Dice loss with a line‑preserving term (e.g., Sobel loss) to encourage crisp edge learning.
Resolution Balance: Down‑sample to ~512 px for speed, but keep enough detail for thin lines; you can upscale the mask later.
Validate Geometry: If you know the expected number of rows/columns, discard outlier lines that don’t fit the pattern.

Conclusion

By leveraging an AI‑driven segmentation model alongside classic morphological processing, you can achieve reliable, high‑precision detection of grid cell boundaries in Python. This hybrid approach is flexible enough for a wide range of applications— from digitizing hand‑drawn tables to analyzing micro‑tissue arrays— while keeping the implementation simple and reproducible.

Feel free to adapt the code snippets for your own dataset, experiment with different backbones (e.g., EfficientNet, Swin‑Transformer), or integrate the pipeline into a larger OCR or measurement system.

Search This Blog

ChippyTech Blog