Open-source cellular image analysis in 2026: a field map of the tools that matter

Microscopes got cheaper and faster. Imaging modalities multiplied. A single live-cell experiment now produces gigabytes per hour; an organoid screen produces terabytes. The bottleneck is no longer acquiring the data — it is understanding it.

The good news: the open-source ecosystem for cellular image analysis has matured fast. Foundation models, generalist segmenters, modern viewers, and pipeline glue all exist — and most are free. The hard part is picking what to start with.

This is a field map. What each tool is good for, where the boundaries are, and a quick “if you have X, start with Y” reference at the end.

The three jobs every tool is doing

Strip away the marketing pages and almost every cellular image analysis tool exists to do some subset of three things:

1. Segment — find each cell, nucleus, or feature in a frame and produce a mask.
2. Track — match the same object across frames (over time, or across z-slices in 3D).
3. Classify — label each detected object (cell type, cell state, phenotype) — often followed by quantitative measurements (size, intensity, neighbor stats).

Pick your tool from “which of these three jobs do I need, and at what scale?” — not from “what is the most powerful framework?” Most pipelines fail not because the model wasn’t strong enough, but because it solved the wrong job.

The landscape at a glance

Before the deep-dives below, here is the whole ecosystem on one page. We have organized it into the three categories you will actually encounter in practice — what is classical, what is deep-learning generalist, and what is pipeline glue — with a starred “start here” card in each row.

The classical stack — still useful, often optimal

These tools predate the deep-learning wave but remain workhorses. They are especially valuable when your data does not fit a generalist model, when you cannot get GPU access, or when reproducibility and auditability matter more than peak accuracy.

CellProfiler

A modular pipeline tool from the Broad Institute. You assemble image processing modules in a GUI; the resulting pipeline is reproducible, batch-friendly, and version-controllable. No ML required.

• Use it when you have many similar images and want a documented pipeline you can hand off to a new lab member.
• Avoid when your data is unusual enough that classical thresholding and feature extraction will not work.
• Pairs well with CellProfiler Analyst for downstream phenotype classification, and with Cellpose modules now built into recent CellProfiler releases.

Fiji / ImageJ

The most widely-used microscopy tool in the world. Probably installed on every microscopy core’s analysis workstation. The plugin ecosystem (TrackMate, MorphoLibJ, Bio-Formats, MIB) means almost any niche operation already has a Fiji macro written for it.

• Use it when you want quick interactive exploration, when you need to read a proprietary microscopy format (Bio-Formats reads ~150), or when you want a well-documented manual workflow.
• Avoid when you need a programmatic pipeline you can call from Python — Fiji macros work but are clunkier than Python notebooks.

ilastik

Interactive machine learning via random forests and shallow networks. You paint a few example pixels (foreground / background / object types); ilastik generalizes to the rest of the image.

• Use it when your problem is unusual — rare object types, tricky backgrounds, a domain a generalist model has not seen — and you only have a handful of examples.
• Avoid when you have plenty of data and a well-formed task; the deep-learning generalists below will beat ilastik on standard segmentation.

The deep-learning generalists — where most segmentation actually happens now

If you are segmenting cells or nuclei in a standard modality in 2026, this is where you should start.

Cellpose (and Cellpose-SAM)

The dominant generalist cell segmentation model. Trained on a broad mix of modalities, Cellpose works out-of-the-box on most fluorescence, brightfield, and phase-contrast data — and fine-tunes well on lab-specific data with surprisingly few annotations.

• Cellpose 3 added image restoration (denoising, deblurring) into the segmentation pipeline.
• Cellpose-SAM couples the generalist with Segment Anything’s foundation-model backbone for better robustness on unfamiliar modalities.
• Use it when you are segmenting cells of any roughly-convex shape in any standard modality.

StarDist

Fits star-convex polygons to objects — which sounds like a constraint until you realize that most nuclei are star-convex, and StarDist segments them faster and often more cleanly than general-purpose models. The 3D version handles confocal stacks.

• Use it when you are segmenting nuclei or similarly convex shapes; ideal for DAPI / Hoechst channels.
• Pairs well with Cellpose: StarDist for nuclei, Cellpose for cytoplasm.

Omnipose

Cellpose extended to filamentous shapes. If your “cells” are bacteria, neurons, fungal hyphae, or anything long-and-thin, the standard Cellpose model will struggle; Omnipose handles them natively.

Mesmer / DeepCell

Designed specifically for multiplexed tissue imaging — CODEX, MIBI, IMC, Vectra. Produces joint whole-cell and nuclear masks from a single forward pass.

• Use it when you are doing spatial biology and need cell boundaries aligned across many antibody channels.

Segment Anything and its bio variants

Meta’s Segment Anything brought general-purpose interactive segmentation to images. The cellular biology community has fine-tuned it into MicroSAM, CellSAM, and other variants that take the foundation-model backbone and adapt it to microscopy.

• Use it when classical generalists miss your modality and you want a “click to segment” interaction in a viewer like Napari.
• Watch this space: foundation models specialized for cellular data are the most active area of the field right now.

The pipeline / workbench layer

Models segment images. But a real workflow needs to load data, run inference, visualize results, fix mistakes, and export measurements. That is where these come in.

Napari

A modern Python-based image viewer and plugin platform. Most new tools ship a Napari plugin first. Lazy-loads large volumes, scrubs time-lapses smoothly, and has a plugin manager you do not have to bring a Java runtime to use.

• Use it as your visualization and interaction layer for any Python-based analysis.

BiaPy

A pipeline tool that wraps many of the segmenters above behind a uniform config. Good for “I want to run Cellpose, then track with TrackMate, then export — all from one declarative file.”

BioImage Model Zoo

A model marketplace. Models published here come with metadata that lets them run in Fiji, Napari, ilastik, and DL4MicEverywhere without manual conversion. The closest thing to “pip install” for image-analysis models.

ZeroCostDL4Mic / DL4MicEverywhere

Colab notebooks that train and run models for biologists without a Python environment. The best path for “I want to fine-tune StarDist on my data, but I do not maintain Python locally.”

Picking by problem

The hardest part of this field is knowing where to start. This quick-pick matrix maps the most common cellular-image-analysis problems to the tool we would reach for first.

This is a starting point, not a finishing line. Most production pipelines combine multiple tools — and the combinations are often more obvious than the individual choices. Here is what a common 2D-fluorescence time-lapse pipeline actually looks like:

The matrix tells you where to begin; let the data tell you what to combine.

What is changing in 2026

Four shifts are reshaping the field this year, and they all point in the same direction: less time wrangling tools, more time interpreting biology.

1. Foundation models specialized for cells

The first wave of deep learning gave us specialized models — one per modality, one per cell type. The current wave is generalist foundation models fine-tuned for cellular contexts: Cellpose-SAM, CellSAM, MicroSAM. They generalize across modalities better than the older specialists, at the cost of larger inference budgets. The trade-off is increasingly worth it as GPU compute gets cheaper.

2. AI agents that compose pipelines

Today, BiaPy lets you template a multi-step pipeline. By the end of 2026, agents will let you describe it: “segment cells, track them, classify mitotic events, export to CSV with per-cell metadata” — and the agent chains the right tools for you. We expect the friction of “stitching tools together” to fall sharply this year. The unit of human effort will move from glue code to which analysis to run.

3. Inference at the microscope

Edge GPUs in or near the imaging system mean segmentation can run as frames arrive, not in batch after the experiment ends. Use cases: adaptive imaging (zoom in on interesting cells in real time), live alerts during long runs (“rare phenotype detected at well B-04, t=312 min”), and triage of incoming data before storage.

4. Multi-modal alignment

Combining brightfield, fluorescence, and spatial transcriptomics — with cell IDs as the join key. Segmentation accuracy now propagates directly into the quality of downstream multi-omic readouts. The lab that gets segmentation right is the lab whose spatial-omics data is publishable without footnotes.

How this fits with what we are building

37degrees is not in the model-publishing business. The community is doing excellent work and we link to it. What we do build is the data infrastructure that makes this analysis usable in practice — the streaming layer that brings images and instrument telemetry into one queryable place.

• OMĒOS ingests streaming data from culture hardware and stores it at instrument-grade resolution, with the time-aligned context analysis pipelines need (CO₂, temperature, door events, perfusion state — see our earlier piece on real-time culture telemetry for why that context matters).
• We maintain an open-source cellular image analysis repository that catalogues the tools above with feature-level comparisons and links to active maintainers. It is updated continuously; if a tool is missing or a comparison is stale, let us know.
• The 37degrees technology platform is designed for the moment when on-device GPU inference becomes standard: the data layer is the same whether segmentation runs in the cloud, on a workstation, or at the microscope.

The pattern across the four shifts above — foundation models, agents, edge inference, multi-modal alignment — is that the value moves from running a model to running a system. Models will continue to commoditize. The lab with a stable, queryable, defensible data foundation will be the one that benefits.

Closing

The tools have caught up to the data. The bottleneck has moved upstream — to picking the right ones, training when you must, and stitching them into a pipeline you can defend in review.

This field map is the shortcut. If you maintain a tool that should be on this list, or you want to contribute to the open-source cellular image analysis repository we keep updated, get in touch.

---

Working on a cellular image analysis pipeline and want the data infrastructure to match? Explore OMĒOS, the data layer that turns instrument streams and analysis outputs into a single queryable record.