Skip to content

Segmentation

Jump to bottom
Estibaliz Gómez-de-Mariscal edited this page Nov 11, 2025 · 31 revisions

Segmentation using DL approaches

Segmentation is an important task in most image analysis pipelines. Typically, segmentation extracts single-cell instances from images containing many bacterial cells. Ideally, every instance is a single cell, and the outlines are then used to measure parameters such as cell size, shape or fluorescence intensity. Segmentation can be achieved either manually (very tedious work), using algorithms based on thresholding and cell separation, classical machine learning (i.e. training a classifier in ilastik) or deep learning approaches.

The image above illustrates segmentation using a U-Net based neural network. Paired training image data (top) is fed into the U-Net, which learns to transform the input into the segmented output. The network can then be used to segment new input data (bottom).

As microbial datasets are very diverse, they often require a separate image analysis pipeline, classifier or model for a specific condition (strain, growth condition, microscopy technique).

In DeepBacs, we provide the following datasets:

Organism Image type Link to datasets and pretrained models Associated publication
S. aureus Brightfield and Fluorescence here (data) unpublished
E. coli Brightfield here (data) unpublished
B. subtilis Fluorescence here (data) here
B. subtilis Fluorescence - Multilabel dataset here (data and model) here
Mixed dataset Fluorescence and brightfield here (data and model) unpublished and here

Important disclaimer

For segmentation, we made use of various DL networks. Here's a list of the original articles/repositories.

U-Net:

U-Net original code and documentation is freely available in github


StarDist:

StarDist original code and documentation are freely available in GitHub


SplineDist:

SplineDist original code is freely available in GitHub


CARE:

CARE original code and documentation is freely available in GitHub


pix2pix:

The source code of the PyTorch implementation of pix2pix is freely available in GitHib

Please also cite these original papers when using U-Net, StarDist, SplineDist, CARE or pix2pix with our notebook.

Preparation of data for segmentation task

The networks we tested on our example datasets use supervised learning. Thus, paired images are required to train the networks. Input image format ground-truth data varies depending on the network used.

  • U-Net/CARE: TIFF format; binary segmentation masks as ground truth
  • StarDist/SplineDist: TIFF format; instance-labeled mask images as ground truth
  • pix2pix: PNG format; For ground truth, binary mask images worked best for us

Instance-labeled mask images can be created using Fiji. This requires the segmented single-cell instances being present as ROIs in the ROI Manager. When e.g. annotating images manually using the "Custom ROI" tool, the selection can be added to the ROI Manager pressing "CNTR + t" or by pressing the "Add" button in the ROI Manager. When all ROIs are drawn, a ROI map can be created using the LOCI plugin (it can be downloaded via its Fiji download site). For this, call the "ROI Map" function via the command line, which creates an image with a unique gray value for each ROI. This is required by the networks to detect the single cell instances and map it to the input image.

Preparation of data for semantic segmentation of boundaries and cell bodies

Once the images have been annotated for instance or binary segmentation, the semantic segmentation labels can be computed using this ImageJ macro which calls MorpholibJ plugin:

  • Open Fiji and install MorpholibJ from ImageJ update sites. Detailed instructions here.
  • Restart Fiji.
  • Download the ImageJ macro from this GitHub repository.
  • Open the macro in Fiji (File > Open...).
  • Change the PATH to the directory containing the folder with the segmentations, e.g. PATH = "/Users/myuser/Documents/DeepBact/EColi/train/";.
  • Change the input_folder with the name of the folder that contains the segmentations, e.g. input_folder = "target/";.
  • Click on Run > Run (or Ctrl + r).

The macro will create a new folder with the name specified in new_folder_seg. The pixels in the new images will have value 0, 1 and 2 for the background, cell body and cell boundary respectively (see example below).


Datasets

S. aureus dataset

The S. aureus data provided here consists of two datasets and was provided by Pedro Pereira and Mariana Pinho.

(1) Paired DIC images and segmentation mask
(2) Paired fluorescence membrane images and segmentation mask

Data acquisition and preparation:
For S. aureus time-lapse experiments overnight cultures of S. aureus strain JE2 were back-diluted 1:500 in TSB and grown to mid-exponential phase (OD600nm ≈ 0.5). One milliliter of the culture was incubated for 5 min (at 37°C) with the membrane dye Nile Red (5 µg ml−1, Invitrogen), washed once with phosphate buffered saline (PBS), subsequently pelleted and resuspended in 20 µl PBS. One microliter of the labelled culture was then placed on a microscope slide covered with a thin layer of agarose (1.2% (w/v) in 1:1 PBS/TSB solution). Imaging was performed in a GE HealthCare Deltavision OMX system (with temperature and humidity control, 37°C). The images were acquired using 2 PCO Edge 5.5 sCMOS cameras (one for DIC, one for fluorescence), an Olympus 60x 1.42NA Oil immersion objective (oil refractive index 1.522), Cy3 fluorescence filter sets (for the 561 nm laser) and DIC optics.

To generate the training dataset for StarDist segmentation, individual channels were separated and preprocessed using Fiji. Nile Red fluorescence images were manually annotated using ellipsoid selections to approximate the S. aureus cell shape. Resulting ROIs were used to generate the required ROI map images (using the “ROI map” command included in the Fiji plugin LOCI) in which each individual cell is represented by an area with a unique integer value. Training images (512 x 512 px²) were further split into 256 x 256 px² images, resulting in 28 training images pairs. 5 full field-of-view test image pairs were provided for model quality control. For segmentation of S. aureus bright field images (corresponding to the Nile Red fluorescence images) were paired with the ROI masks created for fluorescence image segmentation.


Here are examples for the matched brightlight and corresponding mask images:


And here for the matched Nile Red fluorescence and corresponding mask images:


Network training parameters

Networks for S. aureus segmentation were trained using the following parameters:

Mode Network Images (train/test) Epochs Steps Image size Patch size Batch size LR % Valid. Augmentation Misc. Train time
Brightfield StarDist 28/5 400 22 256 x 256 px² 256 x 256 px² 4 0.0003 20 4x (flip/rot.) grid 1, 32 rays 24 min 53 sec
Fluorescence StarDist 28/5 400 24 256 x 256 px² 256 x 256 px² 4 0.0003 20 4x (flip/rot.) grid 1, 32 rays 53 min 27 sec

E. coli dataset

The E. coli dataset consists of paired brightfield images and segmentation masks.

Data acquisition and preparation: E. coli MG1655 cultures were grown in LB Miller at 37°C and 220 rpm overnight. Working cultures were inoculated 1:200 and grown at 23°C and 220 rpm to OD600 ~ 0.5 – 0.8. For time lapse imaging, cells were immobilized under agarose pads prepared using microarray slides (VWR, catalogue number 732-4826) as described in de Jong et al., 2011. Brightfield time series (1 frame/min, 80 min total length) of 10 regions of interest were recorded with an Andor iXon Ultra 897 EMCCD camera (Oxford instruments) attached to a Nikon Eclipse Ti inverted microscope (Nikon Instruments) bearing a motorized XY-stage (Märzhäuser) and an APO TIRF 1.49NA 100x oil objective (Nikon Instruments). To generate the segmentation training data, 19 individual frames from different regions of interest were rescaled using Fiji (2x scaling without interpolation) to allow for betterer annotation. Resulting images were annotated manually using the freehand selection ROI tool in Fiji. For quality control, a test dataset of 15 frames was generated similarly. Contrast was enhanced in Fiji and images were either converted into 8-bit TIFF (CARE, U-Net, StarDist) or PNG format (pix2pix).




Network training parameters

Networks for E. coli brightfield segmentation were trained using the following parameters:

Mode Network Images (train/test) Epochs Steps Image size Patch size Batch size LR % Valid. Augmentation Misc. Train time
Brightfield U-Net 19/15 100 30 1024 x 1024 px² 512 x512 px² 2 0.0003 10 Flip/Shear 23 min
Brightfield Multilabel U-Net 19/15 500 17 1024 x 1024 px² 512 x512 px² 4 0.0003 10 4x (all options) 2 pooling steps 1 h 8 min
Brightfield CARE 19/15 100 300 1024 x 1024 px² 128 x 128 px² 8 0.0004 20 4x (flip/rot.) 42 min 16 sec
Brightfield StarDist 19/15 200 30 1024 x 1024 px² 512 x512 px² 2 0.0003 20 4x (flip/rot.) grid 1, 80 rays 45 min 23 sec
Brightfield pix2pix 19/15 500 n.a. 1024 x 1024 px² 256 x 256 px² 1 0.0002 n.a. 4x (flip/rot.) 47 min 57 sec

B. subtilis dataset

The B. subtilis dataset consists of paired brightfield images and segmentation masks and was provided by Mia Conduit and Séamus Holden.

B. subtilis cells expressing FtsZ-GFP were prepared as described in Whitley et al., 2021 (strain SH130, PY79 Δhag ftsZ::ftsZ-gfp-cam). Strains were taken from glycerol stocks kept at -80°C and streaked onto nutrient agar (NA) plates containing 5µg/ml chloramphenicol then grown overnight at 37°C. Liquid cultures were started by inoculating time-lapse medium (TLM) (de Jong et al., 2011) with a single colony and growing overnight at 30°C with 200 rpm agitation. The following morning, cultures were diluted into chemically defined medium (CDM) containing 5µg/ml chloramphenicol to OD600 0.1, and grown at 30°C until the required optical density was achieved. All imaging was done on a custom-built, 100X inverted microscope. A 100x TIRF objective (Nikon CFI Apochromat TIRF 100XC Oil), a 200mm tube lens (Thorlabs TTL200) and Prime BSI sCMOS camera (Teledyne Photometrics) were used achieving an imaging pixel size of 65nm/pixel. Cells were illuminated with a 488 nm laser (Obis) and imaged using a custom ring-TIRF module operated in ring-HiLO (Ellefsen et al., 2015). A pair of galvanometer mirrors (Thorlabs) spinning at 200Hz provides uniform, high SNR illumination (Whitley et al., 2021).

Slides were prepared as described previously. Molten 2% agarose made with CDM was poured into gene frames to form flat agarose pads, then cut down to thin 5mm strips. 0.5 µl of cell culture grown to mid-exponential phase (OD600 0.2-0.3) was spotted onto the agarose and allowed to absorb (approx. 30 seconds). A plasma-cleaned coverslip was then placed atop the gene frame and sealed in place. Before imaging, the prepared slides were then pre-warmed inside the microscope body at least 15 minutes before imaging (Whitley et al., 2021). Time-lapse images were then taken in TIRF using a custom built 100x inverted microscope. Images were taken at 1 second exposure, 1 frame/minute at 1-8 W/cm2 (Whitley et al., 2021). Videos were denoised using ImageJ plugin PureDenoise (Luisier et al., 2009, group page link) then lateral drift was corrected using StackReg (Thévenaz et al., 1998). To create the training dataset, 10 frames were extracted from each time-lapse approximately 10 frames apart. This was to ensure sufficient difference between the images used for training. Ground truth segmentation maps were generated by manual annotation of cells in each frame using the Fiji/ImageJ LabKit plugin lab from Matthias Arzt. This process assigns a distinct integer to all pixels within a cell region, and background pixels are labelled 0. A total of 4,672 cells were labelled across 80 distinct frames to create the final training dataset.



From semantic segmentation to cell instances: post-processing of the multilabel U-Net outputs.

Cell instances can be computed from the semantic segmentations using this this ImageJ macro, which calls MorpholibJ plugin:

  • Open Fiji and install MorpholibJ from ImageJ update sites. Detailed instructions here.
  • Restart Fiji.
  • Download the ImageJ macro from this GitHub repository.
  • Open the macro in Fiji (File > Open...).
  • Change the PATH to the directory containing the folder with the segmentations, e.g. PATH = "/Users/myuser/Documents/DeepBact/EColi/results/";.
  • Change the input_folder with the name of the folder that contains the segmentations, e.g. input_folder = "predictions/";.
  • Click on Run > Run (or Ctrl + r).

The macro will create a new folder with the name specified in new_folder_seg. Each cell in the image will be labelled with a unique pixel value.

Network training parameters

Networks for B. subtilis segmentation were trained using the following parameters:

Mode Network Images (train/test) Epochs Steps Image size Patch size Batch size LR % Valid. Augmentation Misc. Train time
Fluorescence U-Net 80/10 100 29 1024 x 1024 px² 512 x 512 px² 10 0.001 10 4x (flip/rot.) 47 min
Fluorescence Multi-label U-Net 80/10 200 83 1024 x 1024 px² 256 x 256 px² 8 0.0005 10 4x (all options) 3 pooling steps 1 h 10 min
Fluorescence SplineDist 32/10 800 25 512 x 512 px² 256 x 256 px² 4 0.0003 20 4x (flip/rot.) grid 2 5 h 11 min
Fluorescence CARE 80/10 50 2000 1024 x 1024 px² 64 x 64 px² 16 0.001 20 4x (flip/rot.) 1h 7 min
Fluorescence StarDist 80/10 400 80 1024 x 1024 px² 512 x 512 px² 4 0.0003 10 4x (flip/rot.) grid 2, 64 rays 4 h 20 min
Fluorescence pix2pix 80/10 1000 n.a. 1024 x 1024 px² 256 x 256 px² 1 0.0002 n.a. 4x (flip/rot.) 5h 50 min

Mixed dataset (S. aureus, E. coli and B. subtilis)

In addition to the separate datasets, we also trained a StarDist model on a pooled dataset. To train the mixed model, we used the following parameters:

Mode Network Images (train/test) Epochs Steps Image size Patch size Batch size LR % Valid. Augmentation Misc. Train time
mixed model StarDist 155/35 200 120 varying 256 x 256 px² 4 0.0003 10 3x (flip/rot.) grid 2 44 min 34 sec

This model can also be downloaded from Zenodo (here) and used either in the ZeroCostDL4Mic platform or in the StarDist Fiji plugin.


Links to the notebooks

The ZeroCostDL4Mic notebooks used for segmentation in DeepBacs can be accessed here

Network Paper(s) Direct link to notebook in Colab
U-Net (2D) here and here Open In Colab
U-Net (2D) multilabel here and here Open In Colab
StarDist (2D) here and here Open In Colab
CARE (2D) here Open In Colab
pix2pix here Open In Colab
SplineDist here Open In Colab

Note that CARE and pix2pix were not designed for the segmentation task. However, network outputs can be segmented using further image postprocessing (thresholding, particle detection, etc.).

For further documentation, visit the ZeroCostDL4Mic wiki page.

Home

Overview

Clone this wiki locally