pyHiM in 3D

Tutorial for an analysis of sequential DNA-FISH data

This notebook describes basic functioning of pyHiM. It is divided into 6 sections corresponding to the different steps of sequential DNA-FISH data analysis.

• Section 0: Selection of dataset

• Section 1: Creation of Z-projections

• Section 2: Global 2D registration

• Section 3: Local 3D registration

• Section 4: Segmentation and localization

• Section 5: Building chromatin traces and pair-wise distance matrices

# Import python packages
from jupyter_functions_plots import *
import os

# Enable inline plots
%matplotlib inline

# Basic check of PyHiM installation
!pyhim --help

[VERSION] pyHiM 0.9.1
usage: pyHiM.py [-h] [-C CMD] [-F ROOTFOLDER] [-P PARAMETERS] [-T THREADS]

optional arguments:
  -h, --help            show this help message and exit
  -C CMD, --cmd CMD     Comma-separated list of routines to run. DEFAULT: proj
                        ect,register_global,register_local,mask_2d,localize_2d
                        ,mask_3d,localize_3d,filter_localizations,register_loc
                        alizations,build_traces,build_matrix
  -F ROOTFOLDER, --rootFolder ROOTFOLDER
                        Folder path with input images. DEFAULT: Current
                        directory
  -P PARAMETERS, --parameters PARAMETERS
                        Path of the parameters.json file. DEFAULT: pyHiM
                        expect this file inside your current directory
  -T THREADS, --threads THREADS
                        Thread number to run with parallel mode. DEFAULT: 1
                        (sequential mode)

0. Selection of dataset

To run on your computer, download this jupyter lab.

If you are not familiar with jupyter notebooks, you can find a tutorial here.

Download data

Download the raw data from OSF. These datasets were acquired using Qudi-HiM (Barho et al. 2022).

• Unzip the dataset in a folder of your choice.

• The folder containing the dataset has to be indicated in the next block under the variable input_folder.

# Indicate the directory that contain the dataset to use
input_folder =r'/path/to/your/pyHiM_test_dataset/'
os.chdir(input_folder)
print(f"The current working directory: {os.path.abspath(os.curdir)}")

Download parameters file

Download the parameters file and move it inside your input_folder.

Your folder structure should look like this:

.
├── notebooks
│   ├── full_pyHiM_run.ipynb
│   ├── jupyter_functions_plots.py
│   └── _static
└── test_dataset
    ├── localizations_3D_barcode_example.dat
  **├── parameters.json**
    ├── scan_001_DAPI_005_ROI_converted_decon_ch00.tif
    ├── scan_001_DAPI_005_ROI_converted_decon_ch01.tif
    ├── scan_001_RT26_005_ROI_converted_decon_ch00.tif
    ├── scan_001_RT26_005_ROI_converted_decon_ch01.tif
    ├── scan_001_RT27_005_ROI_converted_decon_ch00.tif
    └── scan_001_RT27_005_ROI_converted_decon_ch01.tif

Dataset description

Nuclei

• A DAPI image of nuclei (collected at 405 nm): scan_006_DAPI_001_ROI_converted_decon_ch00.tif

• The corresponding fiducial for the DAPI image (collected at 561 nm): scan_006_DAPI_001_ROI_converted_decon_ch01.tif

These 3D images are collected only once per Region Of Interest (ROI).

Barcodes

• Barcode image for cycle RTX (collected at 647nm): scan_001_RTX_001_ROI_converted_decon_ch01.tif

• The corresponding fiducial for cycle RTX (collected at 561nm): scan_001_RTX_001_ROI_converted_decon_ch00.tif

These 3D images are collected for each cycle of hybridization, for each ROI.

To speed up execution, the images in the dataset contain only a region of 512x512x60 at the center of the field of view of the original images. Only 2 barcode images and their corresponding fiducials are provided.

Localization table

• A localization table (localizations_3D_barcode_example.dat) generated using the full field of view and 25 barcodes.

1. Creation of Z-projections

The project module will project the 3D input images in 2D, using the method specified in the parameters.json file (default: maximum intensity projection). This step is required for the registration of XY mechanical shifts, and for 2D segmentations.

%%capture
#To hide the terminal outputs, uncomment if you want to display them

# Execute project
!pyhim -C project

DAPI projections

# Set datatype ('DAPI','RT')
datatype = 'DAPI'

# Set titles
titles = ['DAPI fiducial','DAPI masks']

# Display DAPI and fiducial signal
plot_zprojection(input_folder,_,titles,datatype)

Barcode projections

# Set datatype ('DAPI','RT')
datatype = 'RT'

# Set reference RTs
rt_references = 'RT26'

# Set titles
titles = [rt_references+' fiducial signal',rt_references+' signal']

# Display RTs and fiducial signal
plot_zprojection(input_folder,rt_references,titles,datatype)

2. Registration

This section will use fiducial images to register images and correct the mechanical drift of the microscope. For this, a reference barcode fiducial needs to be provided in the parameters.json file. The register_global method will find the shifts in XY that best registers the fiducial images and will apply these shifts to the DAPI and barcode images. Different methods for global alignment are available, please see the register_global user documentation for more information.

%%capture
# Indicate the directory that contains the test dataset
os.chdir(input_folder)

# Execute the alignment
! pyhim -C register_global

Displaying results

# Set figure titles
cycle = 'RT27'
titles = [cycle+' & DAPI '+'Uncorrected - zoom',cycle+' & DAPI '+'Corrected - zoom',cycle+' & DAPI '+'Overlay- zoom']

# Display alignement results
plot_alignment(input_folder,cycle,titles)

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

The left panel displays the difference between the fiducial image of the reference barcode (RT26, red) and the fiducial of the DAPI image (blue). The middle panel displays the same image registered for drift. In this middle panel, red and blue spots should overlap if images were correctly registered. The right panel shows the two registered images in RGB. Regions of perfect co-localization appear in yellow.

3. Local 3D registration

This step aims to improve global 2D registration from the previous module by taking into account local deformations of the sample. In addition, this function will check and correct residual Z-drifts that may have occurred despite the auto-focus control system. Different methods and options are available, please see the register_local user documentation for more information.

%%capture

! pyhim -C register_local

Displaying results

# Display block alignments in 3D: XY (left panel), XZ-YZ (right panels)
plot_matrix(input_folder,data_type='3D_alignments,RT27')

4. Mask segmentation and barcode localization

This section will process the segmentation of nuclei images using AI-based methods. For more details, see the mask_3d user guide.

%%capture

! pyhim -C mask_3d

Displaying results

# Set datatype ('DAPI','RT','TRACES')
datatype = 'DAPI'

# Set titles
titles = ['DAPI raw','DAPI segmented']

# Display segmented DAPI masks
plot_segment_object(input_folder,titles,datatype)

$ Will plot: ['/home/xdevos/Documents/tempo/test_dataset/mask_3d/scan_001_DAPI_005_ROI_converted_decon_ch00.tif_3Dmasks.png']

Segmentation and 3D localization of barcodes

This section will segment barcodes using AI-based methods and then localize the center of each barcode using 3D gaussian fitting. For more details, see the localize_3d user guide.

%%capture

! pyhim -C localize_3d

Displaying results

# Set datatype ('DAPI','RT','TRACES')
datatype = 'RT'
rt_references = 'RT26'

# Set titles
titles = [rt_references+' Segmented ']

# Display segmented barcode masks
plot_segment_object(input_folder,titles,datatype,rt_references)

$ Will plot: ['/home/xdevos/Documents/tempo/test_dataset/localize_3d/scan_001_RT26_005_ROI_converted_decon_ch01.tif_3DimageNlocalizations.png']

5. Building chromatin traces and pair-wise distance matrices

This section will build chromatin trace tables from barcode localization tables, and calculate pairwise distance (PWD) matrices for each trace.

From section 1 to section 4, we have provided a very small dataset with only two barcodes and a reduced field of view (512x512x60). In this section, we will provide you with a localization matrix containing data from 25 barcodes over a full field of view (2048x2048x60). This localization table will be used to build chromatin traces and from these construct pair-wise distance and proximity maps.

Building chromatin traces

# copies localization table
copy_localization_table(input_folder)

# change folder for another analysis
input_folder = os.path.join(input_folder,"analysis_localizations")
os.chdir(input_folder)
print(f"The current working directory: {os.path.abspath(os.curdir)}")

%%capture

# builds traces
! pyhim -C build_traces

Displaying chromatin traces

# Set datatype ('DAPI','RT','TRACES')
datatype = 'TRACES'

# Set titles
titles = ['Traces','Traces zoomed']

# Display traces
plot_segment_object(input_folder,titles,datatype)

$ Will plot: ['/home/xdevos/Documents/tempo/test_dataset/analysis_localizations/tracing/Trace_3D_barcode_KDtree_ROI-5_traces_XYZ_ROI5.png']

Building pairwise distance and proximity maps

%%capture

# builds traces
! pyhim -C build_matrix

Displaying pairwise distance and proximity maps

# Display the HiM proximity matrix
plot_matrix(input_folder,data_type='proximity')

# Display the PWD matrix
plot_matrix(input_folder,data_type='PWD')