Guide · Spatial transcriptomics

GeoMx DSP data analysis: a practical guide from QC to differential expression

GeoMx gives deep RNA or protein profiles of the tissue regions you choose. Getting reliable results out of it comes down to a handful of quality-control and statistical decisions, made in the right order.

By Mark Zaidi, PhDUpdated November 20264 min read

The short answer

In one paragraph

Run segment QC, then probe QC, then filter genes and segments against the limit of quantification, then normalize (Q3 is the usual choice), and only then test for differences. For differential expression, use a linear mixed model with patient as a random effect, because several regions from the same patient are not independent. Skipping that last point is the most common reason GeoMx results fail to replicate.

How GeoMx data are structured

On a GeoMx slide you select regions of interest (ROIs) guided by immunofluorescence morphology markers. Each ROI can be split into areas of illumination (AOIs), or segments, using masks, for example tumour (PanCK+) and stroma (PanCK−). Each segment is profiled separately.

For RNA read out by sequencing, the GeoMx NGS Pipeline turns FASTQ files into one .dcc count file per segment. Together with the probe kit configuration (.pkc) and your annotation sheet, these form the starting point for analysis in R, typically with Bioconductor packages such as GeomxTools and standR.

The annotation sheet matters more than it looks. Patient, slide, region type, segment type, area and nuclei count all feed into QC and the statistical model, so it is worth getting them complete and consistent before anything else.

Step 1–3: quality control

Segment QC

Flag segments with too few raw or aligned reads, low sequencing saturation, very small area or few nuclei, or high no-template-control counts. Thresholds depend on tissue and segment type, so look at the distributions rather than applying defaults blindly. Small, low-nuclei segments are the usual casualties.

Probe QC

In whole-transcriptome assays, most genes are measured by a single probe, but negative-control probes come in sets. Remove probes that are global or local outliers, then collapse probes to gene-level counts.

Limit of quantification and filtering

Estimate a per-segment limit of quantification (LOQ) from the negative probes, commonly the geometric mean of the negatives multiplied by their geometric standard deviation squared. Then:

  • drop segments where only a small fraction of genes are above LOQ, and
  • keep genes detected above LOQ in a reasonable fraction of segments (often around 5–10%), always keeping genes of known biological interest.

Step 4: normalization

Q3 (upper-quartile) normalization scales each segment to the 75th percentile of its counts and is the standard choice for whole-transcriptome data. Before using it, plot each segment's Q3 against its negative-probe background: the two should separate clearly. If they don't, the segment is close to background and Q3 can mislead.

Background normalization (to the negative probes) is an alternative for low-signal data. For protein assays, normalization to housekeeping proteins, area or nuclei count is common, with IgG controls used to judge background. Whichever you use, check that the normalized data no longer separate by slide or batch in a PCA or UMAP.

Step 5: differential expression done properly

A typical GeoMx study has several ROIs per patient. Treating each ROI as an independent sample inflates the sample size and produces long lists of “significant” genes that don't replicate.

  • Between groups of patients (for example responders vs non-responders): use a linear mixed model with a random intercept for patient, or aggregate ROIs per patient first.
  • Within patients (for example tumour vs stroma segments from the same ROIs): include patient (and ROI, where segments are paired) as random effects so the comparison is made within each patient.
  • Across slides: if groups are confounded with slides, no model can fully separate biology from batch, which is why balanced slide layouts are worth planning before the run.

Correct for multiple testing, and report effect sizes alongside p-values. Volcano plots are useful, but a heatmap of the top genes split by patient often tells you more about whether a result is real.

Beyond differential expression

  • Cell type deconvolution. Methods such as SpatialDecon estimate the cell type mix within each segment from a single-cell reference. They are estimates, not counts, and work best when the reference matches the tissue.
  • Pathway analysis. Gene set enrichment on mixed-model results is usually more informative than individual genes.
  • Linking to imaging. Morphology marker intensities, ROI location and distance to landmarks (such as a tumour edge) can be added as covariates.

Study design tips

  • Spread each patient's ROIs across region types, and spread groups across slides.
  • Aim for enough nuclei per segment; very small AOIs are often lost at QC.
  • Record ROI selection criteria in advance, so region choice doesn't bias the comparison.
  • Plan the statistical model before choosing ROIs, not after the data arrive.

Frequently asked questions

Does GeoMx give single-cell data?

No. GeoMx profiles regions or compartments, each containing tens to hundreds of cells. Deconvolution can estimate the cell type mix, and single-cell questions are better suited to CosMx, Xenium or multiplexed imaging.

How many ROIs do I need per patient?

Enough to capture the region types you want to compare, often three to twelve per patient. More patients usually add more statistical power than more ROIs from the same patients.

Should I use Q3 or quantile normalization?

Q3 is the standard for whole-transcriptome GeoMx data and works well when segments are clearly above background. Check Q3 against negative-probe background before relying on it.

Can you reanalyze a GeoMx dataset that has already been processed?

Yes. A common request is to redo QC and replace ROI-level tests with mixed models, which often changes which genes hold up.

Planning a study or sitting on data?

Tell us about your samples, platform and question, and we'll suggest a practical next step.