Tomography Jobs: Picking and Subtomogram Averaging

This page documents every registered tomography picking, template-matching, and subtomogram-averaging job type in ccpem-pipeliner. Starting from raw tilt-series data imported with relion.importtomo, these jobs let you pick particles manually or by template matching, generate pseudo-subtomograms, classify them in 3D, refine a consensus structure, and polish particle motion — forming a complete, reproducible subtomogram-averaging pipeline. Understanding the exact process names, job option keys, continuability, and input/output nodes for each step is essential when scripting automated workflows or integrating pipeliner into facility pipelines.

Before working with tomography jobs you need:

• ccpem-pipeliner ≥ 1.6.1 installed in a Python ≥ 3.9 environment (see the installation guide)
• RELION built with tomography support — the jobs relion.importtomo, relion.pseudosubtomo, relion.reconstructparticletomo, relion.initialmodel.tomo, relion.class3d.tomo, relion.refine3d.tomo, relion.refine3d.tomo.helical, relion.ctfrefinetomo, and relion.framealigntomo all call RELION executables (relion_python_tomo_import, relion_tomo_subtomo, relion_tomo_reconstruct_particle, relion_refine, relion_tomo_refine_ctf, relion_tomo_align, etc.)
• RELION Python tomo tools on PATH — the Napari picker (relion.picktomo) requires relion_python_tomo_pick and relion_python_tomo_get_particle_poses
• LTM on PATH — the template-matching jobs (ltm.template.match, ltm.create.mask, ltm.create.template, ltm.extract.peaks, ltm.halftomo.localcc) require the ltm executable
• A project directory — change into an initialised pipeliner project before running any job; see the project-creation guide
• (Optional) A GPU for template matching (ltm.template.match) and for RELION GPU-accelerated steps
• (Optional) An MPI runtime (e.g. OpenMPI) for jobs that support MPI parallelism

Install ccpem-pipeliner with pip:

pip install ccpem-pipeliner

Verify that pipeliner and its console scripts are available:

pipeliner.check_setup

If any tomography-related executables (relion_python_tomo_import, relion_tomo_subtomo, ltm, etc.) are reported as '<command>' missing from system path, ensure that your RELION and LTM installations are on PATH before starting pipeliner. You can add them permanently by setting PIPELINER_ADDITIONAL_PROGRAM_PATHS or by listing a setup script in PIPELINER_PATH_TO_SOURCE_FILES; both take effect when pipeliner is imported.

Tomography jobs share the same settings infrastructure as all other pipeliner jobs. The settings file is ccpem-pipeliner-settings.json (located at ~/.config/ccpem/pipeliner/ on Linux or ~/Library/Application Support/ccpem/pipeliner/ on macOS). The most relevant settings for tomography workflows are:

Queue-related options (do_queue, queuename, qsub, qsubscript, min_dedicated) appear on the Run tab of every job that calls get_runtab_options(). Set queue_use to yes, true, y, or 1 (case-insensitive) to default new jobs to cluster submission.

All tomography jobs follow the same pipeliner pattern: generate a parameter file, optionally edit it, and run or schedule the job. Below is a summary of each registered job type in workflow order, followed by common usage patterns.

Job catalogue

relion.importtomo — Import raw tilt images

Class: RelionImportTiltSeriesMovies | Output dir: Import/
External program: relion_python_tomo_import
can_continue: False
is_tomo: True

Imports raw tilt-series movies (or motion-corrected images) described by SerialEM .mdoc files and produces a tilt_series.star node (TiltSeriesGroupMetadata).

Key job options:

Output nodes: tilt_series.star (TiltSeriesGroupMetadata)

relion.importtomo.coordinates — Import particle coordinates (tomography)

Class: RelionImportCoordinates | Output dir: Import/
can_continue: False
is_tomo: True

Imports particle coordinates into a tomography project. Key options include particle_star, use_wildcard, in_coords, remove_substring, is_center, scale_factor, and add_factor. Output: particles.star (ParticleGroupMetadata).

eulermate.import.coordinates — EulerMate subtomogram import

Class: EulermateJob | Output dir: Import/
External program: python3 + bundled eulermate.py script
can_continue: False
is_tomo: True

Reads an IMOD .mod model file and generates a RELION STAR file containing particle coordinates and Euler angles derived from dipole tangent/normal vectors.

Key job options:

Output nodes: imod2relion_particles.star (ParticleGroupMetadata)

relion.picktomo — Napari manual particle picking

Class: RelionTomoNapariPickJob | Output dir: Picks/
External programs: relion_python_tomo_pick, relion_python_tomo_get_particle_poses
can_continue: False (default)
is_tomo: not set (inherits False)

Note: Not recommended for remote sessions. Launches the Napari GUI for interactive picking.

Key job options:

Run-tab options: MPI (nr_mpi, mpi_command), additional args (other_args).

Output nodes: particles.star (ParticleGroupMetadata), optimisation_set.star (TomoOptimisationSet)

ltm.create.template — Create template for template matching

Class: LtmCreateTemplateJob | Output dir: TemplateMatching/
External program: ltm
can_continue: False
is_tomo: True

Prepares a CTF-modulated template map for use with ltm.template.match.

Key job options:

Output nodes: template.mrc (DensityMap)

ltm.create.mask — Create mask for template matching

Class: LtmCreateMaskJob | Output dir: TemplateMatching/
External program: ltm
can_continue: False
is_tomo: True

Creates a spherical or disc-shaped soft-edged mask for use with ltm.template.match.

Key job options:

Output nodes: ltm_template_mask.mrc (Mask3D)

ltm.template.match — Template matching with LTM

Class: LtmJob | Output dir: TemplateMatching/
External program: ltm
can_continue: False
is_tomo: True

Runs GPU-accelerated template matching across a tomogram.

Key job options:

Output nodes: best_score_map.mrc (DensityMap), best_angle_map.npy (DensityMap), std_cc_map.mrc (DensityMap), tm_config.yaml (ParamsData)

ltm.extract.peaks — Extract peaks from template matching results

can_continue: False
is_tomo: not set

Extracts particle coordinate peaks from template-matching score maps. Required options: fn_best_score_map, fn_best_angle_map, fn_std_cc_map, tomo_id, num_peaks_to_extract, cc_min. Output nodes include histogram.pdf and cc_vs_std_scatter.pdf.

ltm.halftomo.localcc — Local cross-correlation on half-tomograms

can_continue: False
is_tomo: not set

Computes local cross-correlation between two half-tomograms masked by a template mask. Required options: fn_half1_tomogram, fn_half2_tomogram, fn_template_mask_path.

relion.pseudosubtomo — Generate pseudo-subtomograms

Class: RelionPseudoSubtomoJob | Output dir: Extract/
External program: relion_tomo_subtomo
can_continue: True
is_tomo: False

Extracts windowed subtomogram volumes (2D stacks or 3D) from tilt-series data, ready for classification and refinement.

Input: provide either an optimisation_set.star (TomoOptimisationSet) via in_optimisation, or set do_use_direct_entries = True and supply in_tomograms (TomogramGroupMetadata) and in_particles (ParticleGroupMetadata) directly.

Key job options:

Run-tab options: MPI, threads, additional args.

Output nodes: particles.star (ParticleGroupMetadata), optimisation_set.star (TomoOptimisationSet)

relion.initialmodel.tomo — De-novo initial model (tomography)

Class: RelionTomoInimodelJob | Output dir: InitialModel/
External programs: relion_refine, relion_align_symmetry
can_continue: True
is_tomo: True

Generates a de-novo 3D initial model from pseudo-subtomograms using the VDAM algorithm.

Input: optimisation set or direct entries (use_direct_entries). On continuation, set fn_cont to the *_optimiser.star file.

Key job options:

Run-tab options: threads, additional args (no MPI — get_runtab_options(mpi=False)).

Output nodes: iteration-named run_itNNN_optimiser.star (OptimiserData) and class maps.

relion.class3d.tomo — 3D classification (tomography)

Class: TomoRelionClass3DJob | Output dir: Class3D/
External program: relion_refine
can_continue: True
is_tomo: False

Unsupervised 3D multi-reference classification of pseudo-subtomograms.

Input: optimisation set (in_optimisation) or direct entries (use_direct_entries with in_particles, in_tomograms, optional in_trajectories). On continuation, set fn_cont.

Key job options (in addition to standard Class3D options):

Run-tab options: MPI, threads, additional args.

Output nodes: Iteration-named class maps (DensityMap) and run_itNNN_optimiser.star (OptimiserData).

relion.refine3d.tomo — 3D auto-refinement (tomography)

Class: TomoRelionRefine3D | Output dir: Refine3D/
External program: relion_refine
can_continue: True
is_tomo: False

Gold-standard 3D auto-refinement of pseudo-subtomograms. Produces half-maps and an optimiser STAR file.

Input: optimisation set or direct entries (same pattern as relion.class3d.tomo). On continuation, set fn_cont (= in_optimiser in the class) via continue_job.star.

Key job options (beyond shared options):

Output nodes: run_half1_class001_unfil.mrc (DensityMap), run_half2_class001_unfil.mrc (DensityMap), optimiser STAR (OptimiserData).

relion.refine3d.tomo.helical — Helical 3D refinement (tomography)

Class: TomoRelionRefine3DHelical | Output dir: Refine3D/
External program: relion_refine
can_continue: True
is_tomo: False

Helical variant of relion.refine3d.tomo, adding helical symmetry options.

Key additional job options:

Important: Use helical_rise_initial and helical_twist_initial (not helical_rise / helical_twist) — passing unknown parameter names raises ValueError from job_from_dict().

relion.reconstructparticletomo — Reconstruct 3D particle

Class: RelionReconstructParticleJob | Output dir: Reconstruct/
External program: relion_tomo_reconstruct_particle
can_continue: False
is_tomo: False

Backprojects pseudo-subtomograms to produce a 3D average. Typically run after refinement to get final half-maps at a chosen binning and box size.

Input: optimisation set or direct entries (do_use_direct_entries).

Key job options:

Run-tab options: MPI (default nr_mpi = 1), threads, additional args.

Output nodes: merged.mrc (DensityMap), half1.mrc (DensityMap), half2.mrc (DensityMap), optimisation_set.star (TomoOptimisationSet)

Note: relion.reconstructparticletomo.helical has a class (RelionReconstructParticleJobHelical) but is not registered as an entry-point. Calling new_job_of_type('relion.reconstructparticletomo.helical') will fail.

relion.ctfrefinetomo — Refine CTF parameters (tomography)

Class: TomoRelionCtfRefine | Output dir: CtfRefine/
External program: relion_tomo_refine_ctf
can_continue: False
is_tomo: False

Optimises per-tilt defocus, contrast scale, and optionally beam tilt/higher-order aberrations.

Input: optimisation set or direct entries (use_direct_entries). Also requires a reference half-map (in_halfmaps; must contain half1 or half2 in the filename), a mask (in_refmask), and a postprocess STAR file (in_post).

Key job options:

Run-tab options: MPI (default nr_mpi = 1), threads, additional args.

Output nodes: particles_ctf_refine.star (ParticleGroupMetadata), optimisation_set.star (TomoOptimisationSet), logfile.pdf (LogFile)

relion.framealigntomo — Tomogram Bayesian polishing (frame alignment)

Class: TomoRelionBayesPolishJob | Output dir: Polish/
External programs: relion_tomo_align, relion_tomo_align_mpi
can_continue: True
is_tomo: False

Refines tilt-series projection matrices and optionally fits per-particle 3D motion trajectories (the tomography equivalent of Bayesian polishing).

Input: optimisation set or direct entries. Requires a reference half-map (in_halfmaps), mask (in_refmask), and postprocess STAR (in_post).

Key job options:

On continuation, --only_do_unfinished is appended to the command (both relion.framealigntomo variants inherit this from the base class).

Run-tab options: MPI, threads, additional args.

Output nodes: logfile.pdf (LogFile), optimisation_set.star (TomoOptimisationSet)

Common workflow patterns

Generate a default parameter file and run a job:

# Generate a run.job file with default values
pipeliner --default_runjob relion.pseudosubtomo
# Edit relion_pseudosubtomo_run.job as needed, then run:
pipeliner --run_job relion_pseudosubtomo_run.job

Continue a supported job:

# Edit the continuation parameters in the job directory
# then continue:
pipeliner --continue_job Reconstruct/job005/

Schedule multiple jobs and run them as a pipeline:

pipeliner --schedule_job relion_pseudosubtomo_run.job
pipeliner --schedule_job relion_initialmodel_tomo_run.job
pipeliner --schedule_job relion_class3d_tomo_run.job
pipeliner --run_schedule --name tomo_workflow --jobs Extract/job001/ InitialModel/job002/ Class3D/job003/

Example 1 — Run the full subtomogram averaging pipeline via the Python API

This example shows how to chain relion.importtomo → relion.pseudosubtomo → relion.initialmodel.tomo → relion.class3d.tomo → relion.reconstructparticletomo programmatically. Change into your project directory before running.

import os
os.chdir("/path/to/my_tomo_project")

from pipeliner.api.manage_project import PipelinerProject
from pipeliner.api.api_utils import job_default_parameters_dict

proj = PipelinerProject(make_new_project=True)

# --- Step 1: Import tilt-series movies ---
import_params = job_default_parameters_dict("relion.importtomo")
import_params["_rlnJobTypeLabel"] = "relion.importtomo"
import_params["fn_in_raw"] = "frames/*.tif"
import_params["fn_mdoc"] = "mdoc/*.mdoc"
import_params["angpix"] = "1.35"
import_params["kV"] = "300"
import_params["Cs"] = "2.7"
import_params["Q0"] = "0.1"
import_params["dose_rate_pertilt"] = "3.0"
import_params["tilt_angle"] = "85.3"
import_job = proj.run_job(import_params, run_in_foreground=True)
print(import_job.output_dir)  # e.g. Import/job001/

# --- Step 2: Generate pseudo-subtomograms ---
subtomo_params = job_default_parameters_dict("relion.pseudosubtomo")
subtomo_params["_rlnJobTypeLabel"] = "relion.pseudosubtomo"
subtomo_params["in_optimisation"] = import_job.output_dir + "optimisation_set.star"
subtomo_params["box_size"] = "64"
subtomo_params["binfactor"] = "4"
subtomo_params["do_output_2dstacks"] = "Yes"
subtomo_params["do_float16"] = "Yes"
subtomo_job = proj.run_job(subtomo_params, run_in_foreground=True)

# --- Step 3: Generate initial model ---
inimodel_params = job_default_parameters_dict("relion.initialmodel.tomo")
inimodel_params["_rlnJobTypeLabel"] = "relion.initialmodel.tomo"
inimodel_params["in_optimisation"] = subtomo_job.output_dir + "optimisation_set.star"
inimodel_params["nr_iter"] = "200"
inimodel_params["particle_diameter"] = "180"
inimodel_params["sym_name"] = "C1"
inimodel_job = proj.run_job(inimodel_params, run_in_foreground=True)

# --- Step 4: 3D classification ---
class3d_params = job_default_parameters_dict("relion.class3d.tomo")
class3d_params["_rlnJobTypeLabel"] = "relion.class3d.tomo"
class3d_params["in_optimisation"] = subtomo_job.output_dir + "optimisation_set.star"
class3d_params["fn_ref"] = inimodel_job.output_dir + "run_it200_class001.mrc"
class3d_params["nr_classes"] = "3"
class3d_params["nr_iter"] = "25"
class3d_params["particle_diameter"] = "180"
class3d_params["sym_name"] = "C1"
class3d_job = proj.run_job(class3d_params, run_in_foreground=True)

# --- Step 5: Reconstruct 3D particle ---
recon_params = job_default_parameters_dict("relion.reconstructparticletomo")
recon_params["_rlnJobTypeLabel"] = "relion.reconstructparticletomo"
recon_params["do_use_direct_entries"] = "Yes"
recon_params["in_tomograms"] = "Tomograms/job003/tomograms.star"
recon_params["in_particles"] = class3d_job.output_dir + "run_it025_data.star"
recon_params["binfactor"] = "2"
recon_params["box_size"] = "128"
recon_params["point_group"] = "C1"
recon_job = proj.run_job(recon_params, run_in_foreground=True)

Example 2 — Set job options via the .value property and schedule without running

import os
os.chdir("/path/to/my_tomo_project")

from pipeliner.api.manage_project import PipelinerProject
from pipeliner.job_factory import new_job_of_type

proj = PipelinerProject()

# Create and configure a CTF refinement job object
job = new_job_of_type("relion.ctfrefinetomo")
job.joboptions["in_optimisation"].value = "Refine3D/job010/optimisation_set.star"
job.joboptions["in_refmask"].value = "MaskCreate/job008/mask.mrc"
job.joboptions["in_post"].value = "PostProcess/job009/postprocess.star"
job.joboptions["box_size"].value = 256
job.joboptions["do_defocus"].value = True
job.joboptions["do_scale"].value = True

# Schedule without running immediately
scheduled_job = proj.schedule_job(job)
print(scheduled_job.output_dir)  # e.g. CtfRefine/job011/

Example 3 — Template matching pipeline via CLI

# Step 1: Create a CTF-weighted template
pipeliner --default_runjob ltm.create.template
# Edit ltm_create_template_run.job: set fn_tomogram, fn_ref, box_size_vox, kV, etc.
pipeliner --run_job ltm_create_template_run.job

# Step 2: Create a mask
pipeliner --default_runjob ltm.create.mask
# Edit ltm_create_mask_run.job: set fn_ref, center_x/y/z, radius
pipeliner --run_job ltm_create_mask_run.job

# Step 3: Run template matching
pipeliner --default_runjob ltm.template.match
# Edit ltm_template_match_run.job: set fn_tomogram, fn_template, fn_template_mask,
# particle_diameter, tilt_min, tilt_max, gpu_id
pipeliner --run_job ltm_template_match_run.job

# Step 4: Extract peaks from score maps
pipeliner --default_runjob ltm.extract.peaks
# Edit ltm_extract_peaks_run.job and run
pipeliner --run_job ltm_extract_peaks_run.job

Example 4 — Continue a tomography 3D refinement

After relion.refine3d.tomo completes, its continue_job.star is written in the job directory. Edit it (it is a STAR job file — use edit_jobstar) and then continue:

from pipeliner.api.manage_project import PipelinerProject
from pipeliner.api.api_utils import edit_jobstar

proj = PipelinerProject()

# Modify the continuation parameters in continue_job.star
edit_jobstar(
    fn_template="Refine3D/job012/continue_job.star",
    params_to_change={"fn_cont": "Refine3D/job012/run_it015_optimiser.star"},
    out_fn="Refine3D/job012/continue_job.star",
)

# Trigger the continuation
proj.continue_job("Refine3D/job012/")

Equivalently from the CLI:

pipeliner --continue_job Refine3D/job012/

Example 5 — Import IMOD coordinates with EulerMate

pipeliner --default_runjob eulermate.import.coordinates
# Edit eulermate_import_coordinates_run.job:
#   IMOD model: == my_picks.mod
#   Tomogram name: == TS_001
#   Binning factor to apply on coordinates: == 4.0
pipeliner --run_job eulermate_import_coordinates_run.job

relion_python_tomo_pick or relion_tomo_subtomo missing from system path

Symptom: pipeliner.check_setup reports 'relion_python_tomo_pick' missing from system path or a tomography job loads as unavailable.

Cause: Your RELION installation (or its Python tool wrappers) is not on PATH when pipeliner starts.

Fix: Add the RELION bin directory to PIPELINER_ADDITIONAL_PROGRAM_PATHS before importing pipeliner, or list a setup script in PIPELINER_PATH_TO_SOURCE_FILES so it is sourced at import time. Both take effect once when pipeliner is imported.

export PIPELINER_ADDITIONAL_PROGRAM_PATHS=/path/to/relion/bin

ltm not found — template-matching jobs unavailable

Symptom: ltm.template.match, ltm.create.template, or ltm.create.mask report unavailable.

Cause: The ltm executable is not on PATH.

Fix: Add the LTM installation directory to PIPELINER_ADDITIONAL_PROGRAM_PATHS.

relion.reconstructparticletomo.helical raises an error when creating a job

Symptom: new_job_of_type('relion.reconstructparticletomo.helical') raises ValueError: Cannot find job type relion.reconstructparticletomo.helical.

Cause: RelionReconstructParticleJobHelical exists in the codebase but is not registered as an entry-point. There is no relion.reconstructparticletomo.helical entry point.

Fix: Use relion.reconstructparticletomo for all particle reconstruction, including helical assemblies.

helical_rise / helical_twist raise ValueError from job_from_dict()

Symptom: Passing helical_rise or helical_twist as keys to run_job() or job_from_dict() for relion.refine3d.tomo.helical raises ValueError: Parameter helical_rise is not found in jobtype relion.refine3d.tomo.helical.

Cause: The correct option keys for relion.refine3d.tomo.helical are helical_rise_initial and helical_twist_initial, not helical_rise / helical_twist.

Fix: Use helical_rise_initial and helical_twist_initial in your parameters dict or run.job file.

Continuation fails with NotImplementedError

Symptom: Calling proj.continue_job('CtfRefine/job011/') or proj.continue_job('Reconstruct/job007/') raises NotImplementedError.

Cause: relion.ctfrefinetomo and relion.reconstructparticletomo have can_continue=False. Among the core tomo averaging jobs, only relion.pseudosubtomo, relion.framealigntomo, relion.initialmodel.tomo, relion.class3d.tomo, relion.refine3d.tomo, and relion.refine3d.tomo.helical support continuation.

Fix: For unsupported jobs, run a new job with proj.run_job(..., overwrite='JobType/jobNNN/') to replace the existing result, or leave the completed job and run a downstream step.

run_in_foreground=True raises ValueError when do_queue=Yes

Symptom: proj.run_job(params, run_in_foreground=True) raises ValueError: Cannot wait for completion when submitting jobs to a queue.

Cause: A job with do_queue set to Yes cannot be run in the foreground — the job runner cannot wait for queue completion.

Fix: Either set do_queue to No in the job parameters, or remove run_in_foreground=True and let the job be submitted to the queue asynchronously.

cryocare_path validation error at scheduling time

Symptom: Scheduling relion.denoisetomo raises a validation error about cryocare_path.

Cause: relion.denoisetomo loads normally (it does not mark itself unavailable when the directory is empty), but cryocare_path is is_required=True. Scheduling fails if it is blank.

Fix: Set PIPELINER_CRYOCARE_EXECUTABLES_DIR to the directory containing your cryoCARE executables, or set cryocare_executables_dir in the settings file.

Project locked after crash — .relion_lock present

Symptom: Running any pipeliner command fails because the project is locked.

Cause: A previous job left a .relion_lock directory in the project root.

Fix: Use pipeliner.recover_project to restore the pipeline and remove the lock:

pipeliner.recover_project --assume-yes

Verify that default_pipeline.star has been restored and .relion_lock has been removed.