Difference between revisions of "Integration with Warp and M"

From Dynamo
Jump to navigation Jump to search
Line 72: Line 72:
  
 
= run alignments/classifications in dynamo or relion =
 
= run alignments/classifications in dynamo or relion =
 +
When warp exports subtomograms it provides a STAR file for the output. This can be used to run refinements in RELION.
 +
 +
If we want to import particles back into Dynamo, we run the ''warp2dynamo'' script from [[https://github.com/alisterburt/dynamo2m]] on this STAR file.
 +
 +
This script creates a table file and corresponding tomogram-table map file, this table file should be used for refinements with the Warp particles.
 +
 +
Due to some normalisation differences, it is better to create a dynamo formatted data folder from these particles. Two extra files are provided for this, with the suffix ''_reextract'' in their names.
 +
 +
A valid data folder can be cropped from these particles using the command
 +
 +
<nowiki>dtcrop <tomogram_table_map_reextract.doc> <table_reextract.tbl>  <outputfolder> <sidelength> -asBoxes 1</nowiki>
 +
 +
You then have a valid data folder and table file with which you can start refinements. If you want to perform subboxing or reextract particles for whatever reason, simply use the ''dynamo2warp'' script on the correct table and tomogram-table map file at any time and reextract in Warp.
  
 
= run multi-particle refinements in M =
 
= run multi-particle refinements in M =

Revision as of 11:36, 13 August 2020

This page describes a method for integration of Dynamo, Warp and M to be able to take advantage of...

Warp's

  • streamlined preprocessing
  • robust CTF estimation procedures for tilt-series data

Dynamo's

  • automated tilt-series alignment
  • geometric picking tools
  • flexible subtomogram averaging procedures

M's

  • multi-particle refinement
  • half-map regularisation by denoising


Overview

  • Preprocess from multi-frame micrographs to tilt-series using Warp
  • Align tilt-series using [Dynamo]
  • Generate downscaled tomograms in warp
  • Pick particles in Dynamo and run first alignments
  • Reextract unbinned particles in warp using [dynamo2m]
  • Run alignments/classifications in dynamo or relion
  • Run multi-particle refinements in M

The last steps can be repeated to further improve final reconstruction quality

preprocess

Preprocess your multi-frame micrographs in warp following the guide [here]

Bad images should be deselected in Warp at this stage.

This stage is finished once you have your tilt-series in the imod folder

If you collected your data in Tomo rather than SerialEM and don't have mdoc files, you can try [mdocspoofer]

align tilt-series

Tilt series can either be aligned in IMOD or using dynamo's new tilt series alignment tools

A function to automate the alignment of datasets with fiducials and convert all necessary metadata for import into warp is provided as [| autoalign_dynamo].

Once you've followed the setup from that page, simply navigate to the imod directory generated by warp and run

dautoalign4warp(<pixel_size_angstrom>, <fiducial_diameter_nm>, <nominal_rotation_angle>, <output_folder>)

The output folder is a folder which will contain all of your alignments for this dataset.

Briefly, tilt-series are aligned in Dynamo with the automated procedure. The fiducial positions are then converted into an IMOD model file and tiltalign is run on this to calculate the necessary transforms to apply to the tilt series in the format Warp wants.

generate downscaled tomograms

Tomograms are reconstructed at a larger pixel size in Warp for picking using the [tomography task dialogue] in Warp.

Choose to also generate a deconvolved version, we will use this for visualisation.

Generate a catalogue using the function warp2catalogue()provided in [autoalign_dynamo]

warp2catalogue(<warp_reconstruction_folder>, <pixel_size_angstrom>)

pick particles in dynamo

Pick particles using your favourite type of model through the catalogue created in the previous step.

Make sure to crop particles from the catalogue directly, this will make sure that even though you picked particles on filtered volumes for visualisation the particles are cropped from the unfiltered tomograms.

You can choose to run first sets of alignments on these downscaled particles to find which particles you would like to use for later refinements.

reextract unbinned particles in warp

Once you have a set of positions and orientations you want to use in refinements at a smaller pixel size, we need to convert this metadata into something Warp can understand so it can reconstruct our particles for us at the desired voxel size.

A script dynamo2warp is provided in the [dynamo2m] package. It requires a table and a [ Tomogram-table map file | tomogram-table map file] and produces a STAR file with all the necessary information.

Particles are then extracted using the [export subtomograms] task dialogue in Warp.

run alignments/classifications in dynamo or relion

When warp exports subtomograms it provides a STAR file for the output. This can be used to run refinements in RELION.

If we want to import particles back into Dynamo, we run the warp2dynamo script from [[1]] on this STAR file.

This script creates a table file and corresponding tomogram-table map file, this table file should be used for refinements with the Warp particles.

Due to some normalisation differences, it is better to create a dynamo formatted data folder from these particles. Two extra files are provided for this, with the suffix _reextract in their names.

A valid data folder can be cropped from these particles using the command

dtcrop <tomogram_table_map_reextract.doc> <table_reextract.tbl>  <outputfolder> <sidelength> -asBoxes 1

You then have a valid data folder and table file with which you can start refinements. If you want to perform subboxing or reextract particles for whatever reason, simply use the dynamo2warp script on the correct table and tomogram-table map file at any time and reextract in Warp.

run multi-particle refinements in M