Difference between revisions of "Walkthrough for template matching"

From Dynamo
Jump to navigation Jump to search
Line 51: Line 51:
  
 
=== Through manual alignment ===
 
=== Through manual alignment ===
 +
 +
We will use a model inside [[Dtmslice | <tt>tmslice</tt>]] to pick a set of ~10 particles. We will then crop them, and align them manually using <tt>dgallery</tt>
  
 
====Manual selection of some particles ====
 
====Manual selection of some particles ====
Line 60: Line 62:
 
[[ File:TemplateMatchingTomosliceGeneral.png |thumb|center|500px| <tt>dtmslice</tt> on tomogram ]]
 
[[ File:TemplateMatchingTomosliceGeneral.png |thumb|center|500px| <tt>dtmslice</tt> on tomogram ]]
  
 +
Use the <tt>model pool</tt> menu to add a new model of type <tt>General</tt>
 +
 +
[[ File:TemplateMatchingTomosliceModel.png |thumb|center|500px| Opening a model ]]
  
[[ File:TemplateMatchingTomosliceModel.png |thumb|center|500px| <tt>dtmslice</tt> on tomogram ]]
+
Try to pick particles on different orientations, both in top and side views.  
  
 
[[ File:TemplateMatchingTomoslicePick.png |thumb|center|500px| Pick particles with the <tt>C</tt> key ]]
 
[[ File:TemplateMatchingTomoslicePick.png |thumb|center|500px| Pick particles with the <tt>C</tt> key ]]
 +
 +
You can move the slice with the ''z'' slidebar in the control panel, or dragging it while the mouse button is pressed. Top views are more abundant below the crowded accumulation of side views in the central slice.
  
 
[[ File:TemplateMatchingTomoslicePickTop.png |thumb|center|500px| Use the wheel to go up and down in the tomogram to locate top views ]]
 
[[ File:TemplateMatchingTomoslicePickTop.png |thumb|center|500px| Use the wheel to go up and down in the tomogram to locate top views ]]
  
[[ File:TemplateMatchingMeasure.png |thumb|center|500px  ]]
+
When you have clicked around ten, we will prepare to crop them out of the tomogram. For this, we need to get first an idea of how big should the box. With the keys [1] and [2] you can place anchor points in the scene. Auxiliary click in the bar between them shows the distance (in pixels of the unbinned tomogram).
 +
 
 +
[[ File:TemplateMatchingMeasure.png |thumb|center|500px| Measure distances with the [1] and [2] anchors ]]
 +
 
 +
Now we can open the <tt>dtcrop</tt> utility for this model, by visiting the ''Active model" menu.
  
 
[[  File:TemplateMatchingMenuForDtcrop.png |thumb|center|500px  ]]
 
[[  File:TemplateMatchingMenuForDtcrop.png |thumb|center|500px  ]]
 +
 +
In the GUI that pops up we can fix the sidelength.
  
 
[[  File:TemplateMatchingDtcrop.png |thumb|center|500px  ]]
 
[[  File:TemplateMatchingDtcrop.png |thumb|center|500px  ]]
  
 +
After completion of the cropping process, you will get a [[data folder]] called  <tt>t20s_Particles</tt>, which will also contain a [[table]] file called <tt>t20s_Particles/crop.tbl</tt>
 +
 +
====Manual alignment of some particles ====
 +
 +
Now we want to align the particles. We open them inside the <tt>dgallery</tt> browser:
 +
 +
<tt>dgallery -d t20s_Particles -t t20s_particles/crop.tbl </tt>
  
 +
This GUI has several functionalities. We will focus in using it to quickly get an alignment of the particles.  For this, we first load the particles expressed in the table into the internal memory of the <tt>gallery</tt> by clicking into ''Load''.
  
====Manual alignment of some particles ====
+
[[ File:TemplateMatchingGalleryLoad.png |thumb|center|500px | Preload all the particles into memory ]]
 +
 
 +
This seems not to have any effect in the depiction: you need to fix the range of the depicted particles moving the slider bar. When you do so, each particle will appear in the scene as  a single red box.
 +
 
 +
[[ File:TemplateMatchingGalleryShowAll.png |thumb|center|500px  | Adjust the scene to show all the particles ]]
 +
 
 +
You can play with different settings for viewing, like the direction (''x'', ''y'' or ''z'')
  
 +
[[  File:TemplateMatchingControlViewing.png |thumb|center|500px| viewing direction controls  ]]
  
[[ File:TemplateMatchingGalleryLoad.png |thumb|center|500px  ]]
+
or the number of slices that are projected into the depicted image for each particle.
  
[[ File:TemplateMatchingGalleryShowAll.png |thumb|center|500px  ]]
+
[[ File:TemplateMatchingControlThickness.png |thumb|center|500px | Thickness controls ]]
  
[[ File:TemplateMatchingControlViewing.png |thumb|center|500px  ]]
+
Then, you go through each particle and indicate a point that you considere that could lay in the central axis of the particle by clicking on <tt>N</yy>. The particle will rotate on screen to match this direction. The center of the particle can also be set by clicking on <tt>N</tt>
  
[[  File:TemplateMatchingControlThickness.png |thumb|center|500px  ]]
+
[[  File:TemplateMatchingClickNorth.png |thumb|center|500px| Select the north direction with <tt>N</tt> ]]
  
[[  File:TemplateMatchingClickNorth.png |thumb|center|500px  ]]
+
[[  File:TemplateMatchingClickNorthTopViewX.png |thumb|center|500px | Select the north direction with <tt>N</tt]]
  
[[  File:TemplateMatchingClickNorthTopViewX.png |thumb|center|500px  ]]
+
You may need to complete two rounds of operating on the particles, changing the viewing direction (first in ''x'', then in ''y'')
  
 
==== Averaging manually aligned particles ====
 
==== Averaging manually aligned particles ====

Revision as of 20:40, 21 August 2017


Dynamo includes a set of tools for location of particles inside tomograms. The most simple one is template matching.

Template matching

In this technique, a template representing a molecule of interest is systematically cross-correlated against a tomogram, producing a cross-correlation map of the tomogram. Each pixel in this map represents a score assigned the corresponding pixel in the tomogram map. This score measures the similarity of the neighbourhood of the tomogram pixel to the used template. This similarity is measured exclusively inside a mask.

Data set

Tomogram description

The tomogram contains a buffer with T20S proteasome on a holey carbon grid collected on a Krios + K2. Original pixelsize was 1.76 angstroms. The tomogram provided here has bin binned twice (yielding thus 7.04 ang), defocus is 4.4 microns, no CTF correction, no energy filter.

Acknowledgements

The tomogram has been kindly provided by Alex Noble, from the New York Structural Biology Center. Data collection was performed using Leginon and Appion-Protomo at the Simons Electron Microscopy Center and National Resource for Automated Molecular Microscopy located at the New York Structural Biology Center, supported by grants from the Simons Foundation (349247), NYSTAR, and the NIH National Institute of General Medical Sciences (GM103310) with additional support from Agouron Institute [Grant Number: F00316].

Getting the tomogram

In Linux, you can write:

wget  https://wiki.dynamo.biozentrum.unibas.ch/w/doc/data/t20s/t20s.mrc

the corresponding command is curl -O in the Mac.

Visualizing the tomogram

We can get a first glance on how the tomogram looks like:

dtmshow -otf t20s.mrc  

to use the on-the-fly access to the slice shown at each given moment, or

v = dread('t20s.mrc');dtmshow(v); 

to preload the full tomogram into a memory variable (arbitrarily called v). In either option, you will see, the proteasomes are densely packed in an layer. The layer is slightly oblique, what can be seen browsing through z or y.

dtmshow on the pt20s tomogram, View along z

Navigating on-the-fly you'll see that transitions in y are slower than transitions in z, because all the pixels of the same slice are stored sequentially in the disk.

dtmshow on the pt20s tomogram, View along y


Estimation of the missing wedge

Estimation of the missing wedge'

Creating a template

There are different strategies to create the first template. In the case where the general shape of each protein is roughly recognisable by eye, it is not difficult to just crop and align manually some of the particles. When this is not possible, you have the option of using a density map that mimics the general topology of your protein.

Through manual alignment

We will use a model inside tmslice to pick a set of ~10 particles. We will then crop them, and align them manually using dgallery

Manual selection of some particles

We use for this our tool dtmslice. As the tomogram is provided is fairly small you can probably just open it without any further binning.

dtmslice t20s.mrc -cc ct20

dtmslice on tomogram

Use the model pool menu to add a new model of type General

Opening a model

Try to pick particles on different orientations, both in top and side views.

Pick particles with the C key

You can move the slice with the z slidebar in the control panel, or dragging it while the mouse button is pressed. Top views are more abundant below the crowded accumulation of side views in the central slice.

Use the wheel to go up and down in the tomogram to locate top views

When you have clicked around ten, we will prepare to crop them out of the tomogram. For this, we need to get first an idea of how big should the box. With the keys [1] and [2] you can place anchor points in the scene. Auxiliary click in the bar between them shows the distance (in pixels of the unbinned tomogram).

Measure distances with the [1] and [2] anchors

Now we can open the dtcrop utility for this model, by visiting the Active model" menu.

TemplateMatchingMenuForDtcrop.png

In the GUI that pops up we can fix the sidelength.

TemplateMatchingDtcrop.png

After completion of the cropping process, you will get a data folder called t20s_Particles, which will also contain a table file called t20s_Particles/crop.tbl

Manual alignment of some particles

Now we want to align the particles. We open them inside the dgallery browser:

dgallery -d t20s_Particles -t t20s_particles/crop.tbl 

This GUI has several functionalities. We will focus in using it to quickly get an alignment of the particles. For this, we first load the particles expressed in the table into the internal memory of the gallery by clicking into Load.

Preload all the particles into memory

This seems not to have any effect in the depiction: you need to fix the range of the depicted particles moving the slider bar. When you do so, each particle will appear in the scene as a single red box.

Adjust the scene to show all the particles

You can play with different settings for viewing, like the direction (x, y or z)

viewing direction controls

or the number of slices that are projected into the depicted image for each particle.

Thickness controls

Then, you go through each particle and indicate a point that you considere that could lay in the central axis of the particle by clicking on N</yy>. The particle will rotate on screen to match this direction. The center of the particle can also be set by clicking on N

Select the north direction with N
Select the north direction with N</tt

You may need to complete two rounds of operating on the particles, changing the viewing direction (first in x, then in y)

Averaging manually aligned particles

TemplateMatchingAverageManual.png

Creating a tight mask

Measure distances
TemplateMatchingMarkZaxis.png
TemplateMatchingMarkXYPlane.png
TemplateMatchingClickNorthTopViewX.png
Create mask
TemplateMatchingMaskOptions.png


TemplateMatchingMaskSave.png


Cropping the borders of a template

TemplateMatchingCircshiftTemplate.png
Template cropped to 32 pixels

Through geometrical shapes

Alternatively, you can use dynamo_mask or dynamo_tube to create a synthetic model.

Creating a cross correlation process

ptm = dynamo_match(fileFull,'average48.em','mask','maskData48.em',....
    'outputFolder','cs30',...
    'ytilt',[-39,36],'sc',[1000,1000,300],'cr',360,'cs',30,'bin',1);
------------------------------------------------------------
 
 Template matching process.
 computing in CPU
 Output folder: cs30.TM
A total of 1 tiles have been created
  - Mb per tile (reading)   : 724.19
  - Mb per tile (operation) : 663.84
 ... initializing output elements
Preparing to run on 1 blocks
Running on single processor
 Computing tile 1
   -Range (original block): x[1:958] y[1:926] z[1:214]
   -Range (binned block)  : x[1:479] y[1:463] z[1:107]
  ... tile 1 finished in 387 seconds (8 for setup;  7.73 per triplet)
  [ok] ... template matching process completed upto creation of cross correlation
       you can proceed to peak location and particle extraction.
------------------------------------------------------------

The Process object

The return of the function <notwiki>ptm</nowiki> is an object of class XXX. In short, this is an object created


Considerations when creating a process

Locating cross correlation peaks

Looking at the cross correlation map

TemplateMatchingCCRaw.png

Looking at the cross correlation profile

TemplateMatchingPeakGood.png
TemplateMatchingPeakGoodVolume.png
TemplateMatchingPeakKink.png
TemplateMatchingPeakBad.png


Extracting a table

A table can be extracted through:

myTable = pff.peaks.computeTable('mcc',0.378);

Visual evaluation of results

Looking at table positions

TemplateMatchingTable3d.png
TemplateMatchingTable3dSide.png

Looking at cropped particles

We can check how the individual particles look like on a gallery modus:

pff.peaks.browse();

This order just opens ddbrowse. We are using here the support object peaks, but this command is equivalent to just invoking ddbrowse

ddbrowse('d','cr30.TM','t',myTable);

ddbrowse with a tomogram as a data source, needing the input of a sidelength in pixels
Result launched by ddbrowse without a connected table
The table must be connected to align the particles
z-view of aligned particles
x-view of aligned particles
y-view of aligned particles

Looking at averages

o = pff.peaks.average();

TemplateMatchingAverageRaw.png
TemplateMatchingAverageRandomized.png

Getting the tables back into the catalogue

You don't need to operate with the catalogue: you could just use dtcrop on the just computed table to extract particles from the original tomogram into a data folder. However, cataloguing has to advantages:

  • Eases keeping track of all the steps you've performed, specially if you are going to use several tomograms, and..
  • allows you to visualize the peaks on the tomogram interactively, so that you can delete false positives or add peaks that were not located by the template matching.

Rescaling the table

Remember that the table that we are working with has the scale of the auxiliary binned volume that has been created along with the CC matrix. In the catalogue, we want to work with the particles in their original scale.

tableOriginalScale = dynamo_table_rescale(myTable,'factor',2);

In the syntax of dynamo_table_rescale, the factor is expressed in terms of how many times is the apix in the original table bigger than in the target table to be computed. In our case, the target table was computed with an apix of 14 Angstrom (one time binned in relation to the tomogram t20s.rec, with an apix of 7.04A). The factor is thus 2. In case of doubt, it is convenient to just run

dtinfo(tableOriginalScale);

to check the extent of the entries in the columns 24, 25 and 26, which are positions (in pixels) of the particles indexed by the table. Now, if we write the upscaled table into a file

dwrite(tableOriginalScale,'peaks.tbl');

we can just put this table back into the catalogue through:

dmimport -t peaks.tbl -c ct20s -i 1 -mn ccpeaks

which will create a model called ccpeaks and assign it to the first volume (-i 1) in the catalogue ct20s. You can check it by writing:

dcm -c ct20s -i 1 -l m

which asks for a listing of models (-l m) in the first volume of the catalogue.


TemplateMatchingIntoCatalogue.png
TemplateMatchingIntoCatalogueZoom.png