https://wiki.dynamo.biozentrum.unibas.ch/w/api.php?action=feedcontributions&user=Stefano+Scaramuzza&feedformat=atomDynamo - User contributions [en]2024-03-29T09:49:00ZUser contributionsMediaWiki 1.34.0https://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Starters_guide_(EMBO2021)&diff=4228Starters guide (EMBO2021)2021-09-13T14:17:39Z<p>Stefano Scaramuzza: </p>
<hr />
<div>This tutorial shows how to manually pick particles in a tomogram, extract them, align and classify them. This tutorial is a shortened version of the original [[Starters_guide | starters guide]] that was adapted for the use in ''Dynamo'' workshops.<br />
<br />
=Manual particle picking and ''Dynamo'' catalogues=<br />
The ''Dynamo'' catalogues are databases that manage tomograms and link the tomographic data to the extracted particles. Start the catalogue manager by typing the command:<br />
<br />
<tt>dcm</tt><br />
After the catalogue manager opens, we create 3 synthetic tomograms that include our particles (thermosomes) in the following way:<br />
<br />
[[File:StartersGuideFigure1.png|thumb|center|600px| Figure 1: creating a set of synthetic tomograms with ''Dynamo'' Catalogue Manager]]<br />
<br />
You can see the list of your tomograms and their metadata in a table in the bottom of your catalogue manager. When workin on your own projects, you can add tomograms to the catalogue by ''Catalogue -> Browse for new volume''. The tomograms are situated in the directory <tt>testCatalogue</tt>. They contain a small amount of noise and a missing wedge associated to rotation around Y-axis. We added the information about location of the particles in the tomograms in an additional catalogue called <tt>testCatalogue_withmodels</tt>. If needed, it can be opened from the current folder with ''Catalogue -> look for local catalogues'' in the catalogue manager window.<br />
<br />
==Viewing tomograms==<br />
Select the first tomogram in the list and go to ''View volume>Full tomogram file in tomoslice''. <br />
<br />
[[ File:StartersSelectTomoSliceInCatalogue.png |thumb| center |600px| Figure 2: Selecting a volume in the catalogue manager to be inspected in <tt>dpreview</tt>]]<br />
<br />
The volume browser tomoslicer loads the entire tomograms into memory and allows making annotations to the regions of interest. The tomograms in this tutorial are small, and you can load them directly into memory. For real life tomograms, bear in mind that you will need to [[Prebinned tomograms| prebin]] the files before loading them into memory. Tomoslice has a simple set of controls and is suitable for visualization tasks that require oblique sections through the tomogram. It uses the same tool as other ''Dynamo'' browsers to keep track of your annotations: a pool of models. You might need to adjust the contrast (blue arrow). To move through the tomogram slices, you can either use the mouse wheel, click and drag the tomogram slice up and down (orange arrows), or move the position control left and right (orange box).<br />
<br />
<br />
[[ File:StartersTomoSlice.png |thumb| center |600px|]]<br />
<br />
==Picking and extracting particles in tomograms==<br />
Coordinates of picked particles are represented by data types called models. In the tomoslicer window go to ''Model Pool -> Create new model in pool (choose type) -> General''. This is the simplest type of [[model]] where each clicked/model point corresponds to a single isolated particle. Now you can navigate up and down the tomogram, place the mouse on the center of a particle and press the [c] key on your keyboard to add a new model point. (Note the ''Help -> all hot keys'' options that lists the different the actions of the different keystrokes). Backspace button deletes the last clicked point (you can also use the right-click to delete single points).<br />
<br />
[[File:StartersGuideFigure4.png|thumb|center|600px| Generating a new model]]<br />
<br />
After you are done clicking about 10 particles click on ''Active model > Update Crop points'' in the tomoslice window.<br />
<br />
<br />
[[File:StartersGuideUpdate.png|thumb|center|600px|]]<br />
<br />
Save the model into the catalogue by ''Active model -> Save active model into catalogue (disk)'' and close the slicer window.<br />
<br />
[[File:StartersGuideFigure6.png|thumb|center|600px| ]]<br />
<br />
Pick particles for tomograms 2 and 3. In total, there are around 30-40 particles in the generated dataset. When you open a new tomogram, make sure that you delete the pool of models from memory when asked. This will have no effect onto the models stored in disk, and it is necessary in order to ensure that you are not mixing models from different tomograms.<br />
<br />
==An alternate visualization: orthogonal projections==<br />
<br />
Clicking in the menu on ''Projection -> project full shown fragment along z'' (still in dtmslice) and you will get a screen where the x-y projection of the tomogram is shown. Use the secondary click on it to launch the orthogonal views of x-z and y-z planes that traverse that point. These views can also be used to click particles, in case the standard view is not sufficient.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:orthoPickMain.png|thumb|upright|250px| click on the main projection determines two orthogonal slices]]<br />
| [[File:orthoPickX.png|thumb|upright|250px| Slice xz ]]<br />
| [[File:orthoPickY.png|thumb|upright|250px| Slice xy]]<br />
|}<br />
<br />
=Extracting particles from tomograms=<br />
In the catalogue manager window select the rows for tomograms from which you want to extract particles from. You can either select them one by one with a mouse click and holding the [ctrl] key or by clicking the ''Select all'' button. Then, go to ''Crop Particles -> Open Volume List Manager''.<br />
<br />
[[File:StartersGuideopenVolumeList.png|thumb|center|600px|]]<br />
<br />
A new window opens with all the models in the catalogue listed in the bottom. Pick (by checking boxes) the models of type ''general'' that you have just clicked. Click ''Create list'' and then ''Crop particle''.<br />
<br />
[[File:StartersGuideSelectModels.png|thumb|center|600px|]]<br />
<br />
In the new window, change the data name to something meaningful, such as ''thermosomeParticles'', change the sidelength from 32 to 48 and click start cropping.<br />
<br />
[[File:StartersGuideCrop.png|thumb|center|600px|]]<br />
<br />
After cropping is done, explore your cropped particles by clicking ''ddbrowse'' (in the window above). A new window opens where you can simply click on ''show''.<br />
<br />
[[File:StartersGuideShow.png|thumb|center|500px|]]<br />
<br />
You see a 2D representation (projections) of all cropped 3D subtomograms. Make sure your particles are well centered and fit the box.<br />
<br />
[[File:StartersGuideParticles.png|thumb|center|500px|]]<br />
<br />
We also introduce the concept of the ''dynamo'' table: The information about each particle is stored in tables. Each particle has an entry in the table that contains shifts and rotations to describe its orientation (by default these values are initialized as zeroes). It also contains the particle ID (tag), the orientation of the missing wedge and others. For full info type the command <tt>dthelp</tt>. The ''Dynamo'' catalogue generates an initial table during particle extraction. Its location is in <tt>particles/crop.tbl </tt>. <br />
<br />
=Subtomogram alignment and averaging=<br />
<br />
==Initial reference generation==<br />
We want to align the extracted tomograms to a common reference. For that, we need an initial reference (which will later be refined during the iterative alignment). Here, we create such an initial reference by manually align a few particles and average them. We use the command <tt>dgallery</tt> to generate initial orientations for some of our particles that will be used to generate the initial reference. Close all open ''Dynamo'' windows and type into the ''Dynamo'' command line:<br />
<br />
<tt>dgallery('data','thermosomeParticles.Data');</tt><br />
<br />
The gallery opens. Click on ''load'' to load all aprticles in memory, move the ''shown'' bar to display the particles and use the x-, y- and z-buttons to see different views:<br />
<br />
[[File:StartersGuideGallery.png|thumb|center|600px|]]<br />
<br />
To manually align the particles, do the following for about 10 particles:<br />
# Place the mouse over the particle center and press the key [c] (this centers the particle).<br />
# Place the mouse over its top (or bottom) part and press the key [n] (this aligns the particle).<br />
# Click on the particle to make sure its number turns from red to blue (blue means it is selected for further processing). To de-select a particle use the right-click.<br />
Change between the x-, y- and z-views to correct the orientations if necessary. This is just done to create an initial reference, so orientations don't need to be very exact.<br />
<br />
[[File:StartersGuideGallery2.png|thumb|center|500px|]]<br />
<br />
Save the selected tags and corresponding table by clicking ''quick save'' button in the top right. It saves a quickbuffer.tbl and quickbuffer.tags to the hard drive that will be used later. To generate the average you need to apply the table on the particles. For this click the ''average'' button in the Particle selection field.<br />
<br />
[[File:StartersGuideGallery3.png|thumb|center|500px|]]<br />
<br />
It opens a new window with a lot of controls. Click ''compute average'' in the bottom of the window. Then, right-lick on the output filename and click ''ok'' in the next window to see the result. If you are not satisfied with the result, close the window and refine your manual alignment and/or add more particles to the average.<br />
<br />
[[File:StartersGuideGallery4.png|thumb|center|600px|]]<br />
<br />
[[File:StartersGuideGallery5.png|thumb|center|500px|]]<br />
<br />
==Alignment projects==<br />
Iterative alignment of subtomograms to their average is performed by ''dynamo'' projects that you can run in various high-performance computational environments. To run an alignment project you need the particles, an initial reference and a table. All of these files we generated before. You can close all the previous GUIs and start the alignment project GUI by typing<br />
<br />
<tt> dcp</tt><br />
<br />
In thw new window, do the following:<br />
# Add the project name ''drun1'', press the ''enter'' key and select ''create a new project'' in the pop-up window.<br />
# Click on ''particles'' and provide the particle folder name ''thermosomeParticles.Data''. Click ok.<br />
# Click on ''table'' and provide the table name ''thermosomeParticles.Data/crop.tbl''. Click ok.<br />
# Click on ''template'' and provide the initial reference name ''my_average.em''. Click ok.<br />
# Click on ''masks'' and simply click on ''use default masks''. Click ok. You could also specify the semi-axis of the ellipsoid masks or go to “Mask editor” to make more sophisticated masks. Note that ''Dynamo'' by default uses Rossman correlation which eliminates the artefacts associated with hard (non-soft) mask. <br />
<br />
[[File:StartersGuideAli1.png|thumb|center|500px|]]<br />
<br />
Click on ''numerical parameters'' to set all parameters and details about the different iterations of the alignment project. You can select any parameter in the table with a mouse and click the ''?'' button on the top right of the parameter window to see a description of the parameters. The most important parameters to consider are:<br />
<br />
* ''Number of iterations'': Make 3 rounds with 2 iterations each. The first round will be a global search with a coarse angular step and the following rounds will be used for refinement.<br />
* ''Angular search ranges'': Cone aperture is the scan range for the first two Euler angles around the initial orientation defined in the table. 360 degrees is the full scan range. Azimuth rotation range defines the rotation range around the new vertical axis of the particle.<br />
* ''High- and lowpass values'': Fourier voxels to limit the used frequency range.<br />
*''Particle dimensions'': Defined as sidelength of your subvolume. If you put a lower value, the particles will be downsampled for the particular round. This will speed up the process.<br />
* ''Refine'': After each angular scan, the search step is reduced by the ''refine factor''. This is repeated ''refine'' times. I.e., if your cone sampling is 10 degrees, ''refine'' is 3 and ''refine factor'' is 2, then 10, 5, 2.5 and 1.25 degrees will be sampled. This is the optimization of angular search space.<br />
*''Shift limits'': Limits the translation of particles from the center of the box (if shifts limiting way is 1) or from the previously estimated center (if shifts limiting way is 2). The previous estimates for the shifts and rotations are taken from input tables and are updated at the end of each iteration.<br />
*''Symmetry'': If you know the symmetry of your protein it will speed up the convergence and result in higher resolution.<br />
<br />
Set the parameters as follows and click ''ok'':<br />
<br />
[[File:StartersGuideFigure10.png|thumb|center|600px|]]<br />
<br />
Click on ''Computing environment'' and select ''standalone''. Set ''CPU cores'' and ''parallelize averaging step'' to the maximum available (see the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have). Leave the rest as it is and click ''ok''. In the alignment GUI click ''check'' and ''unfold''. To run the project open a new terminal (in a new tab), load ''Dynamo'' but do not run it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and run the project by typing <tt>./drun1.exe</tt>.<br />
<br />
While the project is running, go back to the terminal with ''Dynamo'' running and you can monitor the progress of the alignment project by typing:<br />
<br />
<tt>dvstatus drun1</tt><br />
<br />
While the project is running, you can look at projections of the intermediate results by typing:<br />
<br />
<tt>ddb drun1:a:ite=* -j c10</tt><br />
<br />
or at the latest average by typing:<br />
<br />
<tt>ddb drun1:a -v </tt><br />
<br />
The final result should look similar to:<br />
<br />
[[File:StartersGuideResultsAlignment.png|thumb|center|400px| Results of alignment.]]<br />
<br />
<br />
=Subtomogram classification=<br />
To demonstrate a classification example we provide particles with 2 slightly different sizes. We can then classify them with a simple Multi-reference Analysis (MRA). To do that, we first generate a dataset which has 2 populations of particles. For this, type:<br />
<br />
<tt>dynamo_tutorial('data_classification','M',8,'N',8);</tt><br />
<br />
It generates two sets of particles, 8 particles in each including their table. The idea behind [[Multireference Analysis|MRA]] is the following: The particles are aligned to several references (here only 2). After each iteration, each particle is assigned to only one single reference where it fits the best. This procedure is repeated iteratively. Similar particles will have higher correlation to similar references and will eventually group together. For classification into N classes we will need N initial references and N initially identical tables.Follow these steps to run the MRA:<br />
<br />
# Create a new project by typing “drun2” in the project name of the alignment GUI.<br />
# Set number of references to 2.<br />
# Activate the ''Swap particles'' button.<br />
# Go to ''particles'' and add <tt>data_classification/data</tt> to the particle data. Click ok.<br />
# Go to ''table'' and type <tt>data_classification/real.tbl</tt> into the field ''clone this'' and press ''copy''. Click ok.<br />
# Go to ''template'' and clone <tt>data_classification/original_template.em</tt> with addition of some extra noise with amplitude 1. <br />
# Click on ''masks'' and click on ''use default masks''. Click ok.<br />
# Click on ''numerical parameters'' and set the following parameters. Click ok.<br />
<br />
[[File:StartersGuideResultsClassification2.png|thumb|center|400px|]]<br />
<br />
Choose the same computing environment as in the previous alignment project and run the project in the same way too. MRA projects usually run longer, because each particle will be aligned to two references. Here, however, we chose the parameters in a way that the process is still fast (e.g., small dimensions). Monitor the progress of the project same as before:<br />
<br />
<tt>dvstatus drun2</tt><br />
<br />
And visualize the results using the command:<br />
<br />
<tt>ddb drun2:a:ref=* -j c5</tt><br />
<br />
You should get two averages with particle in one slightly class larger than in the other.<br />
<br />
[[File:StartersGuideResultsClassification.png|thumb|center|600px| Results of classification.]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4227Advanced starters guide (EMBO2021)2021-09-13T10:09:32Z<p>Stefano Scaramuzza: /* Data */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory of day 6 (<tt>Practical 6</tt> or <tt>prac-6</tt>). See [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory.<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units in dmapview:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or Chimera:<br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4226Advanced starters guide (EMBO2021)2021-09-13T10:08:58Z<p>Stefano Scaramuzza: /* Data */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory. Make sure you are in the directory of day 6 (<tt>Practical 6</tt> or <tt>prac-6</tt>). See [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory.<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units in dmapview:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or Chimera:<br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4225Advanced starters guide (EMBO2021)2021-09-13T10:08:20Z<p>Stefano Scaramuzza: </p>
<hr />
<div>[[Category:Walkthroughs]]<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory. Make sure you are in the directory of day 6 (''Practical 6'' or ''prac-6''). See [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory.<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units in dmapview:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or Chimera:<br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4224Advanced starters guide (EMBO2021)2021-09-13T10:07:43Z<p>Stefano Scaramuzza: </p>
<hr />
<div>[[Category:Walkthroughs]]<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory. Make sure you are in the directory of day 6 (practical 6). See [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory.<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units in dmapview:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or Chimera:<br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4219EMBO workshop 20212021-09-10T12:32:08Z<p>Stefano Scaramuzza: /* Schedule */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30, London time) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work '''only''' within these two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time (London)'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time (London)'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4218EMBO workshop 20212021-09-10T12:31:47Z<p>Stefano Scaramuzza: </p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30, London time) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work '''only''' within these two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time (London)'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4217EMBO workshop 20212021-09-07T16:25:22Z<p>Stefano Scaramuzza: /* Data location */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work '''only''' within these two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4216EMBO workshop 20212021-09-07T16:25:09Z<p>Stefano Scaramuzza: /* Data location */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work '''only''' within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4215EMBO workshop 20212021-09-07T15:59:20Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4214EMBO workshop 20212021-09-07T15:48:24Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4213EMBO workshop 20212021-09-07T15:47:20Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera, ...). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4212EMBO workshop 20212021-09-07T15:46:28Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects, copy files, use chikmera, ...). Be aware that if you press ctrl+c within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4211EMBO workshop 20212021-09-07T15:44:55Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab'') and to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else. Be aware that if you press ctrl+c within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4210Advanced starters guide (EMBO2021)2021-09-07T15:42:22Z<p>Stefano Scaramuzza: /* Subboxing project */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units in dmapview:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or Chimera:<br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4209Advanced starters guide (EMBO2021)2021-09-07T15:40:32Z<p>Stefano Scaramuzza: /* Project with rotational randomization */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera (click on ''save'' in the bottom left of the dmapview window and open the file <tt>temp.em</tt> in chimera):<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4208Advanced starters guide (EMBO2021)2021-09-07T15:37:08Z<p>Stefano Scaramuzza: /* Creating an average */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type:<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4207Advanced starters guide (EMBO2021)2021-09-07T15:36:51Z<p>Stefano Scaramuzza: /* Project to find membrane orientations */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type"<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera) you get:<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4206Advanced starters guide (EMBO2021)2021-09-07T15:36:12Z<p>Stefano Scaramuzza: /* Project to find membrane orientations */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type"<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (save the file by clicking the ''Export>into file'' option in the <tt>dview</tt> window and then open the file <tt>temp.em</tt> in chimera).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4205Advanced starters guide (EMBO2021)2021-09-07T15:30:51Z<p>Stefano Scaramuzza: /* Creating an average */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type"<br />
<br />
<tt> /usr/local/bin/chimera first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4204Advanced starters guide (EMBO2021)2021-09-07T15:27:49Z<p>Stefano Scaramuzza: /* Creating an average */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want, you can look at the average in chimera. To do that, first save the average:<br />
<br />
<tt> dwrite(oa.average,'first_average.em') </tt><br />
<br />
Then, if you haven't already, open a new terminal (in a new tab) and type"<br />
<br />
<tt> /usr/local/bin/chimera myFile.mrc first_average.em</tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4203Advanced starters guide (EMBO2021)2021-09-07T09:56:26Z<p>Stefano Scaramuzza: </p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4202Advanced starters guide (EMBO2021)2021-09-07T09:20:29Z<p>Stefano Scaramuzza: /* Subboxing project */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<nowiki>dvput subboxBig dst standalone<br />
dvput subboxBig cores 24<br />
dvput subboxBig mwa 24 </nowiki><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4201EMBO workshop 20212021-09-07T09:02:42Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
We suggest to open a new tab in the terminal (''File -> Open Tab'') and to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else. Be aware that if you press ctrl+c within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4200Advanced starters guide (EMBO2021)2021-09-07T09:00:33Z<p>Stefano Scaramuzza: /* Subboxing project */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<tt> dvput subboxBig dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4199EMBO workshop 20212021-09-06T16:07:22Z<p>Stefano Scaramuzza: </p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
We suggest to open a new tab in the terminal and to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else. Be aware that if you press ctrl+c within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| <br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021)&diff=4198Walkthrough on command line based tilt series alignment (EMBO2021)2021-09-06T16:06:47Z<p>Stefano Scaramuzza: </p>
<hr />
<div>Dynamo includes a package for automated alignment and reconstruction of tilt series. This walkthrough guides users through a series of steps on how to use it in command line based mode. The [[Walkthrough on GUI based tilt series alignment | GUI based alignment walkthrough]] shows how to operate this procedure through the GUI. Here, we go through each command one by one.<br />
<br />
= Create the workflow =<br />
The following commands create a new alignment workflow in a separate folder. Make sure ''Dynamo'' is loaded and running (as explained in the [[EMBO_workshop_2021 | workshop page]]) and type the following commands into the console. Note: The workshop setup does not allow you to copy paste commands from your desktop onto the workshop server. You need to open a browser within the workshop server and copy paste the commands from there.<br />
<br />
<nowiki>name = 'hivCommandLine';<br />
folder = 'workflows';<br />
u = dtsa(name,'--nogui','-path',folder,'fp',1); </nowiki><br />
<br />
Type <tt>whos</tt> to see a list of the variables that you just created with these commands.<br />
<br />
= Entering the data =<br />
<br />
The variable <tt>u</tt> is an object that contains several ''fields'' to interact with the workflow. The different fields can be found through autocompletion by pressing the enter key (see image below). In the following, these different fields will be used to pass our inputs to the workflow.<br />
<br />
[[ File:alignCommandLine_autocompletionOfUserObjectEMBO2021.png |thumb|center| 700px|autocompletion of user object to access different fields in standalone version]]<br />
<br />
== Basic data==<br />
Basic data can be entered using the ''enter'' field. Here we use a binned version of the first tilt series from the EMPIAR entry 10164, depicting a set of virus like particles (VLP), which is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory). The file of the tilt series and the angles are passed to the workflow:<br />
<br />
<nowiki>u.enter.tiltSeries('b001ts001.mrc');<br />
u.enter.tiltAngles(-57:3:60); </nowiki><br />
<br />
If necessary (not in this tutorial), you can use the ''discardedTiltIndices'' field to reject tilts that are of too bad quality using the following command (e.g., tilt 1 and 40):<br />
<tt>u.enter.discardedTiltIndices([1,40]); </tt><br />
<br />
== Acquisition settings==<br />
Here you can enter parameters such as <tt>Cs</tt>, nominal defocus, etc. For this tutorial we just provide the pixel size in Angstroms:<br />
<tt>u.enter.settingAcquisition.apix(2.7);</tt><br />
<br />
== Computation settings==<br />
We enable the use of parallel cores<br />
<tt>u.enter.settingComputing.parallelCPUUse(1); </tt><br />
This will be using mainly for gold bead detection (cc computation) and reconstruction.<br />
<br />
To use all available physical cores use:<br />
<tt>u.enter.settingComputing.cpus('*');</tt><br />
<br />
== Detection settings==<br />
These here are the actual design decisions when running an alignment workflow. The gold bead radius has to be known beforehand ( see [[Walkthrough on GUI based tilt series alignment | GUI based alignment walkthrough]] on how to measure it). Binning is used internally only for quick and robust bead detection.<br />
<br />
<nowiki>u.enter.settingDetection.detectionBinningFactor(1);<br />
u.enter.settingDetection.beadRadius(16);<br />
u.enter.settingDetection.maskRadius(28);<br />
u.enter.templateSidelength(64);</nowiki><br />
<br />
== Changing generic parameters ==<br />
You can find all parameters of the individuals steps using autocompletion on the <tt>area</tt> field and then by adding <tt>step</tt> in the end. For example, the residual threshold of the tilt gap filler can be changed as follows:<br />
<tt>u.area.indexing.step.tiltGapFiller.parameterSet.residualsThreshold(8);</tt><br />
<br />
= Running the workflow =<br />
After setting the input parameters, the complete workflow can be run using the following command (we skip the CTF correction here). It should take about 5 minutes to run.<br />
<tt>u.run.all('noctf',1);</tt><br />
<br />
== Checking the results ==<br />
After completion, the <tt>view</tt> field allows you to check different output items for diagnose, such as the tilt lines:<br />
<br />
<tt>u.view.tiltLines.unaligned();</tt><br />
<br />
[[ File:alignCommandLine_viewOfTiltLines.png |thumb|center| 600px|view of tilt lines]]<br />
<br />
or the different reconstructions <br />
<br />
<nowiki>u.view.reconstruction.binnedWBP();<br />
u.view.reconstruction.binnedSIRT();</nowiki><br />
<br />
or the aligned stacks themselves.<br />
<br />
<tt>u.view.stack.alignedFull();</tt><br />
<br />
You can also use the autocompletion of the <tt>info</tt> field (as shown in the beginning) to find and display different logs:<br />
<br />
<nowiki>>> u.info.fit;<br />
------------------------------------------------------------<br />
File with info:<br />
workflows/hivCommandLine.AWF/info/fitting.doc<br />
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <br />
rms: 1.41<br />
psi: 85.20<br />
fit modus: normal<br />
total markers: 79<br />
total observations: 2702<br />
fit created at: 11-Aug-2021 10:56:24<br />
------------------------------------------------------------</nowiki><br />
<br />
<nowiki>>> u.info.markers;<br />
------------------------------------------------------------<br />
Info on working markers [used during detection, indexing refinement]<br />
... item saved and in memory<br />
Item in memory:<br />
Number of shapes : 79<br />
Number of frames : 40<br />
Number of observations : 2702 (out of 3160)<br />
Number of empty frames : 1<br />
Number of complete frames : 0<br />
Number of complete shapes : 0<br />
<br />
Item in disk:<br />
Markers in disk and currently in memory contain equal coordinates<br />
<br />
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <br />
<br />
Info on final markers [used from alignment step]<br />
... item saved and in memory<br />
Item in memory:<br />
Number of shapes : 79<br />
Number of frames : 40<br />
Number of observations : 2702 (out of 3160)<br />
Number of empty frames : 1<br />
Number of complete frames : 0<br />
Number of complete shapes : 0<br />
<br />
Item in disk:<br />
Markers in disk and currently in memory contain equal coordinates<br />
------------------------------------------------------------</nowiki></div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4197EMBO workshop 20212021-09-06T16:05:28Z<p>Stefano Scaramuzza: </p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Data location ===<br />
<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
Then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
We suggest to open a new tab in the terminal and to navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else. Be aware that if you press ctrl+c within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4196EMBO workshop 20212021-09-06T16:02:39Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
Open a new tab in the terminal and navigate to the same directory. You can use this tab for normal linux commands so you do not have to quit ''Dynamo'' anytime you want to do something else. Be aware that if you press ctrl+c to within ''Dynamo'' you exit ''Dynamo'' and lose your workspace.<br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4195EMBO workshop 20212021-09-06T16:01:10Z<p>Stefano Scaramuzza: /* Load and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4194EMBO workshop 20212021-09-06T16:00:28Z<p>Stefano Scaramuzza: /* Connect to Guacamole and run Dynamo */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
Open a new tab in the terminal and navigate to the same directory. You can use this tab for normal linux commands so you do not have to quit ''Dynamo'' anytime you want to do something else.<br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4193Advanced starters guide (EMBO2021)2021-09-06T15:55:57Z<p>Stefano Scaramuzza: /* Subboxing project */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4192Advanced starters guide (EMBO2021)2021-09-06T15:54:46Z<p>Stefano Scaramuzza: /* Pick membrane points */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4191Advanced starters guide (EMBO2021)2021-09-06T15:54:20Z<p>Stefano Scaramuzza: /* Project to find membrane orientations */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), loading ''Dynamo'' but not running it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and starting the alignment project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
or, if you are working with the [[standalone]] version, please prepend a "\" symbol:<br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4190Advanced starters guide (EMBO2021)2021-09-06T15:53:07Z<p>Stefano Scaramuzza: </p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. Chose the ''standalone'' option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box). See the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have. <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by first opening a new terminal (in a new tab), load ''Dynamo'' but do not run it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and run the project by typing:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
or, if you are working with the [[standalone]] version, please prepend a "\" symbol:<br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Main_Page&diff=4189Main Page2021-09-06T15:40:57Z<p>Stefano Scaramuzza: /* Cite */</p>
<hr />
<div>{{DISPLAYTITLE:''Dynamo''}}<br />
<br />
<br />
''Dynamo'' is a software environment for subtomogram averaging of cryo-EM data.<br />
<br />
This wiki [[Getting started|guides new ''Dynamo'' users]] all the way from tomograms to averages and classes. In a full workflow, you would organize tomograms in [[Catalogue | catalogues]], use them to [[Particle picking|pick particles]] and create [[Subtomogram alignment|alignment]] and [[Classification |classification projects]] to be run on different [[computing environments]].<br />
<br />
= Participate =<br />
<br />
There is a [[https://groups.google.com/forum/#!forum/dynamo-for-cryo-electron-tomography forum in Google Groups]] where you can present your questions to developers and other users. Bear in mind that you need a Google account for this.<br />
<br />
= Cite =<br />
Please cite these papers if ''Dynamo'' is useful in your research:<br />
<br />
* [http://www.sciencedirect.com/science/article/pii/S1047847711003650|''Dynamo: A flexible, user-friendly development tool for subtomogram averaging of cryo-EM data in high-performance computing environments.''] Castaño-Díez D, Kudryashev M, Arheit M, Stahlberg H., J Struct Biol. 2012.<br />
<br />
* [http://www.sciencedirect.com/science/article/pii/S1047847716301113| ''Dynamo Catalogue: Geometrical tools and data management for particle picking in subtomogram averaging of cryo-electron tomograms''] Castaño-Díez D, Kudryashev M, Stahlberg H., in J Struct Biol. 2017 Feb; 197(2):135-144<br />
<br />
* [https://doi.org/10.1107/S2059798317003369''The Dynamo package for tomography and subtomogram averaging: components for MATLAB, GPU computing and EC2 Amazon Web Services''] Castaño-Díez D. in Acta Crystallographica Section D: Structural Biology 73 (6).<br />
<br />
* [https://doi.org/10.3389/fmolb.2018.00082''Protocols for Subtomogram Averaging of Membrane Proteins in the Dynamo Software Package''] Navarro PP, Stahlberg H and Castaño-Díez D (2018). Front. Mol. Biosci. 5:82.<br />
<br />
* [https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001318''Step-by-step guide to efficient subtomogram averaging of virus-like particles with Dynamo''] Scaramuzza S, Castaño-Díez D (2021). PLoS biology. 19(8):e3001318.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4188Advanced starters guide (EMBO2021)2021-09-06T15:18:04Z<p>Stefano Scaramuzza: /* Data */</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
Type the following command to get information about the tomogram:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
You should get:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. If you are working with Matlab, select ''Matlab''(for one single core) or "Matlab_parfor" (if you are going to several cores). If working with the standalone, use the [[standalone]] option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box) . <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by invoking the name of the execution script. If you are running in Matlab use:<br />
<br />
<tt>first</tt><br />
<br />
If you are running ''Dynamo'' in standalone modus, open a new tab (or window) of the terminal, load Dynamo (but don't run Dynamo) and type:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
or, if you are working with the [[standalone]] version, please prepend a "\" symbol:<br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4187Advanced starters guide (EMBO2021)2021-09-06T15:16:31Z<p>Stefano Scaramuzza: </p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Data ==<br />
<br />
The tomogram is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
===Size check of a file===<br />
<br />
You're probably curious to see what's inside, so that let's write first:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
to let ''Dynamo'' check the dimensions of the file. The header of a <tt>.rec</tt> file is readed as a regular <tt>mrc</tt>, yielding:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
So, it's a tomogram.<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. If you are working with Matlab, select ''Matlab''(for one single core) or "Matlab_parfor" (if you are going to several cores). If working with the standalone, use the [[standalone]] option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box) . <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by invoking the name of the execution script. If you are running in Matlab use:<br />
<br />
<tt>first</tt><br />
<br />
If you are running ''Dynamo'' in standalone modus, open a new tab (or window) of the terminal, load Dynamo (but don't run Dynamo) and type:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
or, if you are working with the [[standalone]] version, please prepend a "\" symbol:<br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4186EMBO workshop 20212021-09-06T15:13:54Z<p>Stefano Scaramuzza: </p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Advanced_starters_guide_(EMBO2021)&diff=4185Advanced starters guide (EMBO2021)2021-09-06T15:13:20Z<p>Stefano Scaramuzza: Created page with "Category:Walkthroughs This walkthrough uses a small size example based on a real tomogram to cover several tasks. == The example data set == The data is a fraction of a..."</p>
<hr />
<div>[[Category:Walkthroughs]]<br />
<br />
This walkthrough uses a small size example based on a real tomogram to cover several tasks.<br />
<br />
== The example data set ==<br />
<br />
The data is a fraction of a tomogram. The full tomogram was used in "Cryo-electron tomography reveals novel features of a viral RNA replication compartment." (Ertel et al.), and represents several FHV viruses docked in the outer membrane of a mitochondrion.<br />
<br />
[[File:FhvDefinitionCrown.png|thumb|center|500px| Average for several hundreds of particles. The "crown" area is shown under a red shadow.]]<br />
<br />
== Downloading ==<br />
<br />
During the Dynamo workshop you will find the needed tilt series in your data folder. Outside the workshop, you can download all the files related to this example with the command:<br />
<br />
<tt>dpkhelp.wiki.downloadExample('fhv');</tt><br />
<br />
If it fails under Matlab or the ''Dynamo'' command line, you can try to directly use the linux order<br />
<br />
<tt>wget https://wiki.dynamo.biozentrum.unibas.ch/w/doc/data/fhv/crop.rec </tt><br />
<br />
or <br />
<br />
<tt>curl -O https://wiki.dynamo.biozentrum.unibas.ch/w/doc/data/fhv/crop.rec </tt><br />
<br />
under MacOS. This should have created the file called <tt>crop.rec</tt> in your current directory. <br />
<br />
===Size check of a file===<br />
<br />
You're probably curious to see what's inside, so that let's write first:<br />
<br />
<tt>dfile crop.rec</tt><br />
<br />
to let ''Dynamo'' check the dimensions of the file. The header of a <tt>.rec</tt> file is readed as a regular <tt>mrc</tt>, yielding:<br />
<br />
<nowiki>filetype: volume<br />
size: 1285 x 956 x 786</nowiki><br />
<br />
So, it's a tomogram.<br />
<br />
===Lightweight visualization===<br />
We can inspect quickly its contents with [[Viewing_tomograms#dtmshow | dtmshow]]<br />
<br />
<tt>dtmshow -otf crop.rec</tt><br />
<br />
Hereby, the flag <tt>-otf</tt> means "on the fly", telling <tt>dtmshow</tt> to ''not'' preload the full tomogram, but to access in disk the individual slices that are needed when inspecting a particular area.<br />
<br />
[[File:FhvTomoshowOnTheFly.png|thumb|center|500px|Basic controls of <tt>dtmshow</tt>]]<br />
<br />
Go up and down. We want to select the locations were the vesicles intersect the mitochondrion membrane and average them together. For this, we need to ''catalogue'' the tomogram, so that our annotations are stored with a clear relationship with the tomogram.<br />
<br />
== Cataloguing the tomogram ==<br />
<br />
We can create catalogues just to contain a single tomogram. They are useful to keep track of all annotations, and of the typical transforms (binning, cropping of fractions) that we usually perform on a larged size tomogram of interest. In this case, we can create the catalogue directly from the command line: <br />
<br />
<tt> dcm -create fhv </tt><br />
<br />
where <tt>dcm</tt> is the short form of <tt>dynamo_catalogue_manager</tt> and <tt>fhv</tt> is just an arbitrary name. The just created catalogue is empty, and we can add our tomogram with:<br />
<br />
<tt>dcm -c fhv -at crop.rec</tt><br />
<br />
We can check that the tomogram is in the catalogue by asking ''Dynamo'' to show the contents of the catalogue<br />
<br />
<tt>dcm -c fhv -l tomograms</tt><br />
<br />
or <br />
<br />
<tt>dcm -c fhv -l t</tt><br />
<br />
The flag <tt>-l</tt> asks ''Dynamo'' to list items of a given category of catalogue contents, in this case <tt>tomograms</tt> <br />
<br />
=== Prebinning the tomograms ===<br />
<br />
We typically want to [[prebinned tomograms | prebin]] the tomogram, i.e., have a version of |smaller size that is known to the catalogue. This version will be useful in some operations that require a full tomogram in memory, an operation that can consume much memory and need a long time. In this example, this would not be necessary: a tomogram with a sidelength on x and y of ~1000 pixels shouldn't pose any visualization problem. We do it anyway for the exercise with the command:<br />
<br />
<tt>dynamo_catalogue_bin('fhv', 1, 'zchunk', 300); </tt><br />
<br />
where the parameter <tt>zchunk</tt> represents the maximum number of z slices that are kept simultaneously in the memory during the binning process. This parameter might be important for larger size tomograms.<br />
<br />
=== Operation with GUI===<br />
<br />
These steps could have been performed thorough the <tt>dcm</tt> GUI<br />
<br />
== Annotation of particle positions ==<br />
Now we can open the tomogram through the catalogue:<br />
<tt>dtmslice crop.rec -c fhv -prebinned 1 </tt><br />
<br />
[[File:FhvTomosliceOpen.png|thumb|center|500px|dtmslice opened on the FHV example tomogram<tt>dtmslice</tt>]]<br />
<br />
Probably you don't like the initial contrast, change it with the button in the toolbar.<br />
<br />
[[File:FhvTomosliceContrast.png|thumb|center|500px|contrast adjustment options inside <tt>dtmslice</tt>]]<br />
<br />
=== Navigating the tomogram ===<br />
<br />
* left click and drag the slice with your mouse to move it up and down<br />
* this can also be achieved with the controls on the left hand side<br />
* ctrl + left click and drag will change your viewpoint of the 3d scene<br />
* shift + left click and drag will move the center of rotation<br />
* ctrl + mouse wheel up/down will zoom in or out of the scene<br />
<br />
Other [[Dtmslice#Viewing controls | axiliary tools]] are the keys x,y,z to change the slice orientation, the number of projected slices (called "thickness" in the GUI controls), the transparency of the slice.<br />
<br />
You can reset the scene at any time by hitting the reset button.<br />
<br />
You can save slices for viewing later with the s key.<br />
<br />
=== Creation of models to contain annotations===<br />
<br />
In this example we just want to manually pick some particles. This can be done creating a general or box [[models|model]], which will [[model pool | reside in memory ]] till we save it into the catalogue. <br />
<br />
[[File:FhvBoxesModel.png|thumb|center|500px|Creating a new model in the scene. Each annotation is represented by a box]]<br />
<br />
After creating the model, it will be only model currently active in the <tt>dtmslice</tt> scene. You can add new points pressing on [c]. The idea is to mark on the positions where you see the "neck" of a vesicle (what we called "crowns") in contact with the outer mitochondrial membrane.<br />
<br />
[[File:FhvCloseUp.png|thumb|center|500px|Clicking the crowns on screen]]<br />
<br />
The last marked point can be deleted by pressing [delete]. An arbitrary point can be deleted by clicking on it with the auxiliary mouse button. This will open a menu that includes the option of deleting the point (through Ctrl+X in Linux or Cmmd+X in Mac).<br />
<br />
At this stage you probably want to change the transparence of the depicted slice, so that you can control which objects have been already clicked below the depicted slide.<br />
<br />
[[File:FhvSelectedPoints.png|thumb|center|500px|Selection of points. Transparency of slide was set to 0.8]].<br />
<br />
When you are done, remember to save the model, using the menu options on active model or simply clicking on the disk icon in tomoslice.<br />
<br />
== Cropping particles ==<br />
<br />
Now we want to use the positions that we have marked to extract the subtomograms and format them as a [[data folder]]. The first thing we need is an estimation of the sidelength in pixels of each of the subtomograms. In [[dtmslice]] We can use the keys [1] and [2] to define two anchor points that appear as rombohedra. Clicking (with the right button) onto the black dashed that links the will show on screen both coordinates and the distance between them. All distances are reported in pixels of the non-binned tomogram: even if you are using a binned version, ''Dynamo'' keeps track of it.<br />
<br />
[[File:FhvMeasureDistance.png|thumb|center|500px|Measuring distances with [1] , [2] and right-click]].<br />
<br />
We will thus choose to create a datafolder with a cubic sidelength of 128 pixels (remember that the particles will be cropped in the ''unbinned'' tomogram). This will ensure that the crowns fit comfortably inside the physical box, even if our manual picking imposes an error of several pixels. If you were using, say, a ''thickness'' parameter of 10 pixles in <tt>dtmslice</tt>, you have to count with at least this inaccuracy in the location of the particles.<br />
<br />
Now, we check that the catalogued tomogram contains the model that we manually picked before:<br />
<br />
<nowiki>>> dcmodels fhv<br />
Volume 1 contains 1 models in total<br />
/Users/casdanie//fhv/tomograms/volume_1/models/mboxes.omd</nowiki><br />
<br />
In your system, instead of <tt>/Users/casdanie/</tt> you will see the path to the folder where you created the catalogue <tt>fhv</tt><br />
<br />
=== Creating a table ===<br />
We could just use the catalogue GUI to extract the particles, be it is also possible to proceed directly with the command line. We will use the <tt>dtcrop</tt> command, .which requires preparing a [[table]] with the information of the model. <br />
<br />
<nowiki>m = dread('/Users/casdanie/fhv/tomograms/volume_1/models/mboxes.omd');<br />
t = m.grepTable();</nowiki><br />
<br />
Here, you read the file into a model object (which we arbitrarily choose to call <tt>m</tt>), and then you use the <tt>grepTable</tt> method on this object to extract a variable into your workspace. We arbitrarily call it <tt>t</tt>. <br />
<br />
Note that we can extract models directly from the output of <tt>dcmodels</tt><br />
<nowiki>dcmodels fhv -i 1 -ws o;<br />
m = dread(o.files{1});<br />
t = m.grepTable();</nowiki><br />
<br />
i.e., we load the answer of <tt>dcmodels</tt> for volume index <tt>-i 1</tt> in the output variable <tt>o</tt>. Inside it, there is a field called <tt>'files'</tt> which contains a [[matlab cell array| cell array]] of files containing models. Then we read the first entry <tt>o.files{1}</tt>. <br />
<br />
Whichever way you create the table variable <tt>t</tt>, it is just a matrix with a row for each particle, and a summary of the information coded inside can be created through:<br />
<br />
<nowiki>>> dtinfo(t);<br />
<br />
size : 22 35 <br />
NaNs : 0<br />
<br />
COLUMN <br />
[ 2 ] marked for alignment: 22<br />
[ 3 ] included in average : 22<br />
[ 4-6 ] shifts : all zero<br />
[ 7-9 ] angles : all zero<br />
[ 10 ] cross correlation : min: 0.00 max: 0.00 mean: 0.00 std: 0.00<br />
[ 13 ] Fourier sampling : 1 (single tilt around y) <br />
[ 13 ] fsampling types : all of the same type <br />
[14-15] ytilt range : min:120.00 max:120.00<br />
[16-17] xtilt range : min:120.00 max:120.00<br />
[ 20 ] linked volumes : total 1 (labels: [1])<br />
[ 21 ] regions inside tomograms : total 1 (labels: [0])<br />
[ 22 ] user-defined classes: total 1 (labels: [0])<br />
[ 23 ] annotation types : total 1 (labels: [0])<br />
[24-26] spatial locations : initialized: 22<br />
[ 24 ] * x : min: 645.21 max: 1001.92 mean: 799.05 std: 109.44<br />
[ 25 ] * y : min: 23.78 max: 917.51 mean: 484.07 std: 271.28<br />
[ 26 ] * z : min: 198.00 max: 563.00 mean: 415.55 std: 114.03<br />
[ 31 ] original tags : total 1 (labels: [0])<br />
[ 32 ] compacted particles : total 1 (labels: [1])<br />
[ 34 ] references : total 1 (labels: [0])<br />
[ 35 ] subreferences : total 1 (labels: [0])<br />
[ 36 ] apix : Warning: column not available in this table<br />
[ 37 ] defocus : Warning: column not available in this table</nowiki><br />
<br />
=== Using <tt>dtcrop</tt> ===<br />
<br />
The simplest syntax of <tt>dtcrop</tt> requires passing the name of the tomogram from which we want to crop (syntax varies for [[ Particle extraction#Extract from different tomograms| cropping from multiple tomograms]]). We know that the file is <tt>crop.rec</tt>, and we could directly insert this name in the command. But a catalogued model already contains information about its source tomogram (inside its property <tt>cvolume</tt>), so that we can always track it back. We could then define a variable <tt>tomogramFile</tt> by accessing this information inside the model variable <tt>m</tt>:<br />
<br />
<tt>tomogramFile = m.cvolume.file();</tt><br />
<br />
if you get a warning at this stage, don't worry, the command still worked.<br />
<br />
You can then launch the cropping order:<br />
<br />
<tt>o = dtcrop(tomogramFile,t,'particlesData',128);</tt><br />
<br />
where you could add the <tt>mw</tt> flag to let ''Dynamo'' use several cores. In any case, for this number or particles the cropping should take some seconds. The last part of the final output into screen should look like this:<br />
<br />
<nowiki>21 [read_subtomogram] Volume has size 1285 956 786<br />
[read_subtomogram] Accessing subvolume x: 713:840; y: 339:466; z: 160:287 totalling ~ 16.0Mb<br />
Elapsed time is 0.191014 seconds.<br />
22 <br />
Total time invested in cropping: 7s<br />
[table_crop] Done extracting 20 particles<br />
from tomogram :"/Users/casdanie/dynamo/devmac/workplace/paris/crop.rec" <br />
destination folder :"particlesData" <br />
excluded particles : 2 <br />
<br />
[ok] table_crop </nowiki><br />
<br />
informing you that some of the particles where excluded, as they were probably too close to the boundary of the tomogram, given the sidelength we asked for. Inside the created data folder,<br />
you will find the table <tt>particlesData/crop.tbl</tt>, which only indexes the actually cropped particles.<br />
<br />
=== Creating an average ===<br />
<br />
The particles can now be averaged together. They have different orientations, but in this tomogram we only have a fraction of the membrane. <br />
<br />
<tt>oa = daverage('particlesData','t','particlesData/crop.tbl');</tt><br />
<br />
If you want to let ''Dynamo'' pass maps into Chimera, you have to inform ''Dynamo'' on the location of Chimera. For instance: <br />
<br />
<tt>dchimera -setPath /usr/local/bin/ </tt><br />
<br />
Please adapt it to the location of Chimera in your file system.<br />
<br />
To open your average in chimera<br />
<br />
<tt> dchimera(oa.average) </tt><br />
<br />
== Project to find membrane orientations==<br />
<br />
We first need to create files with the average and the table that we have till now in memory:<br />
<br />
<nowiki>dwrite(t,'raw.tbl');<br />
dwrite(oa.average,'rawTemplate.em');</nowiki><br />
<br />
We can create a project directly thorugh the command line:<br />
<tt>dcp.new('first','d','particlesData','template','rawTemplate.em','masks','default','t','particlesData/crop.tbl');</tt><br />
<br />
...and go directly to the panel of numerical parameters to create a project adapted for catching the general orientation of the membrane:<br />
[[File:FhvFirstParameters.png|thumb|center|500px| Numerical parameters for just membrane orientation]]<br />
<br />
Also, remember to visit the window for execution environment, and select the appropriate settings. If you are working with Matlab, select ''Matlab''(for one single core) or "Matlab_parfor" (if you are going to several cores). If working with the standalone, use the [[standalone]] option. In both cases, select the number of cores for alignment (green box in the picture below) and averaging (blue box) . <br />
<br />
[[File:FhvEnvironment.png|thumb|center|500px| Selecting the execution modus]]<br />
<br />
You can operate the project management operations from the GUI or the command line. First checking:<br />
<br />
<tt>dvcheck first</tt><br />
<br />
then unfolding <br />
<br />
<tt>dvunfold first</tt><br />
<br />
and finally running it by invoking the name of the execution script. If you are running in Matlab use:<br />
<br />
<tt>first</tt><br />
<br />
If you are running ''Dynamo'' in standalone modus, open a new tab (or window) of the terminal, load Dynamo (but don't run Dynamo) and type:<br />
<br />
<tt>./first.exe</tt><br />
<br />
After running the project, we check the created average:<br />
<br />
<tt>ddb first:a -v </tt><br />
<br />
This command asks the database (hence <tt>db</tt>) of the project <tt>first</tt> to look for the item of type <tt>average</tt>, by default accessing the last computed one. The flag <tt>-v</tt> sends the output of the database query to the depiction GUI called <tt>dview</tt>.<br />
<br />
[[File:FhvFirstAverage.png|thumb|center|500px|First average, no azimuthal rotation was tested. Shown in <tt> dview</tt> program]]<br />
<br />
Apparently, the project finds the membranes and centers the particles correctly. If you show the volume in Chimera (by clicking in the ''Export>Send to Chimera'' option in the <tt>dview</tt> window ).<br />
<br />
[[File:FhvFirstChimera.png|thumb|center|500px|First average, no azimuthal rotation was tested ]]<br />
<br />
We can check the individual particles:<br />
<br />
<tt>ddbrowse -d first:data -t first:rt </tt><br />
<br />
[[File:FhvDdbrowseFirstZ.png|thumb|center|700px| ddbrowse explores the aligned particles, each one represented as a projection. ]].<br />
<br />
== Aligning the axis of symmetry ==<br />
<br />
We would like to bring the axis of symmetry to the ''z''direction. This will create an intuitive separation between the two first euler angles that represent "change of orientation of the main axis", and the third one that represents "azimuthal rotation about the new axis". We want that azimuthal rotations occur about an axis of possible symmetry, which by convention is located along z. There are several ways to achieve this. A simple one is to create a model of the membrane/crown geometry, align our average to it (with <tt>dalign</tt>), and use the alignment parameters to create a new table. With this new table and average we will be able study possible symmetries with a new project.<br />
<br />
<br />
=== Creating a template aligned with z ===<br />
<br />
Geometrical shapes of many types can be created with <tt>dmask,dsphere,dcylinder,dshell</tt> and <tt>dtube</tt>, and combined with <tt>dynamo_mask_combine</tt>. However, there are also some available utilities to create templates representing frequently occurring membraneous geometries:<br />
<br />
<nowiki>mr = dpktomo.examples.motiveTypes.MembraneWithRod();<br />
mr.sidelength = 128;<br />
mr.rodRadius = 20; mr.rodShift = [0,0,10]; mr.rodHeight= 20;<br />
mr.getMask();<br />
template = mr.mask;</nowiki><br />
<br />
Note that you can check the list of properties inside the object <tt>mr</tt> just by typing it without the semicolon and pressing [Return]<br />
<br />
We can check the result with:<br />
<br />
<tt>mr.viewMask();</tt><br />
<br />
[[File:FhvTemplateZ.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Aligning the previous average along z ===<br />
The mask is white on black by definition. The tomogram data in this case represents bright protein on dark background, so that in this case we want to align the previously computed average to the mask of the <tt>mr</tt> object. <br />
<br />
The alignment does not need to scan azimuthal rotations, so that our 'inplane_range' will be 0. The 'cone_range' will be the full sphere, and we take and spacing of 30 degrees. As we are looking for coarse orientations, we can bin the two volumes twice (yielding volumes with a sidelength of 32 pixels). We will also "nail" the new center to the physical center of the box (parameter <tt>limm</tt> set to 1 below).<br />
<br />
Now we read the average computed by the project <tt>first</tt> into a workspace variable which we arbitrarily call <tt>previousAverage</tt><br />
<br />
<tt>ddb first:a -r previousAverage;</tt><br />
<br />
Here, the volume <tt>previousAverage</tt> will be treated as a "particle" to be aligned onto a ''template'', which in this case is the volume contained in <tt>mr.mask</tt>.<br />
<br />
<tt>sal = dalign(previousAverage,mr.mask,'cr',360,'cs',30,'ir',0,'dim',32,'limm',1,'lim',[10,10,10]); </tt><br />
<br />
The flags are the same parameters that would be used in an alignment project. You can check the shorthands with <tt>dvhelp</tt>.<br />
<br />
The output result <tt>sal</tt> contains many fields. We can, for instance directly visualize how the previous average looks like when transformed through the results of this <tt>dalign</tt> operation. For this, use the field <tt>aligned_particle</tt> in the output <tt>sal</tt>:<br />
<br />
<tt>dview(sal.aligned_particle);</tt><br />
<br />
<br />
<br />
[[File:ZAlignedAverage.png|thumb|center|700px| A simple template with rotational symmetry axis aligned along z]]<br />
<br />
=== Applying the z-alignment to the table ===<br />
<br />
We don't want just to get a z-oriented average: we also want to convert the table yield by the <tt>first</tt> project into a table that, when applied onto the original data, produces a correct, z-aligned average. For this, we use the alignment parameters coded inside the output <tt>sal</tt> (as field <tt>.Tp</tt>) and apply them onto the table.<br />
<br />
<tt>ddb first:rt -r tf</tt><br />
<br />
The variable <tt>tf </tt> contains now the original table, which is now rotated and shifted according with the rigid body transform <tt>Tp</tt> inside <tt>sal</tt><br />
<tt>tr = dynamo_table_rigid(tf,sal.Tp); </tt><br />
<br />
now the ''rectified ''table <tt>tr</tt> should contain the metadata that makes the particles "point upward". We use now the typical sanity check to assess that a table behave as expected: we apply it to the particles in its paired [[data folder]] and check the average:<br />
<br />
<tt>oz = daverage('first:data','t',tr,'fc',1); </tt><br />
<br />
whose results can be checked by:<br />
<br />
<tt>dview(oz); </tt><br />
<br />
[[File:FhvSanityCheckZ.png|thumb|center|700px| Particles aligned with rectified table <tt>tr</tt>]]<br />
<br />
<br />
We can visualize this operation by plotting a sketch with the orientation of the particles. We first show the results for the table computed in the project <tt>first</tt><br />
<br />
<nowiki>figure;<br />
dtplot('first:rt','m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchFirst.png|thumb|center|400px| sketch of orientations after computation of the the project <tt>first</tt>]]<br />
<br />
Here, the 'z' orientation computed for each particle is represented by the longest semiaxis. Note that with the original definition of the template, the 'z' points -approximately- along the direction of the 'z' in the tomogram. If we depict the same plot with the table we just computed:<br />
<br />
<nowiki>figure;<br />
dtplot(tr,'m','sketch','sketch_length',100,'sm',30);<br />
view(-230,30);axis equal;</nowiki><br />
<br />
[[File:FhvSketchZ.png|thumb|center|400px| sketch of orientations in the rectified table]]<br />
<br />
we see that now the 'z' axis in each particle is pointing orthogonally(-ish) to the membrane. A different quick check would be to show the projections of the particles along ''y''. Instead of using the GUI <tt>ddbrowse</tt>, we can create this depiction with a single command: <br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projy','*','align',1,'labels','on'); </tt><br />
<br />
[[File:fhvOrientedSlicesX.png|thumb|center|400px| full ''y''projection of the particles according to the orientations in the rectified table ]]<br />
<br />
Lets look at the ''z''-projection. In this case we project only the central slices around the crown and the aperture in the viral vesicle, chosing thus slices 60 to 70. if project the full particle, we won't distinguish any features.<br />
<br />
<tt>figure;dslices('particlesData','t',tr,'projz',60:80,'align',1,'labels','on');</tt><br />
<br />
[[File:fhvOrientedSlicesZ.png|thumb|center|400px| ''z''projection of the central slices in the rectified table ]]<br />
<br />
The depiction clearly shows the missing wedge of the particles. Incidentally, it also shows that averaging all the particles should produce an average that covers all the orientaions in the Fourier space.<br />
<br />
== Using the membrane to impart an orientation ==<br />
<br />
This section describes an '''alternative''' method to compute:<br />
* a rough orientation of the particles, and<br />
* a rough first template<br />
<br />
In this section we will use the membrane of the mithocondrion to assign a roughly orientation to each of the points in the table. This orientation will be defined by the normal of the closest point in the membrane. The membrane will be defined as a a triangulation, to be constructed based on a set of manually picked points.<br />
<br />
=== Pick membrane points===<br />
<br />
We open our tomogram in <tt>dtmslice</tt><br />
<br />
<tt>dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
or, if you are working with the [[standalone]] version, please prepend a "\" symbol:<br />
<br />
<tt>\dtmslice @{fhv}1 -prebin 1</tt><br />
<br />
In this variant of the syntax of <tt>dtmslice</tt>, the string <tt>@{fhv}1</tt> just means "tomogram number 1 inside catalogue <tt>fhv</tt>. On the opened scene, we will use a [[Montage viewer for membrane picking | montage ]] to manually click on a set of membrane points.<br />
<br />
'''Note! (08/06/21) - the montage viewer was recently updated and no longer provides the ability to easily add points, please define your surface model on sequential z-slices in dtmslice directly by annotating points every 20 or so slices.'''<br />
<br />
[[ File:FhvMontage.png |thumb|center|400px| Opening a montage view]]<br />
<br />
By default, this <tt>montage</tt> represents slices taken orthogonally to ''z'' every 20 pixels of the unbinned tomogram. This can be changed in the settings ''before'' opening the montage GUI. Now, we can create a model of type <tt>Surface</tt>, which we will use to store a set of points in the membrane to be picked manually (or semiautomatically). To do this, go to ''Model pool> Create new model in pool (choose type) > surface ''<br />
<br />
[[ File:FhvCreateSurfaceModel.png |thumb|center|400px| Creating a surface model]]<br />
<br />
You need to click on the point switcher (the control in the toolbar with a ''c'' and an ''i'') to allow entering points. The basic controls are: <br />
* Mouse click to enter a point. <br />
* <tt>d</tt> to delete a point.<br />
* <tt>i</tt> to insert a point.<br />
<br />
[[File:FhvMontageActiveClick.png |thumb|center|400px| Point switcher must be toggle on to pick points into the model]]<br />
<br />
Use the ''>>'' button in the bottom of the GUI to go to next set of orthoslices.<br />
<br />
[[File:FhvMontageControls.png |thumb|center|400px| Controls of the montage GUI]]<br />
<br />
Note that the points that you click in the Montage GUI are assigned to a model in the pool, and as such, they appear automatically in the dtmslice GUI<br />
[[ File:FhvTmsliceUpdates.png |thumb|center|600px| Points clicked in the montage GUI reflect in the <tt>tmslice</tt> GUI]]<br />
<br />
You also need <tt>shift + C </tt> in order to place a center inside the mithocondrion. This point is just used to tell ''Dynamo'' what is the inside and what is the outside face of the surface that we are building.<br />
<br />
[[File:FhvSurfaceMarkCenter.png |thumb|center|400px| <tt>shift + C </tt> marks an arbitrary interior point]]<br />
<br />
==== Automatic detection ====<br />
<br />
Once you have clicked points on a level, you can try and check if the automatic tracker can locate them in the next one. To do so, activate the next slice (i.e., secondary click it, so that its frame becomes red), and press <tt>o</tt>. You can edit the points with <tt>d</tt> to delete the closest point, <tt>m</tt> to move the closest point to the position currently under the cursor, or <tt>l</tt> to delete all the points detected in the slice... in such a case, you probably want to check the tracking parameters before using <tt>o</tt> again!<br />
<br />
=== Create a surface ===<br />
<br />
Now, we want to convert the points that we have introduced into a triangulation. This is a part of the [[Membrane Models#Workflow description| workflow]] used to crop particles from [[membrane models]], so we open the workflow GUI for this model in the ''Active Model'' menu tab. <br />
<br />
[[ File:FhvSurfaceWorkflowCreate.png |thumb|center|400px| Invoking the workflow editor of active model ]]<br />
<br />
We can first check the points currently contained in the model, by auxiliary clicking on the first viewing option and choosing the ''User points'' option. They will be depicted in a graphic window which will update with the new graphical elements that we depict there.<br />
<br />
[[ File:FhvSurfacePlotPoints.png|thumb|center|600px| Showing original points in the model workflow GUI ]]<br />
<br />
We first need to create a spline interpolation on each of the levels of our point cloud.<br />
<br />
[[ File:FhvSurfaceControlPoints.png|thumb|center|600px| Control points are created by spline interpolation. ]]<br />
<br />
Now we define a triangulation on this control points.<br />
<br />
[[ File:FhvSurfaceCreateTriangulation.png |thumb|center|600px| Creating a triangulation on the control points ]]<br />
<br />
Which can be subdivided to get a smoother appearance, and also a more accurate representation of the geometry.<br />
<br />
[[File:FhvSurfaceSubdivisionMinimum.png |thumb|center|400px| A single round of subdivision smoothens the appearance of the lattice ]]<br />
<br />
Then, we can change the name of the model for clarity, and then save it into the catalogue (with the disk icon in <tt>tmslice</tt> or the ''Save into catalogue'' options under the ''Active model'' menu)<br />
[[ File:FhvSurfaceSaveModel.png |thumb|center|600px| Renaming a saving a model into the catalogue ]]<br />
<br />
=== Use a surface to impart orientation ===<br />
<br />
We bring the model into our workspace. We search for it in the catalogue:<br />
<br />
<tt>dcmodels fhv -nc mySurface -ws output</tt><br />
<br />
which will prompt the response <br />
<br />
<nowiki>Volume 1. Matching models: 1 (total: 1)<br />
/Users/casdanie/dynamo/examples/fhvParis/fhv/tomograms/volume_1/models/mySurface.omd</nowiki><br />
<br />
now, we can read the model file into our memory space to operate with it:<br />
<br />
<tt>m = dread(output.files{1});</tt><br />
<br />
Now the model <tt>m</tt> is in our space of memory and can be used to impart an orientation to the points in our table.<br />
<br />
<tt>tOrientedBySurface = dpktbl.triangulation.fillTable(m,'raw.tbl');</tt><br />
<br />
In this table, orientations are orthogonal to the membrane (or more precisely, orthogonal to the closest triangle in the mesh that represents the membrane)<br />
<br />
<tt> figure;dslices('particlesData','t', tOrientedBySurface,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPreflipping.png|thumb|center|400px| Slices along the xz plane as seen by the table]]<br />
<br />
<tt>tConsistent = dynamo_table_flip_normals(tOrientedBySurface,'center',m.center)</tt><br />
<br />
We can check the effect of this table on the particles through:<br />
<br />
<tt>figure;dslices('particlesData','t', tConsistent,'projy','c20','align',1,'labels','on');</tt><br />
<br />
[[ File:FhvMembraneSlicesPostflipping.png |thumb|center|400px| Slices along the xz plane as seen by the table, after using the center point to establish the interior side of the surface ]]<br />
<br />
To get a better graphical impression, we can depict our surface (contained in the model <tt>m</tt>) with:<br />
<br />
<tt>m.ezplot</tt> <br />
<br />
and choose the <tt>Surface</tt>option<br />
<br />
[[ File:FhvMembraneSurfaceEzplot.png |thumb|center|400px| Easy way to get all defined plots of a model object. ]]<br />
<br />
Then, we just plot the positions of the particles on the same graphical figure:<br />
<br />
<tt>dtplot(tConsistent,'m','sketch','sketch_length',100,'sm',30);</tt><br />
<br />
[[ File:FhvMembraneSketchDtplot.png |thumb|center|400px| <tt>dtplot</tt> directs its output on the last active figure ]]<br />
<br />
==== Management of the missing wedge ====<br />
<br />
When we create a table considering only the orientations of the points with relation to a membrane, there is no particular preference for rotations of the particles about the normal direction (i.e., azimuthal directions). The <tt>narot</tt> angle in this table is initialized to zero<br />
<br />
Thus, take into account that an average created on these particles will have a strong missing wedge, especially in our case, where we have a preferential direction. This can be checked by averaging the particles against this table:<br />
<br />
<tt>oa=daverage('particlesData','t', tConsistent,'fc',1);</tt><br />
Here, we ask (<tt>'fc',1</tt>) to run a Fourier compensation step, so that we can check in the output <tt>oa</tt> the property <tt>fweight</tt>, which contains the number of times a Fourier component is represented in the average. Bright values mean that the Fourier component is present in many particles (yielding thus an average of better quality for the involved frequencies).<br />
<br />
<tt>dview(oa.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingWedge.png|thumb|center|400px| Fourier component presence without azimuthal randomization ]]<br />
<br />
We can atenuate this effect by randomizing the rotational angle:<br />
<br />
<tt>tConsistentRandomized = dynamo_table_randomize_azimuth( tConsistent);</tt><br />
<br />
and averaging again.<br />
<br />
<tt>oaRandomized=daverage('particlesData','t', tConsistentRandomized,'fc',1);</tt><br />
<br />
Now, the <tt>fweight</tt> map doesn't show such a strongly preferential orientation:<br />
<br />
<tt>dview(oaRandomized.fweight);</tt><br />
<br />
[[ File:FhvMembraneMissingRandomized.png |thumb|center|400px| Fourier component presence with azimuthal randomization ]]<br />
<br />
The effect in direct space can be shown by depicting both averages (with and without randomization side to side)<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview({oa.average,oaRandomized.average});</tt><br />
<br />
[[ File:FhvMembraneCompareWedges.png |thumb|center|400px| The map on the left shows a worse definition and clear traces of missing Fourier components ]]<br />
<br />
== Project for rotational alignment ==<br />
<br />
We have now a template and a coherently defined metadata (i.e. table) that align the particles coarsely along the right direction. We write them into disk: <br />
<nowiki>Insert non-formatted text here</nowiki><br />
If you did not create a membrane model, use the results of the alignment project:<br />
<br />
<nowiki>dwrite(tr,'zOriented.tbl');<br />
dwrite(oz.average,'zOriented.em');</nowiki><br />
<br />
If you did create a membrane model, used the corresponding table and average:<br />
<br />
<nowiki>dwrite(tConsistentRandomized,'zOriented.tbl');<br />
dwrite(oaRandomized.average,'zOriented.em');</nowiki><br />
<br />
<br />
and create a project with these files. Note that the [[data folder]] remains unchanged.<br />
<br />
<tt>dcp.new('zOriented','d','particlesData','template','zOriented.em','masks','default','t','zOriented.tbl');</tt><br />
<br />
In this project, we will ask for a symmetrization 'c57'.. this is just to simulate fully rotational symmetry. We are not assuming that this symmetry is physical: we just want to force any possible symmetry axis in the data along the ''z'' axis. We also keep binning twice the particles, in order to get the computation times short.<br />
<br />
[[File:FhvZOrientedParameters.png|thumb|center|500px| Alignment parameters in <tt>zOriented</tt>]]<br />
<br />
The result shown an incipient formation of the crown... however, the presence of suspicious striations in a preferential direction might be an indication of the effect of the missing wedge.<br />
<br />
[[File:FhvZOrientedMapview.png|thumb|center|500px| Result in <tt>zOriented</tt>]]<br />
<br />
== Project with rotational randomization ==<br />
<br />
We can create a template that does not have a prefererential orintation, by just randomizing the azimuthal rotations in our table, We extract the last refined table into the variable <tt>tns</tt><br />
<br />
<tt>ddb zOriented:rt -r tns </tt><br />
<br />
and impose a perturbation that keeps all other parameters constant but randomly rotates the particles around their axis<br />
<tt>tr = dynamo_table_perturbation(tns,'pshift',0,'paxis',0,'pnarot',360);</tt><br />
<br />
We can use this table to average again the particles and check the average.<br />
<nowiki>oar = daverage('particlesData','t',tr,'fc',1);<br />
dview(oar)</nowiki><br />
<br />
Now, we can write these files in disk and create a project:<br />
<nowiki>dwrite(oar.average,'randomizedAverage.em');<br />
dwrite(tr,'randomized.tbl');<br />
dcp.new('zRandomized','d','particlesData','template','randomizedAverage.em','masks','default','t','randomized.tbl');</nowiki><br />
<br />
[[File:FHV zRandomized Parameters.png|thumb|center|600px| Alignment parameters in <tt>zRandomized</tt> project]]<br />
<br />
After running this projects, the presence of a crown-like is very, very faintly hinted in mapview:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
[[File:FhvRandomizedFaintHintsMapview.png|thumb|center|500px| Result from <tt>zRandomized</tt> project]]<br />
<br />
and barely identifialble in Chimera:<br />
<br />
[[File:FhvZRandomizedChimera.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
== Project for localized alignment ==<br />
<br />
We don't see clearly what is happening on the area of interest... On one hand we know that we are using a very reduced number of particles ( which on top of it are too similarly oriented). But there is still some room to improve in the approach: we can focus the refinement in the area of interest.<br />
<br />
=== Creating a tight mask ===<br />
We want to create a mask that encloses the area of interest. To do this, we open the average in <tt>dmapview</tt>:<br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('zRandomized:a')</tt><br />
<br />
We tune the browser for viewing along ''y'', and we select one of the central slices. We can use a thickness of several pixels to have a better view.<br />
<br />
Then, we select the area that we want to use as seed in this plane to create a revolution solid. We press on <tt>[shift] + [s] </tt> on the selected slide, letting a new window pop up. In this window, you handdraw an area on the XZ plane (main click to start drawing, auxiliary click to stop). ''Dynamo'' will rotate the region about the ''z'' axis, creating a revolution solid into the <tt>file temp_drawn_revolution_mask.em</tt><br />
<br />
[[File:fhvHandDrawn.png|thumb|center|500px| Drawing a revolution mask.]]<br />
<br />
We have to change the default name given by ''Dynamo'' to a relevant one:<br />
<br />
<tt>!mv temp_drawn_revolution_mask.em teethMask.em</tt><br />
<br />
and now continue the previous project by transferring the results of <tt>zRandomized</tt> to a fresh project (lets call it <tt>localized</tt>)<br />
<br />
<tt>dynamo_vpr_branch zRandomized localized -b 1 -noise 0</tt><br />
<br />
where we insert the tight mask <tt>teethMask.em</tt> that we just created as alignment mask of the project. Note that we do '''not''' want impose any symmetry in our parameters:<br />
[[File:FhvParametersLocalized.png|thumb|center|500px| Alignment parameters in <tt>localized</tt> project]]<br />
<br />
Let's inspect the output in <tt>dview</tt><br />
[[File:fhvLocalizedDview.png|thumb|center|800px| Last average in project <tt>localized</tt> as shown by <tt>dview</tt>]]<br />
There are different things that we like in this average<br />
# The localized masking worked well, producing a clearer insight into the region of relevance<br />
# Although we are using signal only inside the mask, the material outside of the mask does '''not'' become smeared. This is clear mark that the alignment is real and non artifactual. That's an important point to check when we use small masks.<br />
# The symmetry in the area becomes apparent on the bare eye. They can be distinguished clearly with mapview and even in Chimera.<br />
<br />
[[File:FhvLocalizedMapview.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
[[File:FhvLocalizedChimera.png|thumb|center|500px| Last average in project <tt>localized</tt> as shown by <tt>mapview</tt>]]<br />
<br />
=== Testing the symmetry ===<br />
<br />
We can check for different symmetry operators to confirm the presence of symmetry. <br />
<br />
==== Basic command ====<br />
<br />
The basic command for this task is <tt>dynamo_symmetry_scan</tt>. In its fundamental syntax, you need to state the type of symmetry operator to be tested, and a set of operator.<br />
<br />
<tt> stm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','nfig',3); </tt><br />
<br />
[[File:FhvSymmetryGlobal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
With a more localized mask:<br />
<tt> slm = dynamo_symmetry_scan('localized:a','c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
[[File:FhvSymmetryLocal.png|thumb|center|500px| Symmetry detected for different masks]]<br />
<br />
==== Interpretation of results ====<br />
<br />
If you got the results above, you might have jumped to the conclusion that they already constitute an objective hint at a real C12 symmetry. This is, however, not true. The fact is that, with a slightly different picking of particles, you could have obtained the profiles in these pictures:<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FhvSymmetryGlobalWrong.png|thumb|upright|400px| Symmetry merit plot (global mask) ]]<br />
| [[File:FhvSymmetryLocalWrong.png|thumb|upright|400px| Symmetry merit plot (local mask) ]]<br />
|}<br />
<br />
These images represent the result of the two symmetry-exploring commands described above obtained by a different user. . Surprisingly, they point at possible symmetry orders of 11 and 13.<br />
So, what is going on here? How is it possible that two different users that operate exactly the same walkthrough get contradictory symmetry estimations? The reason is that different users will pick manually slightly different particle centers, and this have a direct impact on the symmetry estimation. In short, the problem is that the density map <tt>localized:a</tt> is not garanteed to have its possible symmetry axis located along the center of the box, and this misleads <tt> dynamo_symmetry_scan</tt>. This can bechecked by opening the average in <tt>mapview</tt> and overlaying the file <tt>'teethMask.em'</tt> on it.<br />
<br />
[[File:FhvSymmetryDecentered.png|thumb|center|500px| Mapview map n file <tt>localized:a </tt> overlayed with file <tt>'teethMask.em'</tt>]]<br />
<br />
In this non-centered density map, any symmetry determination can only be an artifact. To solve this, we create a centered version: <br />
<br />
<nowiki>ddb zOriented:a -r zo % extract average of zOriented as the ws variable zo<br />
ddb localized:a –r localizedAverage % extract average of localized as the ws variable localized Average<br />
sal = dalign(localizedAverage,zo,'cr',0,'cs',30,'ir',0,'dim',64,'limm',1,'lim',[20,20,20]); % aligns localizedAverage against zo (like that localizedAverage is centered)<br />
centerLocalized = sal.aligned_particle;<br />
dview(centerLocalized); </nowiki><br />
<br />
[[File:FhvSymmetryRecenteredView.png|thumb|center|500px| Recentered average (x and y views) ]]<br />
<br />
We can scan again the symmetry operators on the recentered volume.<br />
<br />
<tt> slmCentered = dynamo_symmetry_scan(centerLocalized,'c','order',3:15,'type','pearson','mask','teethMask.em','nfig',4);</tt><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVsymmetryRecenteringOldMaskSlice.png|thumb|upright|400px| <tt>'teethMask.em'</tt> overlayed on the recentered average ]]<br />
| [[File:FHVsymmetryRecenteringOldMaskPlot.png|thumb|upright|400px| Symmetry merit plot after recentering using <tt>'teethMask.em'</tt> ]]<br />
|}<br />
<br />
Now the peaks located that at 6 and 12 are trustworthy. You can even create a new mask on the recentered average, carefully excluding intensities from the central ring structure to completely rule out any artefact. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:FHVSymmetryAdhocMaskOverlay.png|thumb|upright|400px| Mask constructed ''ad hoc'' on the recentered average ]]<br />
| [[File:FHVSymmetryAdhocMaskPlot.png |thumb|upright|400px| Symmetry merit plot using ''ad hoc'' constructed mask]]<br />
|}<br />
<br />
== Subboxing ==<br />
<br />
The ''subboxing'' technique consists in redefining the area of interest inside a previously defined average. In this example, we have use the full crowns as the subject of alignment and averaging. This has allowed us to use the whole signal carried by the full crown on each of the particles to drive the alignment robustly. This approach, however, will not allow us to identify heterogeneity and flexibility inside the individual crowns. Additionally, the possible hetereogeneity inside the crown will decrease the quality of the alignment when the crown is treated as a whole. <br />
<br />
With the ''subboxing'' technique, we can use our previous results to setup an approach that centers the alignment on the individual teeth. Each one the particles in our new data set will be a ''subbox'' (a tooth) extracted from the previos ''box'' (the crown) using the alignment parametes for the full boxes gained by the projects that we have ran so far. <br />
<br />
=== Defining a subboxed subunit ===<br />
<br />
First, we need to define the location that we want to subbox. We use <tt>mapview</tt><br />
<br />
<tt>dpkdev.legacy.dynamo_mapview('localized:a') </tt><br />
<br />
to show the last average in the project <tt>localized</tt><br />
<br />
[[File:FhvSubboxSelectZ.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt>]]<br />
<br />
You probably want to explore several views in mapviews (''x'/'y' and 'z') to make sure you (roughtly) hit the center of the subunit of interest. Guiding yourself only for a view on ''z'' can be misleading.<br />
<br />
[[File:FhvSubboxSelectY.png|thumb|center|500px| Selecting the center of the tooth subunit with <tt>mapview</tt> on the ''y'' view]].<br />
<br />
As we are in this browser, we can check the size of the subunit that we want to ''subbox'', using the two markers C and N. This will be useful later when we want to actually crop physical particles using <tt>dtcrop</tt>, when we will be required to input a sidelength for the subboxed data folder.<br />
<br />
[[File:FhvSubboxSelectDistance.png|thumb|center|500px| Measuring distances in mapview]].<br />
<br />
We define a variable to contain the position that we read in mapview for the blue (North) anchor. <br />
<tt>rSubunitFromCenter = [88,80,53 ] - [64,64,64]; </tt><br />
We subtract the half sidelength of the full box, because the next command will need the position of the asymmetrical unit expressed in relation to center of the box.<br />
<br />
=== Creating a subboxing table ===<br />
<br />
We extract the last refined table: <br />
<tt>ddb localized:rt -r t</tt><br />
<br />
and define positions related by C12 symmetry along the axis<br />
<tt>ts = dynamo_subboxing_table(t,rSubunitFromCenter,'sym','c12');</tt><br />
<br />
The new subboxed table <tt>ts</tt> will have 12 times as many rows as the original one. The system of reference on each particle will point ''z'' in the direction of its original box, but the ''x'' and ''y'' orientations of each tooth will be symmetrically related in the same crown. We can depict this geometrical relationship by plotting a sketch of all the particles in the table:<br />
<br />
<tt>figure;dtplot(ts,'m','sketch','sketch_length',100,'sm',30);view(-151,12);axis equal;</tt><br />
<br />
[[File:FhvSubboxSketch.png|thumb|center|500px| Sketch of orientations of the new subboxed table <tt>ts</tt>]].<br />
<br />
=== Creating a subboxed data folder ===<br />
<br />
Now we have all we need to go back to the original tomogram and crop the subboxed particles into a new data folder using the sidelength that we checked with <tt>mapview</tt>:<br />
<br />
<tt>dtcrop('crop.rec',ts,'subboxData',32);</tt><br />
<br />
We can run our typical sanity check to ensure that everything ran correctly<br />
<br />
<tt>osb = daverage('subboxData','t',ts,'fc',1);</tt><br />
<br />
To visualize it you can write: <br />
<br />
<tt>dview(osb.average);</tt><br />
<br />
[[File:FhvSubboxAverage.png|thumb|center|700px| Density map of the average of subboxed teeth]].<br />
<br />
As it looks like we expected, we just write it into a file, which we will use to define a new project.<br />
<br />
<tt>dwrite(osb.average,'subboxRaw.em');</tt><br />
<br />
=== Defining masks ===<br />
<br />
There are some tools to quickly format masks for project. If we want to check a reasonable radius for our mask, we can use the <tt>dsphere</tt> command and overlay the created mask on the template <br />
<nowiki>cs = dynamo_sphere(10,32);<br />
figure;dslices(osb.average,'y','ov',cs,'ovas','mask');<br />
figure;dslices(osb.average,'ov',cs,'ovas','mask'); </nowiki><br />
<br />
[[File:FhvSubboxOverlay.png|thumb|center|700px| Testing the radius of created masks by overlay on template]].<br />
In fact, this mask is probably too risky: we need a minimum of signal to drive the alignment <br />
<br />
<tt>dwrite(cs,'maskTooth32.em');</tt><br />
<br />
=== Subboxing project ===<br />
<br />
We create the project through:<br />
<tt>dcp.new('subboxBig','d','subboxData','template','subboxRaw.em','masks','default','t','subboxData/crop.tbl','show',0);</tt><br />
<br />
In this case, we use <tt>'show' 0</tt> would suppress the [[dcm]] GUI. We use typically this option when we want to enter project parameters through the command line. This is exactly equivalent to entering parameters through the GUI. <br />
<br />
<nowiki>dvput subboxBig mask maskTooth32.em<br />
dvput subboxBig ite_r1 3<br />
dvput subboxBig cr_r1 4<br />
dvput subboxBig cs_r1 2<br />
dvput subboxBig ir_r1 4<br />
dvput subboxBig is_r1 2<br />
dvput subboxBig rf_r1 2<br />
dvput subboxBig rff_r1 2<br />
dvput subboxBig dim_r1 32<br />
dvput subboxBig lim_r1 [4,4,4]<br />
dvput subboxBig limm_r1 1 </nowiki><br />
<br />
A list of the names of the parameters can be displayed through the command <tt>dvhelp</tt>. The computing environment is set using the <tt>destination</tt> parameter, shortnamed <tt>dst</tt>. To run it in matlab use:<br />
<br />
<tt> dvput -dst matlab_parfor</tt><br />
<br />
For standalone running, use:<br />
<br />
<tt> dvput -dst standalone</tt><br />
<br />
After running the project, we can check the effect of the independent refinement of the "tooth" units:<br />
<tt>dpkdev.legacy.dynamo_mapview('subboxBig:a:ite=[0,3]')</tt><br />
<br />
[[File:FhvSubboxRefimentMapview.png|thumb|center|800px| Slice-to-slice comparison of initial template and final average of the subboxing project]].<br />
<br />
or send the average to Chimera through:<br />
<br />
<tt>ddb subboxBig:a -c </tt><br />
<br />
[[File:FhvSubboxRefimentChimera.png|thumb|center|600px| Density map of the average of subboxed teeth after refinement]].<br />
<br />
== Visualization ==<br />
<br />
You can continue working on this data in the [[Walkthrough on creation of 3d scenes]], where you will learn to depict tomgraphic slices, place templates in table positions, and depict graphical elements representing model geometries.</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021)&diff=4184Walkthrough on command line based tilt series alignment (EMBO2021)2021-09-06T15:10:25Z<p>Stefano Scaramuzza: /* Basic data */</p>
<hr />
<div>Dynamo includes a package for automated alignment and reconstruction of tilt series. This walkthrough guides users through a series of steps on how to use it in command line based mode. The [[Walkthrough on GUI based tilt series alignment | GUI based alignment walkthrough]] shows how to operate this procedure through the GUI. Here, we go through each command one by one.<br />
<br />
= Create the workflow =<br />
The following commands create a new alignment workflow in a separate folder. Make sure ''Dynamo'' is loaded and running (as explained in the [[EMBO_workshop_2021 | workshop page]]) and type the following commands into the console. Note: The workshop setup does not allow you to copy paste commands from your desktop onto the workshop server. You need to open a browser within the workshop server and copy paste the commands from there.<br />
<br />
<nowiki>name = 'hivCommandLine';<br />
folder = 'workflows';<br />
u = dtsa(name,'--nogui','-path',folder,'fp',1); </nowiki><br />
<br />
Type <tt>whos</tt> to see a list of the variables that you just created with these commands.<br />
<br />
= Entering the data =<br />
<br />
The variable <tt>u</tt> is an object that contains several ''fields'' to interact with the workflow. The different fields can be found through autocompletion by pressing the enter key (see image below). In the following, these different fields will be used to pass our inputs to the workflow.<br />
<br />
[[ File:alignCommandLine_autocompletionOfUserObjectEMBO2021.png |thumb|center| 700px|autocompletion of user object to access different fields in standalone version]]<br />
<br />
== Basic data==<br />
Basic data can be entered using the ''enter'' field. Here we use a binned version of the first tilt series from the EMPIAR entry 10164, depicting a set of virus like particles (VLP), which is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory). The file of the tilt series and the angles are passed to the workflow:<br />
<br />
<nowiki>u.enter.tiltSeries('b001ts001.mrc');<br />
u.enter.tiltAngles(-57:3:60); </nowiki><br />
<br />
If necessary (not in this tutorial), you can use the ''discardedTiltIndices'' field to reject tilts that are of too bad quality using the following command (e.g., tilt 1 and 40):<br />
<tt>u.enter.discardedTiltIndices([1,40]); </tt><br />
<br />
== Acquisition settings==<br />
Here you can enter parameters such as <tt>Cs</tt>, nominal defocus, etc. For this tutorial we just provide the pixel size in Angstroms:<br />
<tt>u.enter.settingAcquisition.apix(2.7);</tt><br />
<br />
== Computation settings==<br />
We enable the use of parallel cores<br />
<tt>u.enter.settingComputing.parallelCPUUse(1); </tt><br />
This will be using mainly for gold bead detection (cc computation) and reconstruction.<br />
<br />
To use all available physical cores use:<br />
<tt>u.enter.settingComputing.cpus('*');</tt><br />
<br />
== Detection settings==<br />
These here are the actual design decisions when running an alignment workflow. The gold bead radius has to be known beforehand ( see [[Walkthrough on GUI based tilt series alignment | GUI based alignment walkthrough]] on how to measure it). Binning is used internally only for quick and robust bead detection.<br />
<br />
<nowiki>u.enter.settingDetection.detectionBinningFactor(1);<br />
u.enter.settingDetection.beadRadius(16);<br />
u.enter.settingDetection.maskRadius(28);<br />
u.enter.templateSidelength(64);</nowiki><br />
<br />
== Changing generic parameters ==<br />
You can find all parameters of the individuals steps using autocompletion on the <tt>area</tt> field and then by adding <tt>step</tt> in the end. For example, the residual threshold of the tilt gap filler can be changed as follows:<br />
<tt>u.area.indexing.step.tiltGapFiller.parameterSet.residualsThreshold(8);</tt><br />
<br />
= Running the workflow =<br />
After setting the input parameters, the complete workflow can be run using the following command (we skip the CTF correction here). It should take about 5 minutes to run.<br />
<tt>u.run.all('noctf',1);</tt><br />
<br />
== Checking the results ==<br />
After completion, the <tt>view</tt> field allows you to check different output items for diagnose, such as the tilt lines:<br />
<br />
<tt>u.view.tiltLines.unaligned();</tt><br />
<br />
[[ File:alignCommandLine_viewOfTiltLines.png |thumb|center| 600px|view of tilt lines]]<br />
<br />
or the different reconstructions <br />
<br />
<nowiki>u.view.reconstruction.binnedWBP();<br />
u.view.reconstruction.binnedSIRT();</nowiki><br />
<br />
or the aligned stacks themselves.<br />
<br />
<tt>u.view.stack.alignedFull();</tt><br />
<br />
You can also use the autocompletion of the <tt>info</tt> field (as shown in the beginning) to find and display different logs:<br />
<br />
<nowiki>>> u.info.fit;<br />
------------------------------------------------------------<br />
File with info:<br />
workflows/hivCommandLine.AWF/info/fitting.doc<br />
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <br />
rms: 1.41<br />
psi: 85.20<br />
fit modus: normal<br />
total markers: 79<br />
total observations: 2702<br />
fit created at: 11-Aug-2021 10:56:24<br />
------------------------------------------------------------</nowiki><br />
<br />
<nowiki>>> u.info.markers;<br />
------------------------------------------------------------<br />
Info on working markers [used during detection, indexing refinement]<br />
... item saved and in memory<br />
Item in memory:<br />
Number of shapes : 79<br />
Number of frames : 40<br />
Number of observations : 2702 (out of 3160)<br />
Number of empty frames : 1<br />
Number of complete frames : 0<br />
Number of complete shapes : 0<br />
<br />
Item in disk:<br />
Markers in disk and currently in memory contain equal coordinates<br />
<br />
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - <br />
<br />
Info on final markers [used from alignment step]<br />
... item saved and in memory<br />
Item in memory:<br />
Number of shapes : 79<br />
Number of frames : 40<br />
Number of observations : 2702 (out of 3160)<br />
Number of empty frames : 1<br />
Number of complete frames : 0<br />
Number of complete shapes : 0<br />
<br />
Item in disk:<br />
Markers in disk and currently in memory contain equal coordinates<br />
------------------------------------------------------------</nowiki><br />
<br />
If you are following this tutorial during a ''Dynamo'' workshop, the exercise ends here.<br />
<br />
= On the fly reconstruction of particles=<br />
<br />
In this section we show how to avoid having to reconstruct a '''full''' tomogram at '''full''' resolution for the case where only a set of subtomograms are need. For this exercise we will use the gold beads as proxies that represent any particles of interest. We will first locate a few of them in the '''binned''' tomogram, and then use those coordinates to directly reconstruct the subtomograms in full resolution and skipping the need to reconstruct the whole full sized tomogram..<br />
<br />
== Define coordinates in the binned tomogram==<br />
<br />
First, we open the binned reconstruction tht we created before using the visualization tool <tt>dtmslice</tt> and adjust the contrast if necessary (arrow):<br />
<br />
<tt>dtmslice 'workflows/hivCommandLine.AWF/reconstruction/binnedReconstructionWBP.mrc';</tt><br />
<br />
[[ File:alignCommandLine_dtmsliceOfBinnedReconstruction.png |thumb|center| 600px|dtmslice of binned reconstruction]]<br />
[[ File:alignCommandLine_dtmsliceAfterContrastCorrection.png |thumb|center| 600px|dtmslice after contrast correction]]<br />
To manually define the particle coordinates, we first open a new model of type <tt>general</tt>:<br />
[[ File:alignCommandLine_createANewModel.png |thumb|center| 600px|create a new model]]<br />
To quickly find the 3d positions of a set of particles, we use a projection of the tomogram:<br />
[[ File:alignCommandLine_showAFullProjection.png |thumb|center| 600px|show a full projection]]<br />
[[ File:alignCommandLine_fullProjectionOfBinnedReconstruction.png |thumb|center| 600px|full projection of binned reconstruction]]<br />
Right click on the (x,y) position of a particle opens two small windows showing the x-z and y-z planes:<br />
[[ File:alignCommandLine_rightClickOnAPoint.png |thumb|center| 600px|right click on a point]]<br />
Use the y-z depiction to left clicking on the marker. After clicking, the marker coordinates are saved in the model:<br />
[[ File:alignCommandLine_xViewOfClickedPoint.png |thumb|center| 600px|x view of clicked point]]<br />
[[ File:alignCommandLine_mainClickToFixThe3dCoordinates.png |thumb|center| 600px|main click to fix the 3d coordinates]]<br />
Go back to the projection view and repeat the procedure until you have a few more markers (but no more than ten in this exercise). The x-z and y-z planes will be updated automatically when clicking on new markers on the projection and do not need to be closed every time. When you are done, you can close all the plane views and also the projection view.<br />
[[ File:alignCommandLine_clickAroundTenMarkers.png |thumb|center| 600px|click around ten markers]]<br />
To save the coordinates we first export them into the Matlab workspace in table format:<br />
[[ File:alignCommandLine_exportATableIntoTheMatlabWorkspace.png |thumb|center| 600px|export a table into the matlab workspace]]<br />
The Matlab shell should report the variable name used to store the table:<br />
[[ File:alignCommandLine_theCommandLineShowsTheVariableNameAssignedToTheCreatedTable.png |thumb|center| 400px|the command line shows the variable name assigned to the created table]]<br />
<br />
Now we can save the table with the coordinates as a file in the current directory (this table will be used to crop the subtomograms in the next steps):<br />
<tt>dwrite(temp_table,'binnedGoldbeads.tbl');</tt><br />
<br />
=== Cropping the gold bead particles ===<br />
<br />
We can now crop the particles using the table file that we generated before:<br />
<nowiki>tomo = 'workflows/hivCommandLine.AWF/reconstruction/binnedReconstructionWBP.mrc';<br />
tbl = 'binnedGoldbeads.tbl';<br />
targetBinnedParticles = 'binnedGoldBeads.Data';<br />
sidelengthBinned = 32;<br />
o = dtcrop(tomo,tbl, targetBinnedParticles, sidelengthBinned);</nowiki><br />
<br />
To monitor if everything worked correctly, we visualize a projection along <tt>z</tt> of each cropped particle using the command:<br />
<tt>dslices(targetBinnedParticles,'proj','c');shg;</tt><br />
<br />
[[ File:alignCommandLine_dslicesOnBinnedGoldBeads.png |thumb|center| 600px|dslices on binned gold beads]]<br />
== Reconstruct particles on the fly ==<br />
<br />
The command that reconstructs full sized particles from an aligned stack is <tt>dynamo_table_trec</tt>. This is an independent function, not part of the workflow, implying that we will need to perform some format conversion.<br />
<br />
<tt>tbl = dread('binnedGoldbeads.tbl');</tt><br />
<br />
We scale the table. "Bin level 2" means that each pixel in the binned tomogram is 2 times twice its size in the full resolution tomogram. This is the factor that we input into <tt>dynamo_table_rescale</tt>:<br />
<nowiki>alignmentBinLevel = 2;<br />
tblFull = dynamo_table_rescale(tbl,'factor',2^alignmentBinLevel); </nowiki><br />
<br />
Now we fetch the full sized aligned stack stored inside the alignment folder:<br />
<tt>source = 'workflows/hivCommandLine.AWF/align/alignedFullStack.mrc';</tt><br />
<br />
We create a data folder to contain the particles to be reconstructed on the fly:<br />
<tt>targetDirectory = 'tempCrop.Data'</tt>;<br />
<br />
Angles ''actually'' used during the reconstruction are contained in the ''reconstructionTiltAngles' item of the output folder. Note that they might differ from the nominal angles, as some views might have been eliminated, both explicitely by the user, of by the program not being able to align all tilts.<br />
<tt>angles = 'workflows/hivCommandLine.AWF/align/reconstructionTiltAngles.tlt';</tt><br />
<br />
The (binned) coordinates were measured in a binned tomogram, whose dimensionality is requested by <tt>dynamo_table_rec</tt>. We need to provide <tt>dynamo_table_rec</tt> with <br />
<nowiki>visualizationTomogram = 'workflows/hivCommandLine.AWF/reconstruction/binnedReconstructionWBP.mrc';<br />
sizeTomogram = dynamo_read_size(visualizationTomogram);<br />
sizeTomogramFull = sizeTomogram*2^alignmentBinLevel;</nowiki><br />
<br />
The tomogram center can be shifted in order to account for a possible shift in the centre in the binned reconstruction that was used to crop the particles.<br />
<tt>shiftTomogramCenter = [0,0,0];</tt><br />
We proceed now to the reconstruction. Our sid length should be 4 times the size we used before:<br />
<tt>sidelengthFullSize = 128;</tt><br />
We use the <tt>applyRampFilter</tt> option in order to explicitely estate the kind of filter that we want during the back projection.<br />
<nowiki>dynamo_table_rec(tblFull,source,angles,targetDirectory, sidelengthFullSize, 'applyRampFilter',1, 'sizeTomogram',sizeTomogramFull, 'shiftTomogramCenter',shiftTomogramCenter);</nowiki><br />
<br />
We can now check that the particles have been reconstructed in full size:<br />
<tt>figure;dslices(targetDirectory,'proj','c');shg;</tt><br />
[[ File:alignCommandLine_dslicesOnFullSizedGoldBeads.png |thumb|center| 600px|dslices on full sized gold beads]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)&diff=4183Walkthrough on GUI based tilt series alignment (EMBO2021)2021-09-06T15:09:53Z<p>Stefano Scaramuzza: /* Data */</p>
<hr />
<div>''Dynamo'' includes a package for automated alignment and reconstruction of tilt series. This walkthrough guides users through a series of steps on how to use it in GUI-based mode. The [[Walkthrough on command line based tilt series alignment | walkthrough on command line based tilt series alignment]] shows how to operate this procedure through the command line.<br />
<br />
= Data =<br />
<br />
Here we use a binned version of the first tilt series from the EMPIAR entry 10164, depicting a set of virus like particles (VLP), which is already prepared in your working directory (see [[EMBO_workshop_2021 | workshop page]] for details on how to access the directory).<br />
<br />
= Creating an alignment workflow= <br />
<br />
Start the alignment GUI by typing the command <tt>dtsa</tt> (short form of <tt>dynamo_tilt_series_alignment</tt>) into the ''Dynamo'' command line: <br />
<br />
<tt>dtsa</tt><br />
<br />
A pop-up window appears. Click on "create a new workflow":<br />
<br />
[[ File:alignGUI_creatingAWorkflow.png |thumb|center| 600px|creating a workflow]]<br />
<br />
Here, the input data for the workflow (tilt series, angles, ...) is introduced:<br />
<br />
[[ File:alignGUI_inputWorkflowParameters.png |thumb|center| 600px|input workflow parameters]]<br />
<br />
The option <tt>configuration file</tt> is used to pass information on possible tilts that should be discarded. Here, we leave this option untouched. Press on ''create workflow'', to open the alignment GUI.<br />
<br />
= GUI description =<br />
The alignment GUI guides the process of generating gold bead coordinates, aligning the tilt series, correcting the CTF and to finally reconstruct the tomogram. It has 5 parts. Starting from top to bottom, they are:<br />
# '''Menu:''' Different tools for visualization and customization of general settings. The option ''Settings'' > ''edit[general]'' for example lets you adjust the pixel size and different acquisition parameters such as Cs etc. Here we keep the default settings.<br />
# '''Icons:''' Quick access to tools such as ''dmarkers'', ''occupancy plot'', ''gold bead gallery'', ''gold bead targeting'', ''fitting'' and ''marker selection''. Their functionalities will be explained later.<br />
# '''Tabs:''' The tabs describe the 6 ''execution areas'' that are executed from left to right: ''detection'', ''indexing'', ''refinement'', ''alignment'', ''ctf'', ''reconstruction''. Each execution areas contains a set of processing steps.<br />
# '''Steps:''' The processing steps of each tab are run sequentually from top to bottom. However, after execution, the steps can be revisited to change or update information. Steps need to be active (letters highlighted in black, not grey) to be executed. They are automatically activated once the previous steps have been executed or the required input is given. Input parameters are edited by clicking the plus button (+) on the left of the step button. When a step is under execution, the text of the button becomes green. After execution the text becomes bold. At the right side of each of the steps, a tool icon is located. The tool icon can be right-clicked to provide a pop-up menu with specific tools for each step. Next to the tool icon there is the ''View'' button which provides specific tools to visualize and inspect the results of each step.<br />
# '''Run options:''' Options to run multiple steps.<br />
<br />
[[ File:alignGUI_alignmentGUIBeforeCompletionOfAnyStep.png |thumb|center| 600px|alignment GUI before completion of any step]]<br />
<br />
= Processing =<br />
We are now ready to run all the processing steps. In the following, each step is described in detail.<br />
== Detection ==<br />
The goal of this step is to find the positions of the gold bead projections in the tilt series. To do that, ''Dynamo'' cross correlates each micrograph against a 2D model of the gold bead and then locate the cross correlation peaks, which provide putative positions for actual projections of gold beads. Further analysis of these peaks will determine which ones correspond to actual gold bead projection (''peak feature'' and ''peak selection'').<br />
<br />
=== Visualization matrix ===<br />
This step just creates a binned version of the tilt series stack to accelerate visualizations that will be invoked in a later point. If you click on the box standing on the left of ''creates a binned tilt series'' button with a ''+'' symbol, an option box appears, indicating the default binning factor which is 2. This parameter can be changed by users. Binning factor 2 in ''Dynamo'' is equivalent to a binning factor of 4 in ''IMOD''.<br />
Click on ''creates a binned tilt series'' button to create a binned tilt series stack.<br />
Rightclick on ''View'' button to visualize the newly binned tilt series stack in <tt>tmshow</tt>. In the <tt>tmshow</tt> window, visualization parameters as binning, band and thickness can be adjusted if needed.<br />
<br />
Now, we are going to measure the diameter of the gold beads precisely in <tt>tmshow</tt> in the original (unbinned) tilt series data. To do that, press the ''IO'' in the upper toolbox of the ''alignment workflow GUI'', then go to ''tilt series matrix > inspect > On-the-fly view [dtmshow]''. The <tt>tmshow</tt> window opens, adjust contrast if necessary for better visualization.<br />
[[ File:openOriginalTiltSeriesDataGUI.png |thumb|center| 600px| open unbinned tilt series data to measure gold bead diameter]]<br />
<br />
Zoom in a gold bead by holding the <tt>shift</tt> key and scrolling up your mouse wheel (alternatively, use the icon with the red rectangle. Avoid using the lens icons). Adjust the visualized region by holding the <tt>shift</tt> key again and dragging with right mouse click on <tt>tmshow</tt> window.<br />
<br />
[[ File:alignGUI_openTomoshowOnTheFullSizedMatrixToCheckTheSizeOfAGoldBead.png |thumb|center| 600px|open tomoshow on the full sized matrix to check the size of a gold bead]]<br />
<br />
Once you have located the gold bead of interest, click on the ruler icon at the top toolbox to activate length measurements on the displayed image data. Draw a line across the gold bead, and the length of the line appears in pixels, which correspond to the gold bead diameter. In this case, the length of this line should be 32 pixel aprox.<br />
<br />
[[ File:OriginalGoldBeadDiameter.png |thumb|center| 600px|the ruler tool allows to measure in pixels the diameter of a gold bead]]<br />
<br />
=== Detection of gold beads ===<br />
Here we provide the size of the gold beads. The measured diameter of the gold beads is 32 pixels in the original image, therefore its radius is 16 pixels. We introduce the bead radius and some other parameters (see screenshot) to create an accurate initial gold bead template (here, the binning is just used internally for quick and robust bead detection).<br />
<br />
[[ File:alignGUI_DetectionParameters.png |thumb|center| 600px|the plus icon at corner reveals the values of the parameters of a given step.]]<br />
<br />
After pressing the button ''detect peaks'', ''Dynamo'' creates a raw cross-correlation matrix based on the initial template which is binary and extracts 300 cc-peaks by default. Then, the extracted peaks are averaged to create a refined template. The refined template is used to compute a refined cc-matrix from which 300 peaks are extracted and used as initial working markers. Once the ''detect peaks'' button turns black bold, the computation is finished. Right click on the <tt>view</tt> area of the step to access the different visualisation options.<br />
<br />
[[ File:alignGUI_markerSetsCanBeViewedAsCloudOfPoints.png |thumb|center| 600px| Secondary click the view panel to open a menu to access different results of a step. Marker sets can be viewed as cloud of points.]]<br />
<br />
This view shows the overlaid sum of the x,y coordinates of ''all'' cc peaks from ''all'' micrographs in the tilt series.<br />
[[ File:alignGUI_cloudOfUnindexedCcPeaks.png |thumb|center| 600px|cloud of unindexed cc peaks]]<br />
We can also check the positions of all the detected spots on each micrograph:<br />
[[ File:alignGUI_showsTheDetectedSpotsOnTheVideoGUI.png |thumb|center| 600px|shows the detected spots on the video GUI]]<br />
This GUI does not allow for edition of the markers. '''Tip:''' enable ''auto'' option at the bottom center of the visualization GUI to facilitate tilt series visualization.<br />
[[ File:alignGUI_staticVideoAnnotationsCannotBeEdited.png |thumb|center| 600px|static video annotations cannot be edited]]<br />
The neighbourhood of cross correlation peaks can be visualised individually as 2D patches from the tilt series image data:<br />
[[ File:alignGUI_EachPatchWillBeAPutativeGoldBead.png |thumb|center| 600px|Each patch will be a putative gold bead]]<br />
[[ File:alignGUI_theSetOfCcPeaksIsShownAsAMontage.png |thumb|center| 600px|the set of cc peaks is shown as a montage]]<br />
This visualization is normally a good gauge on how the detection procedure has worked. The example below would be an example of detection procedure carried with wrong search parameters<br />
[[ File:alignGUI_exampleOfFailedSearch.png |thumb|center| 600px|example of failed gold bead detection]]<br />
Note that you have the option of testing your parameters against a single image before launching a full detection procedure. Right-click on the tool icon on the right side of ''detect peaks'' pushbutton.<br />
[[ File:alignGUI_toolboxOptionForTestingTheParametersOnACentralImage.png |thumb|center| 600px|toolbox option for testing the parameters on a central image]]<br />
[[ File:alignGUI_resultsOfTestingOnACentralImage.png |thumb|center| 600px|results of testing on a central image]]<br />
<br />
=== Computation of observation features ===<br />
At this stage, the observed cross correlation peaks are analysed. A merit figure is assigned to each peak based on a ''rotational merit'' that measures its rotational symmetry. Press the button ''Peak features'' to compute the rotational symmetry of the current working markers. The working markers that do not meet rotational symmetry will be discarded. The results can be depicted on patches representing individual gold beads:<br />
[[ File:alignGUI_patchGalleryForFeatures.png |thumb|center| 600px|patch gallery for features]]<br />
[[ File:alignGUI_theGalleryAllowsRelatingAppearanceOfGoldBeadAnValueOfTheFeatures.png |thumb|center| 600px|the gallery shows the detected gold beads, the CC score (yellow) and the rotational merit (red). Tilt number is purple.]]<br />
or extracting properties of the whole data set.<br />
[[ File:alignGUI_featurePlotterAllowsCheckingOfTheFeatureValuesOnTheWholeSet.png |thumb|center| 600px|feature plotter allows checking of the feature values on the whole set]]<br />
[[ File:alignGUI_scatter2dOption.png |thumb|center| 600px| We select ''scatter'', set the y axis to ''rotationaMerit'' and then press ''Scatter 2d.]]<br />
[[ File:alignGUI_eachPointRepresentsACcPeak.png |thumb|center| 600px| We get a scatter plot of each peak and its relation to the CC and rotational merit score. Seeing two distinct populations means that the majority of the gold beads are distinguishable from other signal such as background.]]<br />
<br />
=== Selection of best gold beads===<br />
The population with the best markers is automatically selected by clicking on ''Peak selection''. The results can be inspected again in a plot<br />
[[ File:alignGUI_checkObservationCloudAfterSelectionOrClustering.png |thumb|center| 600px|check observation cloud after selection or clustering ]]<br />
[[ File:alignGUI_observationCloudHasBeenThinned.png |thumb|center| 600px|observation cloud has been thinned]]<br />
or in the micrographs themselves:<br />
[[ File:alignGUI_checkMarkerPositionsAfterClustering.png |thumb|center| 600px|check marker positions after clustering]]<br />
[[ File:alignGUI_markerPositionsAfterClustering.png |thumb|center| 600px|marker positions after clustering]]<br />
<br />
== Indexing == <br />
The goal of this execution area is to find ''trails'' of gold beads, i.e., we connect the observations from different micrographs that correspond to the same gold bead. All the steps in this area will constantly update a set of ''working markers''.<br />
<br />
=== Rough Alignment ===<br />
In this step, micrographs are compared pair-wise. For each couple, we look for the relative shifts that generate the most matches of cross-correlation peaks between one micrograph and the next one. This procedure induces trails of matched pairs of observations of variable length along the tilt series. The scheme that depicts which trails have representatives in which tilts is called ''occupancy graph''<br />
[[ File:alignGUI_occupancySchemeOfTheResultsOfAStep.png |thumb|center| 600px|occupancy scheme of the results of a step]]<br />
[[ File:alignGUI_roughAlignmentSelectsManyPotentialTrails.png |thumb|center| 600px| The rough alignment selects many potential trails. On the x-axis is the tilt number, on the y-axis the trail number and each red dot marks the presence of a marker.]]<br />
Note that these trails are very lacunary and do not conserve the identity of a trail; whenever a trail that represents an actual 3d marker gets interrupted, the same 3d marker might generate a different trail in other tilts.<br />
<br />
=== Keep long chains ===<br />
Click on ''keep long chains'' to select the longest trails, which will then be used to generate a 3d model of the markers:<br />
[[ File:alignGUI_trailsWithFewMarkersAreThenRejected.png |thumb|center| 600px|short trails have been rejected]]<br />
Note that from this step onwards, the toolbox of each step offers the possibility to revert the current markers to the output generated at that state. Thus, if further processing of the markers in a later step leads you to loose a good marker set, you can always come back to this point.<br />
[[ File:alignGUI_noteWorkingMarkersCanBeRestoredToTheResultOfAPreviousStep.png |thumb|center| 600px|working markers can be restored to the result of a previous step]]<br />
<br />
=== Iterative reindexing ===<br />
The 3d model is projected on each micrograph. Observations that are closest (and below a distance threshold) to each reprojection are assigned to the corresponding 3d marker. This refines the 3d model, and the process is iterated till no further improvement occurs (or till a maximum of ten iterations is reached). Keep the default parameters and click on ''iterative reindexing''.<br />
[[ File:alignGUI_parametersForIterativeReindexing.png |thumb|center| 600px|parameters for iterative reindexing]]<br />
[[ File:alignGUI_resultOfIterativeReindexing.png |thumb|center| 600px|Result of iterative reindexing. Missing markers have been added to the trails]]<br />
<br />
===Tilt gap filling ===<br />
Here, potential tilts with too few (or zero) markers can be rescued. The cloud of observations found on that micrograph will be compared to the cloud or reprojections generated by the current model. Click on ''fill tilts without markers''.<br />
[[ File:alignGUI_parametersForTiltGapFiller.png |thumb|center| 600px|parameters for tilt gap filler]]<br />
[[ File:alignGUI_occupancyAfterTiltGapFiller.png |thumb|center| 600px|Results of tilt gap filler. Note how (in this example) a lot of markers have been rescued on the first tilt.]]<br />
[[ File:alignGUI_weHave66TracesAfterGapFiller.png |thumb|center| 600px|Click on ''working markers'' to see that (in this example) we have 66 traces after tilt gap filling.]]<br />
[[ File:alignGUI_resultsOfTiltGapFiller.png |thumb|center| 600px|results of tilt gap filler (''show markers created by this step on tilt series'')]]<br />
<br />
===Trail extension===<br />
This step aims at locating trails that have been excluded by previous steps of the alignment procedure, or that were never detected as trails because previous, lesser quality stages of the 3d model were not able to analyse the observations correctly.<br />
[[ File:alignGUI_parametersOfTiltExtensor.png |thumb|center| 600px|parameters of tilt extensor]]<br />
[[ File:alignGUI_trailExtensorAddsAFewNewTrails.png |thumb|center| 600px|trail extensor adds a few new trails]]<br />
<br />
Note that newly added trails have empty spaces in the central area, they are likely to be gold beads located far away from the tilt axes and which get into the field of view only for high tilts. <br />
[[ File:alignGUI_someOfTheNewTrailsHaveEmptySpaces.png |thumb|center| 600px|some of the new trails may have empty spaces]]<br />
[[ File:alignGUI_resultOfTrailExtensor.png |thumb|center| 600px|result of trail extensor]]<br />
<br />
==Refinement ==<br />
This area refines the positions of the gold beads. We will run all steps of this area by pressing on a single button (the ''run this tab'' option at the bottom of the GUI). The button will remain green while the different steps are computed.<br />
[[ File:alignGUI_tabForRefinement.png |thumb|center| 600px|tab for refinement]]<br />
Check the current refinement error:<br />
[[ File:alignGUI_betterFiitingIsAttained.png |thumb|center| 600px|better fitting is attained]]<br />
Optionally, we can use the scissors icon to "trim" observations (or full markers) with too high residuals. Click on the scissor icon. A window pops up with options to set different thresholds. Click on ''Tools'' and then ''Show residuals per observation'':<br />
[[ File:alignGUI_trimmingOfMarkersCanBeAccessedThroughTheScissorIcon.png |thumb|center| 600px|trimming of markers can be accessed through the scissor icon]]<br />
You get a plot with the residuals for each observation ordered by the value of the residual. You can use this plot to estimate the residual distribution and to manually adapt the residual thresholds, if necessary.<br />
[[ File:alignGUI_residualsAtEachObservation.png |thumb|center| 600px|residuals at each observation]]<br />
<br />
== Alignment ==<br />
The final step of the alignment procedure is to create the aligned tilt series using the coordinates of the markers that were optimized until now.<br />
<br />
=== Fix the markers ===<br />
Here we freeze the final version of the working markers and create a marker file that will define all subsequent steps of alignment and reconstruction. Click on ''fix alignment markers''.<br />
<br />
[[ File:alignGUI_areaForCreationOfAnAlignedTiltSeries.png |thumb|center| 600px|area for creation of an aligned tilt series]]<br />
<br />
=== Align the tilt series ===<br />
We then proceed to the alignment task itself, generating full resolution aligned tilt series. We also have the option to generate a binned version of the aligned tilt series that can be used for quick visualization or to create binned reconstructions in the next steps. Click on ''Create full-size aligned stack'' and ''Create binned aligned stack''. For a quick assessment, open the binned stack in tomoshow:<br />
[[ File:alignGUI_visualizationOfAlignedTiltStack.png |thumb|center| 600px|visualization of aligned tilt stack]]<br />
[[ File:alignGUI_binnedAlignedTiltStack.png |thumb|center| 600px|binned aligned tilt stack]]<br />
A quick way to gauge the quality of the alignment is by checking that the beads move nicely horizontally over all tilts.<br />
<br />
== CTF ==<br />
In this GUI, CTF estimation and correction are done using CTFFIND4 and IMOD's ctfphaseflip. This part will not be used in the workshop.<br />
<br />
== Reconstruction ==<br />
<br />
=== Binned reconstruction ===<br />
Click on ''binned reconstruction'' to create a binned tomogram using the previously created binned aligned tilt series. Adapt the tomogram height if necessary (the tomogram center is automatically located at the center of the 3d reconstruction of the aligned stack). Select ''binned SIRT-like reconstruction'' to generate a high contrast tomogram. Run the step ''binned reconstruction'' and visualize the resulting binned SIRT-like tomogram: <br />
[[ File:alignGUI_chooseAVisualization.png |thumb|center| 700px|choose a visualization]]<br />
[[ File:alignGUI_binnedWeightedBackProjectionReconstruction.png |thumb|center| 600px|binned SIRT-like reconstruction]]<br />
<br />
=== Full sized reconstruction ===<br />
The full sized reconstruction is the one you will be using for later image processing steps that require the highest resolution possible (e.g., subtomogram averaging refinements). It can be created related to a coordinate directly read from the binned tomogram. This feature is useful to create reconstructions localised on an area of interest. In this tutorial, we will focus on one of the VLPs, resulting in a tomogram containing just one VLP. First, use the pin tool to select a point on the binned reconstruction. We use the previously opened binned SIRT-like tomogram for this:<br />
[[ File:alignGUI_thePinToolIsUsefulForLocatingIsolatedPoints.png |thumb|center| 600px|Click in the center of an object of interest using the pin tool.]]<br />
<br />
We check the size of the structure by using the measuring tool:<br />
[[ File:alignGUI_weCheckTheDistanceOfTheObjectOfInterest.png |thumb|center| 600px|we check the distance of the object of interest]]<br />
<br />
Before generating the final localized tomogram, we need to set some parameters. We take the measured distance of 83 pixels and compensate for the binning factor of two, leading to 83*2*2=332 pixels. For the dimensions of the tomogram we add some spare space and define the x,y and z length to be 500 pixels. Select ''use center of binned coordniates'' and enter the values we measured before with the pin tool. Select ''WBP reconstruction'' and run the step ''full reconstruction''. Use the inspection tools to fetch the result:<br />
[[ File:alignGUI_asAnSpecificViewerIsNotAvailableForThisStepWeCanUseTheToolboxOfTheStepsItems.png |thumb|center| 700px|as an specific viewer is not available for this step we can use the toolbox of the steps items]]<br />
<br />
As it is a WBP reconstruction, it is much noisier than the corresponding area in the SIRT-like tomogram. Adjust the contrast, lowpass filter and thickness to increase the signal:<br />
[[ File:alignGUI_theFullSizeReconstructionIncludesOnlyTheObjectOfInterest.png |thumb|center| 600px|the full size WBP reconstruction includes only the object of interest]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4182EMBO workshop 20212021-09-06T15:08:07Z<p>Stefano Scaramuzza: /* Schedule */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[advanced starters guide | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)&diff=4181Walkthrough on GUI based tilt series alignment (EMBO2021)2021-09-06T15:07:51Z<p>Stefano Scaramuzza: Created page with "''Dynamo'' includes a package for automated alignment and reconstruction of tilt series. This walkthrough guides users through a series of steps on how to use it in GUI-based..."</p>
<hr />
<div>''Dynamo'' includes a package for automated alignment and reconstruction of tilt series. This walkthrough guides users through a series of steps on how to use it in GUI-based mode. The [[Walkthrough on command line based tilt series alignment | walkthrough on command line based tilt series alignment]] shows how to operate this procedure through the command line.<br />
<br />
= Data =<br />
<br />
We use a binned version of the first tilt series from the EMPIAR entry 10164, depicting a set of virus like particles (VLP). Visit the course materials to access the tilt series or download it directly from EMPIAR. You can also follow this tutorial using your own data.<br />
<br />
= Creating an alignment workflow= <br />
<br />
Start the alignment GUI by typing the command <tt>dtsa</tt> (short form of <tt>dynamo_tilt_series_alignment</tt>) into the ''Dynamo'' command line: <br />
<br />
<tt>dtsa</tt><br />
<br />
A pop-up window appears. Click on "create a new workflow":<br />
<br />
[[ File:alignGUI_creatingAWorkflow.png |thumb|center| 600px|creating a workflow]]<br />
<br />
Here, the input data for the workflow (tilt series, angles, ...) is introduced:<br />
<br />
[[ File:alignGUI_inputWorkflowParameters.png |thumb|center| 600px|input workflow parameters]]<br />
<br />
The option <tt>configuration file</tt> is used to pass information on possible tilts that should be discarded. Here, we leave this option untouched. Press on ''create workflow'', to open the alignment GUI.<br />
<br />
= GUI description =<br />
The alignment GUI guides the process of generating gold bead coordinates, aligning the tilt series, correcting the CTF and to finally reconstruct the tomogram. It has 5 parts. Starting from top to bottom, they are:<br />
# '''Menu:''' Different tools for visualization and customization of general settings. The option ''Settings'' > ''edit[general]'' for example lets you adjust the pixel size and different acquisition parameters such as Cs etc. Here we keep the default settings.<br />
# '''Icons:''' Quick access to tools such as ''dmarkers'', ''occupancy plot'', ''gold bead gallery'', ''gold bead targeting'', ''fitting'' and ''marker selection''. Their functionalities will be explained later.<br />
# '''Tabs:''' The tabs describe the 6 ''execution areas'' that are executed from left to right: ''detection'', ''indexing'', ''refinement'', ''alignment'', ''ctf'', ''reconstruction''. Each execution areas contains a set of processing steps.<br />
# '''Steps:''' The processing steps of each tab are run sequentually from top to bottom. However, after execution, the steps can be revisited to change or update information. Steps need to be active (letters highlighted in black, not grey) to be executed. They are automatically activated once the previous steps have been executed or the required input is given. Input parameters are edited by clicking the plus button (+) on the left of the step button. When a step is under execution, the text of the button becomes green. After execution the text becomes bold. At the right side of each of the steps, a tool icon is located. The tool icon can be right-clicked to provide a pop-up menu with specific tools for each step. Next to the tool icon there is the ''View'' button which provides specific tools to visualize and inspect the results of each step.<br />
# '''Run options:''' Options to run multiple steps.<br />
<br />
[[ File:alignGUI_alignmentGUIBeforeCompletionOfAnyStep.png |thumb|center| 600px|alignment GUI before completion of any step]]<br />
<br />
= Processing =<br />
We are now ready to run all the processing steps. In the following, each step is described in detail.<br />
== Detection ==<br />
The goal of this step is to find the positions of the gold bead projections in the tilt series. To do that, ''Dynamo'' cross correlates each micrograph against a 2D model of the gold bead and then locate the cross correlation peaks, which provide putative positions for actual projections of gold beads. Further analysis of these peaks will determine which ones correspond to actual gold bead projection (''peak feature'' and ''peak selection'').<br />
<br />
=== Visualization matrix ===<br />
This step just creates a binned version of the tilt series stack to accelerate visualizations that will be invoked in a later point. If you click on the box standing on the left of ''creates a binned tilt series'' button with a ''+'' symbol, an option box appears, indicating the default binning factor which is 2. This parameter can be changed by users. Binning factor 2 in ''Dynamo'' is equivalent to a binning factor of 4 in ''IMOD''.<br />
Click on ''creates a binned tilt series'' button to create a binned tilt series stack.<br />
Rightclick on ''View'' button to visualize the newly binned tilt series stack in <tt>tmshow</tt>. In the <tt>tmshow</tt> window, visualization parameters as binning, band and thickness can be adjusted if needed.<br />
<br />
Now, we are going to measure the diameter of the gold beads precisely in <tt>tmshow</tt> in the original (unbinned) tilt series data. To do that, press the ''IO'' in the upper toolbox of the ''alignment workflow GUI'', then go to ''tilt series matrix > inspect > On-the-fly view [dtmshow]''. The <tt>tmshow</tt> window opens, adjust contrast if necessary for better visualization.<br />
[[ File:openOriginalTiltSeriesDataGUI.png |thumb|center| 600px| open unbinned tilt series data to measure gold bead diameter]]<br />
<br />
Zoom in a gold bead by holding the <tt>shift</tt> key and scrolling up your mouse wheel (alternatively, use the icon with the red rectangle. Avoid using the lens icons). Adjust the visualized region by holding the <tt>shift</tt> key again and dragging with right mouse click on <tt>tmshow</tt> window.<br />
<br />
[[ File:alignGUI_openTomoshowOnTheFullSizedMatrixToCheckTheSizeOfAGoldBead.png |thumb|center| 600px|open tomoshow on the full sized matrix to check the size of a gold bead]]<br />
<br />
Once you have located the gold bead of interest, click on the ruler icon at the top toolbox to activate length measurements on the displayed image data. Draw a line across the gold bead, and the length of the line appears in pixels, which correspond to the gold bead diameter. In this case, the length of this line should be 32 pixel aprox.<br />
<br />
[[ File:OriginalGoldBeadDiameter.png |thumb|center| 600px|the ruler tool allows to measure in pixels the diameter of a gold bead]]<br />
<br />
=== Detection of gold beads ===<br />
Here we provide the size of the gold beads. The measured diameter of the gold beads is 32 pixels in the original image, therefore its radius is 16 pixels. We introduce the bead radius and some other parameters (see screenshot) to create an accurate initial gold bead template (here, the binning is just used internally for quick and robust bead detection).<br />
<br />
[[ File:alignGUI_DetectionParameters.png |thumb|center| 600px|the plus icon at corner reveals the values of the parameters of a given step.]]<br />
<br />
After pressing the button ''detect peaks'', ''Dynamo'' creates a raw cross-correlation matrix based on the initial template which is binary and extracts 300 cc-peaks by default. Then, the extracted peaks are averaged to create a refined template. The refined template is used to compute a refined cc-matrix from which 300 peaks are extracted and used as initial working markers. Once the ''detect peaks'' button turns black bold, the computation is finished. Right click on the <tt>view</tt> area of the step to access the different visualisation options.<br />
<br />
[[ File:alignGUI_markerSetsCanBeViewedAsCloudOfPoints.png |thumb|center| 600px| Secondary click the view panel to open a menu to access different results of a step. Marker sets can be viewed as cloud of points.]]<br />
<br />
This view shows the overlaid sum of the x,y coordinates of ''all'' cc peaks from ''all'' micrographs in the tilt series.<br />
[[ File:alignGUI_cloudOfUnindexedCcPeaks.png |thumb|center| 600px|cloud of unindexed cc peaks]]<br />
We can also check the positions of all the detected spots on each micrograph:<br />
[[ File:alignGUI_showsTheDetectedSpotsOnTheVideoGUI.png |thumb|center| 600px|shows the detected spots on the video GUI]]<br />
This GUI does not allow for edition of the markers. '''Tip:''' enable ''auto'' option at the bottom center of the visualization GUI to facilitate tilt series visualization.<br />
[[ File:alignGUI_staticVideoAnnotationsCannotBeEdited.png |thumb|center| 600px|static video annotations cannot be edited]]<br />
The neighbourhood of cross correlation peaks can be visualised individually as 2D patches from the tilt series image data:<br />
[[ File:alignGUI_EachPatchWillBeAPutativeGoldBead.png |thumb|center| 600px|Each patch will be a putative gold bead]]<br />
[[ File:alignGUI_theSetOfCcPeaksIsShownAsAMontage.png |thumb|center| 600px|the set of cc peaks is shown as a montage]]<br />
This visualization is normally a good gauge on how the detection procedure has worked. The example below would be an example of detection procedure carried with wrong search parameters<br />
[[ File:alignGUI_exampleOfFailedSearch.png |thumb|center| 600px|example of failed gold bead detection]]<br />
Note that you have the option of testing your parameters against a single image before launching a full detection procedure. Right-click on the tool icon on the right side of ''detect peaks'' pushbutton.<br />
[[ File:alignGUI_toolboxOptionForTestingTheParametersOnACentralImage.png |thumb|center| 600px|toolbox option for testing the parameters on a central image]]<br />
[[ File:alignGUI_resultsOfTestingOnACentralImage.png |thumb|center| 600px|results of testing on a central image]]<br />
<br />
=== Computation of observation features ===<br />
At this stage, the observed cross correlation peaks are analysed. A merit figure is assigned to each peak based on a ''rotational merit'' that measures its rotational symmetry. Press the button ''Peak features'' to compute the rotational symmetry of the current working markers. The working markers that do not meet rotational symmetry will be discarded. The results can be depicted on patches representing individual gold beads:<br />
[[ File:alignGUI_patchGalleryForFeatures.png |thumb|center| 600px|patch gallery for features]]<br />
[[ File:alignGUI_theGalleryAllowsRelatingAppearanceOfGoldBeadAnValueOfTheFeatures.png |thumb|center| 600px|the gallery shows the detected gold beads, the CC score (yellow) and the rotational merit (red). Tilt number is purple.]]<br />
or extracting properties of the whole data set.<br />
[[ File:alignGUI_featurePlotterAllowsCheckingOfTheFeatureValuesOnTheWholeSet.png |thumb|center| 600px|feature plotter allows checking of the feature values on the whole set]]<br />
[[ File:alignGUI_scatter2dOption.png |thumb|center| 600px| We select ''scatter'', set the y axis to ''rotationaMerit'' and then press ''Scatter 2d.]]<br />
[[ File:alignGUI_eachPointRepresentsACcPeak.png |thumb|center| 600px| We get a scatter plot of each peak and its relation to the CC and rotational merit score. Seeing two distinct populations means that the majority of the gold beads are distinguishable from other signal such as background.]]<br />
<br />
=== Selection of best gold beads===<br />
The population with the best markers is automatically selected by clicking on ''Peak selection''. The results can be inspected again in a plot<br />
[[ File:alignGUI_checkObservationCloudAfterSelectionOrClustering.png |thumb|center| 600px|check observation cloud after selection or clustering ]]<br />
[[ File:alignGUI_observationCloudHasBeenThinned.png |thumb|center| 600px|observation cloud has been thinned]]<br />
or in the micrographs themselves:<br />
[[ File:alignGUI_checkMarkerPositionsAfterClustering.png |thumb|center| 600px|check marker positions after clustering]]<br />
[[ File:alignGUI_markerPositionsAfterClustering.png |thumb|center| 600px|marker positions after clustering]]<br />
<br />
== Indexing == <br />
The goal of this execution area is to find ''trails'' of gold beads, i.e., we connect the observations from different micrographs that correspond to the same gold bead. All the steps in this area will constantly update a set of ''working markers''.<br />
<br />
=== Rough Alignment ===<br />
In this step, micrographs are compared pair-wise. For each couple, we look for the relative shifts that generate the most matches of cross-correlation peaks between one micrograph and the next one. This procedure induces trails of matched pairs of observations of variable length along the tilt series. The scheme that depicts which trails have representatives in which tilts is called ''occupancy graph''<br />
[[ File:alignGUI_occupancySchemeOfTheResultsOfAStep.png |thumb|center| 600px|occupancy scheme of the results of a step]]<br />
[[ File:alignGUI_roughAlignmentSelectsManyPotentialTrails.png |thumb|center| 600px| The rough alignment selects many potential trails. On the x-axis is the tilt number, on the y-axis the trail number and each red dot marks the presence of a marker.]]<br />
Note that these trails are very lacunary and do not conserve the identity of a trail; whenever a trail that represents an actual 3d marker gets interrupted, the same 3d marker might generate a different trail in other tilts.<br />
<br />
=== Keep long chains ===<br />
Click on ''keep long chains'' to select the longest trails, which will then be used to generate a 3d model of the markers:<br />
[[ File:alignGUI_trailsWithFewMarkersAreThenRejected.png |thumb|center| 600px|short trails have been rejected]]<br />
Note that from this step onwards, the toolbox of each step offers the possibility to revert the current markers to the output generated at that state. Thus, if further processing of the markers in a later step leads you to loose a good marker set, you can always come back to this point.<br />
[[ File:alignGUI_noteWorkingMarkersCanBeRestoredToTheResultOfAPreviousStep.png |thumb|center| 600px|working markers can be restored to the result of a previous step]]<br />
<br />
=== Iterative reindexing ===<br />
The 3d model is projected on each micrograph. Observations that are closest (and below a distance threshold) to each reprojection are assigned to the corresponding 3d marker. This refines the 3d model, and the process is iterated till no further improvement occurs (or till a maximum of ten iterations is reached). Keep the default parameters and click on ''iterative reindexing''.<br />
[[ File:alignGUI_parametersForIterativeReindexing.png |thumb|center| 600px|parameters for iterative reindexing]]<br />
[[ File:alignGUI_resultOfIterativeReindexing.png |thumb|center| 600px|Result of iterative reindexing. Missing markers have been added to the trails]]<br />
<br />
===Tilt gap filling ===<br />
Here, potential tilts with too few (or zero) markers can be rescued. The cloud of observations found on that micrograph will be compared to the cloud or reprojections generated by the current model. Click on ''fill tilts without markers''.<br />
[[ File:alignGUI_parametersForTiltGapFiller.png |thumb|center| 600px|parameters for tilt gap filler]]<br />
[[ File:alignGUI_occupancyAfterTiltGapFiller.png |thumb|center| 600px|Results of tilt gap filler. Note how (in this example) a lot of markers have been rescued on the first tilt.]]<br />
[[ File:alignGUI_weHave66TracesAfterGapFiller.png |thumb|center| 600px|Click on ''working markers'' to see that (in this example) we have 66 traces after tilt gap filling.]]<br />
[[ File:alignGUI_resultsOfTiltGapFiller.png |thumb|center| 600px|results of tilt gap filler (''show markers created by this step on tilt series'')]]<br />
<br />
===Trail extension===<br />
This step aims at locating trails that have been excluded by previous steps of the alignment procedure, or that were never detected as trails because previous, lesser quality stages of the 3d model were not able to analyse the observations correctly.<br />
[[ File:alignGUI_parametersOfTiltExtensor.png |thumb|center| 600px|parameters of tilt extensor]]<br />
[[ File:alignGUI_trailExtensorAddsAFewNewTrails.png |thumb|center| 600px|trail extensor adds a few new trails]]<br />
<br />
Note that newly added trails have empty spaces in the central area, they are likely to be gold beads located far away from the tilt axes and which get into the field of view only for high tilts. <br />
[[ File:alignGUI_someOfTheNewTrailsHaveEmptySpaces.png |thumb|center| 600px|some of the new trails may have empty spaces]]<br />
[[ File:alignGUI_resultOfTrailExtensor.png |thumb|center| 600px|result of trail extensor]]<br />
<br />
==Refinement ==<br />
This area refines the positions of the gold beads. We will run all steps of this area by pressing on a single button (the ''run this tab'' option at the bottom of the GUI). The button will remain green while the different steps are computed.<br />
[[ File:alignGUI_tabForRefinement.png |thumb|center| 600px|tab for refinement]]<br />
Check the current refinement error:<br />
[[ File:alignGUI_betterFiitingIsAttained.png |thumb|center| 600px|better fitting is attained]]<br />
Optionally, we can use the scissors icon to "trim" observations (or full markers) with too high residuals. Click on the scissor icon. A window pops up with options to set different thresholds. Click on ''Tools'' and then ''Show residuals per observation'':<br />
[[ File:alignGUI_trimmingOfMarkersCanBeAccessedThroughTheScissorIcon.png |thumb|center| 600px|trimming of markers can be accessed through the scissor icon]]<br />
You get a plot with the residuals for each observation ordered by the value of the residual. You can use this plot to estimate the residual distribution and to manually adapt the residual thresholds, if necessary.<br />
[[ File:alignGUI_residualsAtEachObservation.png |thumb|center| 600px|residuals at each observation]]<br />
<br />
== Alignment ==<br />
The final step of the alignment procedure is to create the aligned tilt series using the coordinates of the markers that were optimized until now.<br />
<br />
=== Fix the markers ===<br />
Here we freeze the final version of the working markers and create a marker file that will define all subsequent steps of alignment and reconstruction. Click on ''fix alignment markers''.<br />
<br />
[[ File:alignGUI_areaForCreationOfAnAlignedTiltSeries.png |thumb|center| 600px|area for creation of an aligned tilt series]]<br />
<br />
=== Align the tilt series ===<br />
We then proceed to the alignment task itself, generating full resolution aligned tilt series. We also have the option to generate a binned version of the aligned tilt series that can be used for quick visualization or to create binned reconstructions in the next steps. Click on ''Create full-size aligned stack'' and ''Create binned aligned stack''. For a quick assessment, open the binned stack in tomoshow:<br />
[[ File:alignGUI_visualizationOfAlignedTiltStack.png |thumb|center| 600px|visualization of aligned tilt stack]]<br />
[[ File:alignGUI_binnedAlignedTiltStack.png |thumb|center| 600px|binned aligned tilt stack]]<br />
A quick way to gauge the quality of the alignment is by checking that the beads move nicely horizontally over all tilts.<br />
<br />
== CTF ==<br />
In this GUI, CTF estimation and correction are done using CTFFIND4 and IMOD's ctfphaseflip. This part will not be used in the workshop.<br />
<br />
== Reconstruction ==<br />
<br />
=== Binned reconstruction ===<br />
Click on ''binned reconstruction'' to create a binned tomogram using the previously created binned aligned tilt series. Adapt the tomogram height if necessary (the tomogram center is automatically located at the center of the 3d reconstruction of the aligned stack). Select ''binned SIRT-like reconstruction'' to generate a high contrast tomogram. Run the step ''binned reconstruction'' and visualize the resulting binned SIRT-like tomogram: <br />
[[ File:alignGUI_chooseAVisualization.png |thumb|center| 700px|choose a visualization]]<br />
[[ File:alignGUI_binnedWeightedBackProjectionReconstruction.png |thumb|center| 600px|binned SIRT-like reconstruction]]<br />
<br />
=== Full sized reconstruction ===<br />
The full sized reconstruction is the one you will be using for later image processing steps that require the highest resolution possible (e.g., subtomogram averaging refinements). It can be created related to a coordinate directly read from the binned tomogram. This feature is useful to create reconstructions localised on an area of interest. In this tutorial, we will focus on one of the VLPs, resulting in a tomogram containing just one VLP. First, use the pin tool to select a point on the binned reconstruction. We use the previously opened binned SIRT-like tomogram for this:<br />
[[ File:alignGUI_thePinToolIsUsefulForLocatingIsolatedPoints.png |thumb|center| 600px|Click in the center of an object of interest using the pin tool.]]<br />
<br />
We check the size of the structure by using the measuring tool:<br />
[[ File:alignGUI_weCheckTheDistanceOfTheObjectOfInterest.png |thumb|center| 600px|we check the distance of the object of interest]]<br />
<br />
Before generating the final localized tomogram, we need to set some parameters. We take the measured distance of 83 pixels and compensate for the binning factor of two, leading to 83*2*2=332 pixels. For the dimensions of the tomogram we add some spare space and define the x,y and z length to be 500 pixels. Select ''use center of binned coordniates'' and enter the values we measured before with the pin tool. Select ''WBP reconstruction'' and run the step ''full reconstruction''. Use the inspection tools to fetch the result:<br />
[[ File:alignGUI_asAnSpecificViewerIsNotAvailableForThisStepWeCanUseTheToolboxOfTheStepsItems.png |thumb|center| 700px|as an specific viewer is not available for this step we can use the toolbox of the steps items]]<br />
<br />
As it is a WBP reconstruction, it is much noisier than the corresponding area in the SIRT-like tomogram. Adjust the contrast, lowpass filter and thickness to increase the signal:<br />
[[ File:alignGUI_theFullSizeReconstructionIncludesOnlyTheObjectOfInterest.png |thumb|center| 600px|the full size WBP reconstruction includes only the object of interest]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Starters_guide_(EMBO2021)&diff=4180Starters guide (EMBO2021)2021-09-06T15:04:35Z<p>Stefano Scaramuzza: /* Alignment projects */</p>
<hr />
<div>This tutorial shows how to manually pick particles in a tomogram, extract them, align and classify them. This tutorial is a shortened version of the original [[Starters_guide | starters guide]] that was adapted for the use in ''Dynamo'' workshops.<br />
<br />
=Manual particle picking and ''Dynamo'' catalogues=<br />
The ''Dynamo'' catalogues are databases that manage tomograms and link the tomographic data to the extracted particles. Start the catalogue manager by typing the command:<br />
<br />
<tt>dcm</tt><br />
After the catalogue manager opens, we create 3 synthetic tomograms that include our particles (thermosomes) in the following way:<br />
<br />
[[File:StartersGuideFigure1.png|thumb|center|600px| Figure 1: creating a set of synthetic tomograms with ''Dynamo'' Catalogue Manager]]<br />
<br />
You can see the list of your tomograms and their metadata in a table in the bottom of your catalogue manager. When workin on your own projects, you can add tomograms to the catalogue by ''Catalogue -> Browse for new volume''. The tomograms are situated in the directory <tt>testCatalogue</tt>. They contain a small amount of noise and a missing wedge associated to rotation around Y-axis. We added the information about location of the particles in the tomograms in an additional catalogue called <tt>testCatalogue_withmodels</tt>. If needed, it can be opened from the current folder with ''Catalogue -> look for local catalogues'' in the catalogue manager window.<br />
<br />
==Viewing tomograms==<br />
Select the first tomogram in the list and go to ''View volume>Full tomogram file in tomoslice''. <br />
<br />
[[ File:StartersSelectTomoSliceInCatalogue.png |thumb| center |600px| Figure 2: Selecting a volume in the catalogue manager to be inspected in <tt>dpreview</tt>]]<br />
<br />
The volume browser tomoslicer loads the entire tomograms into memory and allows making annotations to the regions of interest. The tomograms in this tutorial are small, and you can load them directly into memory. For real life tomograms, bear in mind that you will need to [[Prebinned tomograms| prebin]] the files before loading them into memory. Tomoslice has a simple set of controls and is suitable for visualization tasks that require oblique sections through the tomogram. It uses the same tool as other ''Dynamo'' browsers to keep track of your annotations: a pool of models. You might need to adjust the contrast (blue arrow). To move through the tomogram slices, you can either use the mouse wheel, click and drag the tomogram slice up and down (orange arrows), or move the position control left and right (orange box).<br />
<br />
<br />
[[ File:StartersTomoSlice.png |thumb| center |600px|]]<br />
<br />
==Picking and extracting particles in tomograms==<br />
Coordinates of picked particles are represented by data types called models. In the tomoslicer window go to ''Model Pool -> Create new model in pool (choose type) -> General''. This is the simplest type of [[model]] where each clicked/model point corresponds to a single isolated particle. Now you can navigate up and down the tomogram, place the mouse on the center of a particle and press the [c] key on your keyboard to add a new model point. (Note the ''Help -> all hot keys'' options that lists the different the actions of the different keystrokes). Backspace button deletes the last clicked point (you can also use the right-click to delete single points).<br />
<br />
[[File:StartersGuideFigure4.png|thumb|center|600px| Generating a new model]]<br />
<br />
After you are done clicking about 10 particles click on ''Active model > Update Crop points'' in the tomoslice window.<br />
<br />
<br />
[[File:StartersGuideUpdate.png|thumb|center|600px|]]<br />
<br />
Save the model into the catalogue by ''Active model -> Save active model into catalogue (disk)'' and close the slicer window.<br />
<br />
[[File:StartersGuideFigure6.png|thumb|center|600px| ]]<br />
<br />
Pick particles for tomograms 2 and 3. In total, there are around 30-40 particles in the generated dataset. When you open a new tomogram, make sure that you delete the pool of models from memory when asked. This will have no effect onto the models stored in disk, and it is necessary in order to ensure that you are not mixing models from different tomograms.<br />
<br />
==An alternate visualization: orthogonal projections==<br />
<br />
Clicking in the menu on ''Projection -> project full shown fragment along z'' (still in dtmslice) and you will get a screen where the x-y projection of the tomogram is shown. Use the secondary click on it to launch the orthogonal views of x-z and y-z planes that traverse that point. These views can also be used to click particles, in case the standard view is not sufficient.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:orthoPickMain.png|thumb|upright|250px| click on the main projection determines two orthogonal slices]]<br />
| [[File:orthoPickX.png|thumb|upright|250px| Slice xz ]]<br />
| [[File:orthoPickY.png|thumb|upright|250px| Slice xy]]<br />
|}<br />
<br />
=Extracting particles from tomograms=<br />
In the catalogue manager window select the rows for tomograms from which you want to extract particles from. You can either select them one by one with a mouse click and holding the [ctrl] key or by clicking the ''Select all'' button. Then, go to ''Crop Particles -> Open Volume List Manager''.<br />
<br />
[[File:StartersGuideopenVolumeList.png|thumb|center|600px|]]<br />
<br />
A new window opens with all the models in the catalogue listed in the bottom. Pick (by checking boxes) the models of type ''general'' that you have just clicked. Click ''Create list'' and then ''Crop particle''.<br />
<br />
[[File:StartersGuideSelectModels.png|thumb|center|600px|]]<br />
<br />
In the new window, change the data name to something meaningful, such as ''thermosomeParticles'', change the sidelength from 32 to 48 and click start cropping.<br />
<br />
[[File:StartersGuideCrop.png|thumb|center|600px|]]<br />
<br />
After cropping is done, explore your cropped particles by clicking ''ddbrowse'' (in the window above). A new window opens where you can simply click on ''show''.<br />
<br />
[[File:StartersGuideShow.png|thumb|center|500px|]]<br />
<br />
You see a 2D representation (projections) of all cropped 3D subtomograms. Make sure your particles are well centered and fit the box.<br />
<br />
[[File:StartersGuideParticles.png|thumb|center|500px|]]<br />
<br />
We also introduce the concept of the ''dynamo'' table: The information about each particle is stored in tables. Each particle has an entry in the table that contains shifts and rotations to describe its orientation (by default these values are initialized as zeroes). It also contains the particle ID (tag), the orientation of the missing wedge and others. For full info type the command <tt>dthelp</tt>. The ''Dynamo'' catalogue generates an initial table during particle extraction. Its location is in <tt>particles/crop.tbl </tt>. <br />
<br />
=Subtomogram alignment and averaging=<br />
<br />
==Initial reference generation==<br />
We want to align the extracted tomograms to a common reference. For that, we need an initial reference (which will later be refined during the iterative alignment). Here, we create such an initial reference by manually align a few particles and average them. We use the command <tt>dgallery</tt> to generate initial orientations for some of our particles that will be used to generate the initial reference. Close all open ''Dynamo'' windows and type into the ''Dynamo'' command line:<br />
<br />
<tt>dgallery('data','thermosomeParticles.Data');</tt><br />
<br />
The gallery opens. Click on ''load'' to load all aprticles in memory, move the ''shown'' bar to display the particles and use the x-, y- and z-buttons to see different views:<br />
<br />
[[File:StartersGuideGallery.png|thumb|center|600px|]]<br />
<br />
To manually align the particles, do the following for about 10 particles:<br />
# Place the mouse over the particle center and press the key [c] (this centers the particle).<br />
# Place the mouse over its top (or bottom) part and press the key [n] (this aligns the particle).<br />
# Click on the particle to make sure its number turns from red to blue (blue means it is selected for further processing). To de-select a particle use the right-click.<br />
Change between the x-, y- and z-views to correct the orientations if necessary. This is just done to create an initial reference, so orientations don't need to be very exact.<br />
<br />
[[File:StartersGuideGallery2.png|thumb|center|500px|]]<br />
<br />
Save the selected tags and corresponding table by clicking ''quick save'' button in the top right. It saves a quickbuffer.tbl and quickbuffer.tags to the hard drive that will be used later. To generate the average you need to apply the table on the particles. For this click the ''average'' button in the Particle selection field.<br />
<br />
[[File:StartersGuideGallery3.png|thumb|center|500px|]]<br />
<br />
It opens a new window with a lot of controls. Click ''compute average'' in the bottom of the window. Then, right-lick on the output filename and click ''ok'' in the next window to see the result. If you are not satisfied with the result, close the window and refine your manual alignment and/or add more particles to the average.<br />
<br />
[[File:StartersGuideGallery4.png|thumb|center|600px|]]<br />
<br />
[[File:StartersGuideGallery5.png|thumb|center|500px|]]<br />
<br />
==Alignment projects==<br />
Iterative alignment of subtomograms to their average is performed by ''dynamo'' projects that you can run in various high-performance computational environments. To run an alignment project you need the particles, an initial reference and a table. All of these files we generated before. Start the alignment project GUI by typing<br />
<br />
<tt> dcp</tt><br />
<br />
In thw new window, do the following:<br />
# Add the project name ''drun1'', press the ''enter'' key and select ''create a new project'' in the pop-up window.<br />
# Click on ''particles'' and provide the particle folder name ''thermosomeParticles.Data''. Click ok.<br />
# Click on ''table'' and provide the table name ''thermosomeParticles.Data/crop.tbl''. Click ok.<br />
# Click on ''template'' and provide the initial reference name ''my_average.em''. Click ok.<br />
# Click on ''masks'' and simply click on ''use default masks''. Click ok. You could also specify the semi-axis of the ellipsoid masks or go to “Mask editor” to make more sophisticated masks. Note that ''Dynamo'' by default uses Rossman correlation which eliminates the artefacts associated with hard (non-soft) mask. <br />
<br />
[[File:StartersGuideAli1.png|thumb|center|500px|]]<br />
<br />
Click on ''numerical parameters'' to set all parameters and details about the different iterations of the alignment project. You can select any parameter in the table with a mouse and click the ''?'' button on the top right of the parameter window to see a description of the parameters. The most important parameters to consider are:<br />
<br />
* ''Number of iterations'': Make 3 rounds with 2 iterations each. The first round will be a global search with a coarse angular step and the following rounds will be used for refinement.<br />
* ''Angular search ranges'': Cone aperture is the scan range for the first two Euler angles around the initial orientation defined in the table. 360 degrees is the full scan range. Azimuth rotation range defines the rotation range around the new vertical axis of the particle.<br />
* ''High- and lowpass values'': Fourier voxels to limit the used frequency range.<br />
*''Particle dimensions'': Defined as sidelength of your subvolume. If you put a lower value, the particles will be downsampled for the particular round. This will speed up the process.<br />
* ''Refine'': After each angular scan, the search step is reduced by the ''refine factor''. This is repeated ''refine'' times. I.e., if your cone sampling is 10 degrees, ''refine'' is 3 and ''refine factor'' is 2, then 10, 5, 2.5 and 1.25 degrees will be sampled. This is the optimization of angular search space.<br />
*''Shift limits'': Limits the translation of particles from the center of the box (if shifts limiting way is 1) or from the previously estimated center (if shifts limiting way is 2). The previous estimates for the shifts and rotations are taken from input tables and are updated at the end of each iteration.<br />
*''Symmetry'': If you know the symmetry of your protein it will speed up the convergence and result in higher resolution.<br />
<br />
Set the parameters as follows and click ''ok'':<br />
<br />
[[File:StartersGuideFigure10.png|thumb|center|600px|]]<br />
<br />
Click on ''Computing environment'' and select ''standalone''. Set ''CPU cores'' and ''parallelize averaging step'' to the maximum available (see the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have). Leave the rest as it is and click ''ok''. In the alignment GUI click ''check'' and ''unfold''. To run the project open a new terminal (in a new tab), load ''Dynamo'' but do not run it (see [[EMBO_workshop_2021 | workshop page]] on how to load/run ''Dynamo'') and run the project by typing <tt>./drun1.exe</tt>.<br />
<br />
While the project is running, go back to the terminal with ''Dynamo'' running and you can monitor the progress of the alignment project by typing:<br />
<br />
<tt>dvstatus drun1</tt><br />
<br />
While the project is running, you can look at projections of the intermediate results by typing:<br />
<br />
<tt>ddb drun1:a:ite=* -j c10</tt><br />
<br />
or at the latest average by typing:<br />
<br />
<tt>ddb drun1:a -v </tt><br />
<br />
The final result should look similar to:<br />
<br />
[[File:StartersGuideResultsAlignment.png|thumb|center|400px| Results of alignment.]]<br />
<br />
<br />
=Subtomogram classification=<br />
To demonstrate a classification example we provide particles with 2 slightly different sizes. We can then classify them with a simple Multi-reference Analysis (MRA). To do that, we first generate a dataset which has 2 populations of particles. For this, type:<br />
<br />
<tt>dynamo_tutorial('data_classification','M',8,'N',8);</tt><br />
<br />
It generates two sets of particles, 8 particles in each including their table. The idea behind [[Multireference Analysis|MRA]] is the following: The particles are aligned to several references (here only 2). After each iteration, each particle is assigned to only one single reference where it fits the best. This procedure is repeated iteratively. Similar particles will have higher correlation to similar references and will eventually group together. For classification into N classes we will need N initial references and N initially identical tables.Follow these steps to run the MRA:<br />
<br />
# Create a new project by typing “drun2” in the project name of the alignment GUI.<br />
# Set number of references to 2.<br />
# Activate the ''Swap particles'' button.<br />
# Go to ''particles'' and add <tt>data_classification/data</tt> to the particle data. Click ok.<br />
# Go to ''table'' and type <tt>data_classification/real.tbl</tt> into the field ''clone this'' and press ''copy''. Click ok.<br />
# Go to ''template'' and clone <tt>data_classification/original_template.em</tt> with addition of some extra noise with amplitude 1. <br />
# Click on ''masks'' and click on ''use default masks''. Click ok.<br />
# Click on ''numerical parameters'' and set the following parameters. Click ok.<br />
<br />
[[File:StartersGuideResultsClassification2.png|thumb|center|400px|]]<br />
<br />
Choose the same computing environment as in the previous alignment project and run the project in the same way too. MRA projects usually run longer, because each particle will be aligned to two references. Here, however, we chose the parameters in a way that the process is still fast (e.g., small dimensions). Monitor the progress of the project same as before:<br />
<br />
<tt>dvstatus drun2</tt><br />
<br />
And visualize the results using the command:<br />
<br />
<tt>ddb drun2:a:ref=* -j c5</tt><br />
<br />
You should get two averages with particle in one slightly class larger than in the other.<br />
<br />
[[File:StartersGuideResultsClassification.png|thumb|center|600px| Results of classification.]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4179EMBO workshop 20212021-09-06T14:46:34Z<p>Stefano Scaramuzza: </p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough on command line based tilt series alignment | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[advanced starters guide | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&diff=4178EMBO workshop 20212021-09-06T14:46:08Z<p>Stefano Scaramuzza: /* Schedule */</p>
<hr />
<div>The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.<br />
<br />
=Connect to Guacamole and run Dynamo=<br />
=== Credentials ===<br />
Please refer to your personal course material to access your '''two sets''' of credentials:<br />
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.<br />
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.<br />
<br />
=== Connect to Guacamole ===<br />
To connect to Guacamole, first go to the following link in your browser: <br />
<tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt><br />
<br />
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.<br />
<br />
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]<br />
=== Connect to your machine ===<br />
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).<br />
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]<br />
When you select the machine, you will prompted to use your EMBO21 UserID<br />
[[File:embo_2021_3.png|thumb|center|300px|Entering your personal course credentials.]]<br />
This should access your virtual Linux XFCe desktop, which should look like this in your browser<br />
[[File:embo_2021_4.png|thumb|center|300px|Linux XFCe desktop]]<br />
<br />
If you open a terminal (with the terminal icon) you can test by typing<br />
<tt>pwd</tt><br />
that you are in the correct path, which should be:<br />
<tt> /d/embo2021/u/emboXX</tt><br />
where XX stands for your actual workshop participant number.<br />
<br />
=== Load and run Dynamo ===<br />
<br />
First you load the '''standalone''' version of ''Dynamo'' with the command:<br />
<tt> module load dynamo/v1.1.524</tt><br />
and then it can be run by simply typing:<br />
<tt> dynamo</tt><br />
<br />
[[File:embo_2021_5.png|thumb|center|300px|Dynamo console started.]]<br />
<br />
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command<br />
<tt>dynamo_version</tt><br />
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.<br />
<br />
Type the following command to check how many CPU your machine has. You might use this number later for tasks that require parallel processing:<br />
<br />
<tt>mbparse.multicore.checkPhysicalCores</tt><br />
<br />
=== Data location ===<br />
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:<br />
<br />
<tt>cd prac-5</tt><br />
<br />
or <br />
<br />
<tt>cd prac-6</tt><br />
<br />
respectively. Double-check if the data is present by typing<br />
<br />
<tt>ls</tt><br />
<br />
You should see the following files in both directories:<br />
<br />
[[File:embo_files.png|thumb|center|500px|files]]<br />
<br />
During the ''Dynamo'' workshop, work only within those two directories.<br />
<br />
=== Chimera ===<br />
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').<br />
<br />
<tt>/usr/local/bin/chimera myFile.mrc</tt><br />
<br />
= Schedule =<br />
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:<br />
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.<br />
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.<br />
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.<br />
<br />
{| class="wikitable"<br />
! colspan="4" | '''Day 1'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:30<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:30 - 15:30<br />
| Tutorial<br />
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]<br />
|<br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
| <br />
|<br />
|-<br />
| 15:45 - 16:45<br />
| Tutorial<br />
| [[Walkthrough on command line based tilt series alignment | Tilt series alignment with command line]]<br />
| Skip last part "On the fly reconstruction of particles"<br />
|-<br />
| 16:45 - 17:00<br />
| Break<br />
|<br />
|<br />
|-<br />
| 17:00 - 17:45<br />
| Tutorial<br />
| [[Walkthrough_on_filament_models | Models for filaments]]<br />
|<br />
|-<br />
| 17:45 - 18:30<br />
| Tutorial<br />
| [[Walkthrough_on_membrane_models | Models for surfaces]]<br />
|<br />
|-<br />
| Optional:<br />
| Tutorial<br />
| [[Walkthrough for template matching | Template matching]]<br />
|<br />
|-<br />
! colspan="4" | '''Day 2'''<br />
|-<br />
|'''Time'''<br />
|'''Activity<br />
|'''Topic'''<br />
|'''Comment'''<br />
|-<br />
| 14:00 - 14:15<br />
| Presentation<br />
| Introduction<br />
|<br />
|-<br />
| 14:15 - 15:30<br />
| Tutorial<br />
| [[Starters_guide_(EMBO2021) | Starters guide (workshop version)]]<br />
| <br />
|-<br />
| 15:30 - 15:45<br />
| Break<br />
|<br />
|<br />
|-<br />
| 15:45 - 18:15<br />
| Tutorial<br />
| [[advanced starters guide | Advanced starters guide]]<br />
| Take a 15 min break individually during this tutorial<br />
|-<br />
| 18:15 - 18:30<br />
| Presentation<br />
| Closing remarks<br />
|<br />
|-<br />
| Optional:<br />
| Tutorials<br />
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]<br />
|<br />
|}</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Starters_guide_(EMBO2021)&diff=4177Starters guide (EMBO2021)2021-09-06T14:45:23Z<p>Stefano Scaramuzza: </p>
<hr />
<div>This tutorial shows how to manually pick particles in a tomogram, extract them, align and classify them. This tutorial is a shortened version of the original [[Starters_guide | starters guide]] that was adapted for the use in ''Dynamo'' workshops.<br />
<br />
=Manual particle picking and ''Dynamo'' catalogues=<br />
The ''Dynamo'' catalogues are databases that manage tomograms and link the tomographic data to the extracted particles. Start the catalogue manager by typing the command:<br />
<br />
<tt>dcm</tt><br />
After the catalogue manager opens, we create 3 synthetic tomograms that include our particles (thermosomes) in the following way:<br />
<br />
[[File:StartersGuideFigure1.png|thumb|center|600px| Figure 1: creating a set of synthetic tomograms with ''Dynamo'' Catalogue Manager]]<br />
<br />
You can see the list of your tomograms and their metadata in a table in the bottom of your catalogue manager. When workin on your own projects, you can add tomograms to the catalogue by ''Catalogue -> Browse for new volume''. The tomograms are situated in the directory <tt>testCatalogue</tt>. They contain a small amount of noise and a missing wedge associated to rotation around Y-axis. We added the information about location of the particles in the tomograms in an additional catalogue called <tt>testCatalogue_withmodels</tt>. If needed, it can be opened from the current folder with ''Catalogue -> look for local catalogues'' in the catalogue manager window.<br />
<br />
==Viewing tomograms==<br />
Select the first tomogram in the list and go to ''View volume>Full tomogram file in tomoslice''. <br />
<br />
[[ File:StartersSelectTomoSliceInCatalogue.png |thumb| center |600px| Figure 2: Selecting a volume in the catalogue manager to be inspected in <tt>dpreview</tt>]]<br />
<br />
The volume browser tomoslicer loads the entire tomograms into memory and allows making annotations to the regions of interest. The tomograms in this tutorial are small, and you can load them directly into memory. For real life tomograms, bear in mind that you will need to [[Prebinned tomograms| prebin]] the files before loading them into memory. Tomoslice has a simple set of controls and is suitable for visualization tasks that require oblique sections through the tomogram. It uses the same tool as other ''Dynamo'' browsers to keep track of your annotations: a pool of models. You might need to adjust the contrast (blue arrow). To move through the tomogram slices, you can either use the mouse wheel, click and drag the tomogram slice up and down (orange arrows), or move the position control left and right (orange box).<br />
<br />
<br />
[[ File:StartersTomoSlice.png |thumb| center |600px|]]<br />
<br />
==Picking and extracting particles in tomograms==<br />
Coordinates of picked particles are represented by data types called models. In the tomoslicer window go to ''Model Pool -> Create new model in pool (choose type) -> General''. This is the simplest type of [[model]] where each clicked/model point corresponds to a single isolated particle. Now you can navigate up and down the tomogram, place the mouse on the center of a particle and press the [c] key on your keyboard to add a new model point. (Note the ''Help -> all hot keys'' options that lists the different the actions of the different keystrokes). Backspace button deletes the last clicked point (you can also use the right-click to delete single points).<br />
<br />
[[File:StartersGuideFigure4.png|thumb|center|600px| Generating a new model]]<br />
<br />
After you are done clicking about 10 particles click on ''Active model > Update Crop points'' in the tomoslice window.<br />
<br />
<br />
[[File:StartersGuideUpdate.png|thumb|center|600px|]]<br />
<br />
Save the model into the catalogue by ''Active model -> Save active model into catalogue (disk)'' and close the slicer window.<br />
<br />
[[File:StartersGuideFigure6.png|thumb|center|600px| ]]<br />
<br />
Pick particles for tomograms 2 and 3. In total, there are around 30-40 particles in the generated dataset. When you open a new tomogram, make sure that you delete the pool of models from memory when asked. This will have no effect onto the models stored in disk, and it is necessary in order to ensure that you are not mixing models from different tomograms.<br />
<br />
==An alternate visualization: orthogonal projections==<br />
<br />
Clicking in the menu on ''Projection -> project full shown fragment along z'' (still in dtmslice) and you will get a screen where the x-y projection of the tomogram is shown. Use the secondary click on it to launch the orthogonal views of x-z and y-z planes that traverse that point. These views can also be used to click particles, in case the standard view is not sufficient.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:orthoPickMain.png|thumb|upright|250px| click on the main projection determines two orthogonal slices]]<br />
| [[File:orthoPickX.png|thumb|upright|250px| Slice xz ]]<br />
| [[File:orthoPickY.png|thumb|upright|250px| Slice xy]]<br />
|}<br />
<br />
=Extracting particles from tomograms=<br />
In the catalogue manager window select the rows for tomograms from which you want to extract particles from. You can either select them one by one with a mouse click and holding the [ctrl] key or by clicking the ''Select all'' button. Then, go to ''Crop Particles -> Open Volume List Manager''.<br />
<br />
[[File:StartersGuideopenVolumeList.png|thumb|center|600px|]]<br />
<br />
A new window opens with all the models in the catalogue listed in the bottom. Pick (by checking boxes) the models of type ''general'' that you have just clicked. Click ''Create list'' and then ''Crop particle''.<br />
<br />
[[File:StartersGuideSelectModels.png|thumb|center|600px|]]<br />
<br />
In the new window, change the data name to something meaningful, such as ''thermosomeParticles'', change the sidelength from 32 to 48 and click start cropping.<br />
<br />
[[File:StartersGuideCrop.png|thumb|center|600px|]]<br />
<br />
After cropping is done, explore your cropped particles by clicking ''ddbrowse'' (in the window above). A new window opens where you can simply click on ''show''.<br />
<br />
[[File:StartersGuideShow.png|thumb|center|500px|]]<br />
<br />
You see a 2D representation (projections) of all cropped 3D subtomograms. Make sure your particles are well centered and fit the box.<br />
<br />
[[File:StartersGuideParticles.png|thumb|center|500px|]]<br />
<br />
We also introduce the concept of the ''dynamo'' table: The information about each particle is stored in tables. Each particle has an entry in the table that contains shifts and rotations to describe its orientation (by default these values are initialized as zeroes). It also contains the particle ID (tag), the orientation of the missing wedge and others. For full info type the command <tt>dthelp</tt>. The ''Dynamo'' catalogue generates an initial table during particle extraction. Its location is in <tt>particles/crop.tbl </tt>. <br />
<br />
=Subtomogram alignment and averaging=<br />
<br />
==Initial reference generation==<br />
We want to align the extracted tomograms to a common reference. For that, we need an initial reference (which will later be refined during the iterative alignment). Here, we create such an initial reference by manually align a few particles and average them. We use the command <tt>dgallery</tt> to generate initial orientations for some of our particles that will be used to generate the initial reference. Close all open ''Dynamo'' windows and type into the ''Dynamo'' command line:<br />
<br />
<tt>dgallery('data','thermosomeParticles.Data');</tt><br />
<br />
The gallery opens. Click on ''load'' to load all aprticles in memory, move the ''shown'' bar to display the particles and use the x-, y- and z-buttons to see different views:<br />
<br />
[[File:StartersGuideGallery.png|thumb|center|600px|]]<br />
<br />
To manually align the particles, do the following for about 10 particles:<br />
# Place the mouse over the particle center and press the key [c] (this centers the particle).<br />
# Place the mouse over its top (or bottom) part and press the key [n] (this aligns the particle).<br />
# Click on the particle to make sure its number turns from red to blue (blue means it is selected for further processing). To de-select a particle use the right-click.<br />
Change between the x-, y- and z-views to correct the orientations if necessary. This is just done to create an initial reference, so orientations don't need to be very exact.<br />
<br />
[[File:StartersGuideGallery2.png|thumb|center|500px|]]<br />
<br />
Save the selected tags and corresponding table by clicking ''quick save'' button in the top right. It saves a quickbuffer.tbl and quickbuffer.tags to the hard drive that will be used later. To generate the average you need to apply the table on the particles. For this click the ''average'' button in the Particle selection field.<br />
<br />
[[File:StartersGuideGallery3.png|thumb|center|500px|]]<br />
<br />
It opens a new window with a lot of controls. Click ''compute average'' in the bottom of the window. Then, right-lick on the output filename and click ''ok'' in the next window to see the result. If you are not satisfied with the result, close the window and refine your manual alignment and/or add more particles to the average.<br />
<br />
[[File:StartersGuideGallery4.png|thumb|center|600px|]]<br />
<br />
[[File:StartersGuideGallery5.png|thumb|center|500px|]]<br />
<br />
==Alignment projects==<br />
Iterative alignment of subtomograms to their average is performed by ''dynamo'' projects that you can run in various high-performance computational environments. To run an alignment project you need the particles, an initial reference and a table. All of these files we generated before. Start the alignment project GUI by typing<br />
<br />
<tt> dcp</tt><br />
<br />
In thw new window, do the following:<br />
# Add the project name ''drun1'', press the ''enter'' key and select ''create a new project'' in the pop-up window.<br />
# Click on ''particles'' and provide the particle folder name ''thermosomeParticles.Data''. Click ok.<br />
# Click on ''table'' and provide the table name ''thermosomeParticles.Data/crop.tbl''. Click ok.<br />
# Click on ''template'' and provide the initial reference name ''my_average.em''. Click ok.<br />
# Click on ''masks'' and simply click on ''use default masks''. Click ok. You could also specify the semi-axis of the ellipsoid masks or go to “Mask editor” to make more sophisticated masks. Note that ''Dynamo'' by default uses Rossman correlation which eliminates the artefacts associated with hard (non-soft) mask. <br />
<br />
[[File:StartersGuideAli1.png|thumb|center|500px|]]<br />
<br />
Click on ''numerical parameters'' to set all parameters and details about the different iterations of the alignment project. You can select any parameter in the table with a mouse and click the ''?'' button on the top right of the parameter window to see a description of the parameters. The most important parameters to consider are:<br />
<br />
* ''Number of iterations'': Make 3 rounds with 2 iterations each. The first round will be a global search with a coarse angular step and the following rounds will be used for refinement.<br />
* ''Angular search ranges'': Cone aperture is the scan range for the first two Euler angles around the initial orientation defined in the table. 360 degrees is the full scan range. Azimuth rotation range defines the rotation range around the new vertical axis of the particle.<br />
* ''High- and lowpass values'': Fourier voxels to limit the used frequency range.<br />
*''Particle dimensions'': Defined as sidelength of your subvolume. If you put a lower value, the particles will be downsampled for the particular round. This will speed up the process.<br />
* ''Refine'': After each angular scan, the search step is reduced by the ''refine factor''. This is repeated ''refine'' times. I.e., if your cone sampling is 10 degrees, ''refine'' is 3 and ''refine factor'' is 2, then 10, 5, 2.5 and 1.25 degrees will be sampled. This is the optimization of angular search space.<br />
*''Shift limits'': Limits the translation of particles from the center of the box (if shifts limiting way is 1) or from the previously estimated center (if shifts limiting way is 2). The previous estimates for the shifts and rotations are taken from input tables and are updated at the end of each iteration.<br />
*''Symmetry'': If you know the symmetry of your protein it will speed up the convergence and result in higher resolution.<br />
<br />
Set the parameters as follows and click ''ok'':<br />
<br />
[[File:StartersGuideFigure10.png|thumb|center|600px|]]<br />
<br />
Click on ''Computing environment'' and select ''standalone''. Set ''CPU cores'' and ''parallelize averaging step'' to the maximum available (see the [[EMBO_workshop_2021 | workshop page]] to check how many processors you have). Leave the rest as it is and click ''ok''. In the alignment GUI click ''check'' and ''unfold''. To run the project open a new terminal (in a new tab), load ''Dynamo'' (but do not run it) and run the project by typing <tt>./drun1.exe</tt>.<br />
<br />
While the project is running, go back to the terminal with ''Dynamo'' running and you can monitor the progress of the alignment project by typing:<br />
<br />
<tt>dvstatus drun1</tt><br />
<br />
While the project is running, you can look at projections of the intermediate results by typing:<br />
<br />
<tt>ddb drun1:a:ite=* -j c10</tt><br />
<br />
or at the latest average by typing:<br />
<br />
<tt>ddb drun1:a -v </tt><br />
<br />
The final result should look similar to:<br />
<br />
[[File:StartersGuideResultsAlignment.png|thumb|center|400px| Results of alignment.]]<br />
<br />
<br />
=Subtomogram classification=<br />
To demonstrate a classification example we provide particles with 2 slightly different sizes. We can then classify them with a simple Multi-reference Analysis (MRA). To do that, we first generate a dataset which has 2 populations of particles. For this, type:<br />
<br />
<tt>dynamo_tutorial('data_classification','M',8,'N',8);</tt><br />
<br />
It generates two sets of particles, 8 particles in each including their table. The idea behind [[Multireference Analysis|MRA]] is the following: The particles are aligned to several references (here only 2). After each iteration, each particle is assigned to only one single reference where it fits the best. This procedure is repeated iteratively. Similar particles will have higher correlation to similar references and will eventually group together. For classification into N classes we will need N initial references and N initially identical tables.Follow these steps to run the MRA:<br />
<br />
# Create a new project by typing “drun2” in the project name of the alignment GUI.<br />
# Set number of references to 2.<br />
# Activate the ''Swap particles'' button.<br />
# Go to ''particles'' and add <tt>data_classification/data</tt> to the particle data. Click ok.<br />
# Go to ''table'' and type <tt>data_classification/real.tbl</tt> into the field ''clone this'' and press ''copy''. Click ok.<br />
# Go to ''template'' and clone <tt>data_classification/original_template.em</tt> with addition of some extra noise with amplitude 1. <br />
# Click on ''masks'' and click on ''use default masks''. Click ok.<br />
# Click on ''numerical parameters'' and set the following parameters. Click ok.<br />
<br />
[[File:StartersGuideResultsClassification2.png|thumb|center|400px|]]<br />
<br />
Choose the same computing environment as in the previous alignment project and run the project in the same way too. MRA projects usually run longer, because each particle will be aligned to two references. Here, however, we chose the parameters in a way that the process is still fast (e.g., small dimensions). Monitor the progress of the project same as before:<br />
<br />
<tt>dvstatus drun2</tt><br />
<br />
And visualize the results using the command:<br />
<br />
<tt>ddb drun2:a:ref=* -j c5</tt><br />
<br />
You should get two averages with particle in one slightly class larger than in the other.<br />
<br />
[[File:StartersGuideResultsClassification.png|thumb|center|600px| Results of classification.]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Starters_guide_(EMBO2021)&diff=4176Starters guide (EMBO2021)2021-09-06T14:40:00Z<p>Stefano Scaramuzza: Created page with "This tutorial shows how to manually pick particles in a tomogram, extract them, align and classify them. This tutorial is a shortened version of the original Starters_guide..."</p>
<hr />
<div>This tutorial shows how to manually pick particles in a tomogram, extract them, align and classify them. This tutorial is a shortened version of the original [[Starters_guide | starters guide]] that was adapted for the use in ''Dynamo'' workshops.<br />
<br />
=Manual particle picking and ''Dynamo'' catalogues=<br />
The ''Dynamo'' catalogues are databases that manage tomograms and link the tomographic data to the extracted particles. Start the catalogue manager by typing the command:<br />
<br />
<tt>dcm</tt><br />
After the catalogue manager opens, we create 3 synthetic tomograms that include our particles (thermosomes) in the following way:<br />
<br />
[[File:StartersGuideFigure1.png|thumb|center|600px| Figure 1: creating a set of synthetic tomograms with ''Dynamo'' Catalogue Manager]]<br />
<br />
You can see the list of your tomograms and their metadata in a table in the bottom of your catalogue manager. When workin on your own projects, you can add tomograms to the catalogue by ''Catalogue -> Browse for new volume''. The tomograms are situated in the directory <tt>testCatalogue</tt>. They contain a small amount of noise and a missing wedge associated to rotation around Y-axis. We added the information about location of the particles in the tomograms in an additional catalogue called <tt>testCatalogue_withmodels</tt>. If needed, it can be opened from the current folder with ''Catalogue -> look for local catalogues'' in the catalogue manager window.<br />
<br />
==Viewing tomograms==<br />
Select the first tomogram in the list and go to ''View volume>Full tomogram file in tomoslice''. <br />
<br />
[[ File:StartersSelectTomoSliceInCatalogue.png |thumb| center |600px| Figure 2: Selecting a volume in the catalogue manager to be inspected in <tt>dpreview</tt>]]<br />
<br />
The volume browser tomoslicer loads the entire tomograms into memory and allows making annotations to the regions of interest. The tomograms in this tutorial are small, and you can load them directly into memory. For real life tomograms, bear in mind that you will need to [[Prebinned tomograms| prebin]] the files before loading them into memory. Tomoslice has a simple set of controls and is suitable for visualization tasks that require oblique sections through the tomogram. It uses the same tool as other ''Dynamo'' browsers to keep track of your annotations: a pool of models. You might need to adjust the contrast (blue arrow). To move through the tomogram slices, you can either use the mouse wheel, click and drag the tomogram slice up and down (orange arrows), or move the position control left and right (orange box).<br />
<br />
<br />
[[ File:StartersTomoSlice.png |thumb| center |600px|]]<br />
<br />
==Picking and extracting particles in tomograms==<br />
Coordinates of picked particles are represented by data types called models. In the tomoslicer window go to ''Model Pool -> Create new model in pool (choose type) -> General''. This is the simplest type of [[model]] where each clicked/model point corresponds to a single isolated particle. Now you can navigate up and down the tomogram, place the mouse on the center of a particle and press the [c] key on your keyboard to add a new model point. (Note the ''Help -> all hot keys'' options that lists the different the actions of the different keystrokes). Backspace button deletes the last clicked point (you can also use the right-click to delete single points).<br />
<br />
[[File:StartersGuideFigure4.png|thumb|center|600px| Generating a new model]]<br />
<br />
After you are done clicking about 10 particles click on ''Active model > Update Crop points'' in the tomoslice window.<br />
<br />
<br />
[[File:StartersGuideUpdate.png|thumb|center|600px|]]<br />
<br />
Save the model into the catalogue by ''Active model -> Save active model into catalogue (disk)'' and close the slicer window.<br />
<br />
[[File:StartersGuideFigure6.png|thumb|center|600px| ]]<br />
<br />
Pick particles for tomograms 2 and 3. In total, there are around 30-40 particles in the generated dataset. When you open a new tomogram, make sure that you delete the pool of models from memory when asked. This will have no effect onto the models stored in disk, and it is necessary in order to ensure that you are not mixing models from different tomograms.<br />
<br />
==An alternate visualization: orthogonal projections==<br />
<br />
Clicking in the menu on ''Projection -> project full shown fragment along z'' (still in dtmslice) and you will get a screen where the x-y projection of the tomogram is shown. Use the secondary click on it to launch the orthogonal views of x-z and y-z planes that traverse that point. These views can also be used to click particles, in case the standard view is not sufficient.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:orthoPickMain.png|thumb|upright|250px| click on the main projection determines two orthogonal slices]]<br />
| [[File:orthoPickX.png|thumb|upright|250px| Slice xz ]]<br />
| [[File:orthoPickY.png|thumb|upright|250px| Slice xy]]<br />
|}<br />
<br />
=Extracting particles from tomograms=<br />
In the catalogue manager window select the rows for tomograms from which you want to extract particles from. You can either select them one by one with a mouse click and holding the [ctrl] key or by clicking the ''Select all'' button. Then, go to ''Crop Particles -> Open Volume List Manager''.<br />
<br />
[[File:StartersGuideopenVolumeList.png|thumb|center|600px|]]<br />
<br />
A new window opens with all the models in the catalogue listed in the bottom. Pick (by checking boxes) the models of type ''general'' that you have just clicked. Click ''Create list'' and then ''Crop particle''.<br />
<br />
[[File:StartersGuideSelectModels.png|thumb|center|600px|]]<br />
<br />
In the new window, change the data name to something meaningful, such as ''thermosomeParticles'', change the sidelength from 32 to 48 and click start cropping.<br />
<br />
[[File:StartersGuideCrop.png|thumb|center|600px|]]<br />
<br />
After cropping is done, explore your cropped particles by clicking ''ddbrowse'' (in the window above). A new window opens where you can simply click on ''show''.<br />
<br />
[[File:StartersGuideShow.png|thumb|center|500px|]]<br />
<br />
You see a 2D representation (projections) of all cropped 3D subtomograms. Make sure your particles are well centered and fit the box.<br />
<br />
[[File:StartersGuideParticles.png|thumb|center|500px|]]<br />
<br />
We also introduce the concept of the ''dynamo'' table: The information about each particle is stored in tables. Each particle has an entry in the table that contains shifts and rotations to describe its orientation (by default these values are initialized as zeroes). It also contains the particle ID (tag), the orientation of the missing wedge and others. For full info type the command <tt>dthelp</tt>. The ''Dynamo'' catalogue generates an initial table during particle extraction. Its location is in <tt>particles/crop.tbl </tt>. <br />
<br />
=Subtomogram alignment and averaging=<br />
<br />
==Initial reference generation==<br />
We want to align the extracted tomograms to a common reference. For that, we need an initial reference (which will later be refined during the iterative alignment). Here, we create such an initial reference by manually align a few particles and average them. We use the command <tt>dgallery</tt> to generate initial orientations for some of our particles that will be used to generate the initial reference. Close all open ''Dynamo'' windows and type into the ''Dynamo'' command line:<br />
<br />
<tt>dgallery('data','thermosomeParticles.Data');</tt><br />
<br />
The gallery opens. Click on ''load'' to load all aprticles in memory, move the ''shown'' bar to display the particles and use the x-, y- and z-buttons to see different views:<br />
<br />
[[File:StartersGuideGallery.png|thumb|center|600px|]]<br />
<br />
To manually align the particles, do the following for about 10 particles:<br />
# Place the mouse over the particle center and press the key [c] (this centers the particle).<br />
# Place the mouse over its top (or bottom) part and press the key [n] (this aligns the particle).<br />
# Click on the particle to make sure its number turns from red to blue (blue means it is selected for further processing). To de-select a particle use the right-click.<br />
Change between the x-, y- and z-views to correct the orientations if necessary. This is just done to create an initial reference, so orientations don't need to be very exact.<br />
<br />
[[File:StartersGuideGallery2.png|thumb|center|500px|]]<br />
<br />
Save the selected tags and corresponding table by clicking ''quick save'' button in the top right. It saves a quickbuffer.tbl and quickbuffer.tags to the hard drive that will be used later. To generate the average you need to apply the table on the particles. For this click the ''average'' button in the Particle selection field.<br />
<br />
[[File:StartersGuideGallery3.png|thumb|center|500px|]]<br />
<br />
It opens a new window with a lot of controls. Click ''compute average'' in the bottom of the window. Then, right-lick on the output filename and click ''ok'' in the next window to see the result. If you are not satisfied with the result, close the window and refine your manual alignment and/or add more particles to the average.<br />
<br />
[[File:StartersGuideGallery4.png|thumb|center|600px|]]<br />
<br />
[[File:StartersGuideGallery5.png|thumb|center|500px|]]<br />
<br />
==Alignment projects==<br />
Iterative alignment of subtomograms to their average is performed by ''dynamo'' projects that you can run in various high-performance computational environments. To run an alignment project you need the particles, an initial reference and a table. All of these files we generated before. Start the alignment project GUI by typing<br />
<br />
<tt> dcp</tt><br />
<br />
In thw new window, do the following:<br />
# Add the project name ''drun1'', press the ''enter'' key and select ''create a new project'' in the pop-up window.<br />
# Click on ''particles'' and provide the particle folder name ''thermosomeParticles.Data''. Click ok.<br />
# Click on ''table'' and provide the table name ''thermosomeParticles.Data/crop.tbl''. Click ok.<br />
# Click on ''template'' and provide the initial reference name ''my_average.em''. Click ok.<br />
# Click on ''masks'' and simply click on ''use default masks''. Click ok. You could also specify the semi-axis of the ellipsoid masks or go to “Mask editor” to make more sophisticated masks. Note that ''Dynamo'' by default uses Rossman correlation which eliminates the artefacts associated with hard (non-soft) mask. <br />
<br />
[[File:StartersGuideAli1.png|thumb|center|500px|]]<br />
<br />
Click on ''numerical parameters'' to set all parameters and details about the different iterations of the alignment project. You can select any parameter in the table with a mouse and click the ''?'' button on the top right of the parameter window to see a description of the parameters. The most important parameters to consider are:<br />
<br />
* ''Number of iterations'': Make 3 rounds with 2 iterations each. The first round will be a global search with a coarse angular step and the following rounds will be used for refinement.<br />
* ''Angular search ranges'': Cone aperture is the scan range for the first two Euler angles around the initial orientation defined in the table. 360 degrees is the full scan range. Azimuth rotation range defines the rotation range around the new vertical axis of the particle.<br />
* ''High- and lowpass values'': Fourier voxels to limit the used frequency range.<br />
*''Particle dimensions'': Defined as sidelength of your subvolume. If you put a lower value, the particles will be downsampled for the particular round. This will speed up the process.<br />
* ''Refine'': After each angular scan, the search step is reduced by the ''refine factor''. This is repeated ''refine'' times. I.e., if your cone sampling is 10 degrees, ''refine'' is 3 and ''refine factor'' is 2, then 10, 5, 2.5 and 1.25 degrees will be sampled. This is the optimization of angular search space.<br />
*''Shift limits'': Limits the translation of particles from the center of the box (if shifts limiting way is 1) or from the previously estimated center (if shifts limiting way is 2). The previous estimates for the shifts and rotations are taken from input tables and are updated at the end of each iteration.<br />
*''Symmetry'': If you know the symmetry of your protein it will speed up the convergence and result in higher resolution.<br />
<br />
Set the parameters as follows and click ''ok'':<br />
<br />
[[File:StartersGuideFigure10.png|thumb|center|600px|]]<br />
<br />
Click on ''Computing environment'' and select your computing environment (choose ''standalone'' if you are working on the standalone version during a workshop). Set ''CPU cores'' and ''parallelize averaging step'' to the maximum available (e.g., 24). Leave the rest as it is and click ''ok''. In the alignment GUI click ''check'' and ''unfold''. To run the project<br />
<br />
* in a Matlab session, you can just click on “run”. <br />
* in a standalone project it is more convenient to open a new terminal, load ''Dynamo'' (but do not run it) and run the project by typing <tt>./drun1.exe</tt>.<br />
<br />
While the project is running, open another Matlab window (or in standalone open another terminal with ''Dynamo'' running) and you can monitor the progress of the alignment project by typing:<br />
<br />
<tt>dvstatus drun1</tt><br />
<br />
While the project is running, you can look at projections of the intermediate results by typing:<br />
<br />
<tt>ddb drun1:a:ite=* -j c10</tt><br />
<br />
or at the latest average by typing:<br />
<br />
<tt>ddb drun1:a -v </tt><br />
<br />
The final result should look similar to:<br />
<br />
[[File:StartersGuideResultsAlignment.png|thumb|center|400px| Results of alignment.]]<br />
<br />
<br />
=Subtomogram classification=<br />
To demonstrate a classification example we provide particles with 2 slightly different sizes. We can then classify them with a simple Multi-reference Analysis (MRA). To do that, we first generate a dataset which has 2 populations of particles. For this, type:<br />
<br />
<tt>dynamo_tutorial('data_classification','M',8,'N',8);</tt><br />
<br />
It generates two sets of particles, 8 particles in each including their table. The idea behind [[Multireference Analysis|MRA]] is the following: The particles are aligned to several references (here only 2). After each iteration, each particle is assigned to only one single reference where it fits the best. This procedure is repeated iteratively. Similar particles will have higher correlation to similar references and will eventually group together. For classification into N classes we will need N initial references and N initially identical tables.Follow these steps to run the MRA:<br />
<br />
# Create a new project by typing “drun2” in the project name of the alignment GUI.<br />
# Set number of references to 2.<br />
# Activate the ''Swap particles'' button.<br />
# Go to ''particles'' and add <tt>data_classification/data</tt> to the particle data. Click ok.<br />
# Go to ''table'' and type <tt>data_classification/real.tbl</tt> into the field ''clone this'' and press ''copy''. Click ok.<br />
# Go to ''template'' and clone <tt>data_classification/original_template.em</tt> with addition of some extra noise with amplitude 1. <br />
# Click on ''masks'' and click on ''use default masks''. Click ok.<br />
# Click on ''numerical parameters'' and set the following parameters. Click ok.<br />
<br />
[[File:StartersGuideResultsClassification2.png|thumb|center|400px|]]<br />
<br />
Choose the same computing environment as in the previous alignment project and run the project in the same way too. MRA projects usually run longer, because each particle will be aligned to two references. Here, however, we chose the parameters in a way that the process is still fast (e.g., small dimensions). Monitor the progress of the project same as before:<br />
<br />
<tt>dvstatus drun2</tt><br />
<br />
And visualize the results using the command:<br />
<br />
<tt>ddb drun2:a:ref=* -j c5</tt><br />
<br />
You should get two averages with particle in one slightly class larger than in the other.<br />
<br />
[[File:StartersGuideResultsClassification.png|thumb|center|600px| Results of classification.]]</div>Stefano Scaramuzzahttps://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=Starters_guide&diff=4175Starters guide2021-09-06T14:39:45Z<p>Stefano Scaramuzza: </p>
<hr />
<div><br />
For ''Dynamo'' version 1.1.115 or higher.<br />
<br />
This tutorial covers the basic steps to get started with ''Dynamo'':<br />
* Installation<br />
* Tomogram visualization and extraction of particles<br />
* Classification<br />
* Avoiding overfitting via independent half dataset refinement (“gold standard processing”)<br />
<br />
Streamlined workshop version of this tutorial: [[Starters_guide_(workshop_version)| here]].<br />
<br />
==Introduction==<br />
''Dynamo'' is a Matlab based set of scripts and GUIs designed to perform subtomogram averaging from cryo electron tomograms. This document describes the basics including particle picking, initial model generation, alignment, averaging and classification or particles. ''Dynamo'' has a standalone to be run in high performance computational systems (CPU, GPU and hybrid clusters). However for most of the tasks we find it satisfying to have a multi-CPU desktop with several Nvidia Graphical Processors (GPUs) and Matlab.<br />
<br />
===Installation===<br />
[[Downloads|Download]] ''Dynamo'' and copy the .tar file to your desired [[installation]] directory <tt><DYNAMO_ROOT></tt> and untar it:<br />
<br />
<tt>tar -xvf dynamo-v-1.1.50_MCR-8.2.0_GLNXA64.tar </tt><br />
<br />
(for Linux and Mac systems or use a visual tool as 7-zip under Windows)<br />
Follow the instructions in the file <tt>README_dynamo_installation.txt </tt>to finish the [[Installation|installation]] depending on your system. If you want to use GPU computing for this walkthrough, (installation in abstract 4 of README), please make sure you have CUDA5.0 drivers installed in your system. Alternatively you can run this tutorial on in multicore mode without GPU, for this you will need a Matlab version 2014b or higher and a Parallel Computing Toolbox, or a [[standalone]] installation. <br />
<br />
==== Running in Matlab ====<br />
<br />
Start Matlab in your system (can take some time), an inside the Matlab terminal, type<br />
<br />
<tt>>> run <DYNAMO_ROOT>/dynamo_activate.m</tt><br />
<br />
Dynamo is now ready to use. Make a new directory for your project (>> mkdir dtutorial) and go there<br />
(>> cd dtutorial). Type >> dynamo and a command window will appear.<br />
This tutorial focuses on the basic GUI functionality however the real power of ''Dynamo'' lays in its flexibility supported by Matlab scripting. We find it useful to understand the basics of Matlab: variables, conditions, loops, functions, plots. For the advanced use we recommend to go through Getting Started With Matlab (http://www.mathworks.ch/help/pdf_doc/matlab/getstart.pdf).<br />
<br />
''Dynamo'' has a help center that may be called by typing <br />
<br />
<tt>>> dhelp</tt><br />
<br />
It contains descriptions for all the dynamo functions that may be also called from the Matlab command line with customized parameters. You can also type >> help dynamo_function_name for the lists of parameters for each function. Help center also contains descriptions of data types (list of database items). Finally you can view a list of PDF tutorials with examples that demonstrate advanced usage of ''Dynamo''. Note that simple copy-pasting of some commands from PDF into Matlab may not work under Linux due to special characters. Another way to access documentation is to ask ''Dynamo'' about it. Type >> dapropos to get the list of potential entries for the subject. You can use it without parameters get the list of topics or ask about a particular topic like >> dapropos table. More advanced users can use >>dlookfor to find functions containing a given string in their name or code.<br />
<br />
==== As standalone ====<br />
In a Linux/Mac shell, type:<br />
<br />
<tt>>> source <DYNAMO_ROOT>/dynamo_activate_linux_shippedMCR.sh</tt><br />
<br />
==Particle picking and ''Dynamo'' catalogues==<br />
We assume that your tomograms are already generated using external packages (i.e. IMOD, Protomo, Inspect3D, TOM); for this tutorial ''Dynamo'' will generate simulated data. The ''Dynamo'' catalogues are databases that manage tomograms and link the tomographic data to the extracted particles. Start [[catalogue manager|Dcm GUI ]] from the ''Dynamo'' command window.<br />
<br />
<tt>dcm</tt><br />
<br />
This command will work equally in Matlab and in the standalone, producing the [[Dcm GUI]]. Once it opens, in the catalogue manager window use menu ''Help -> Create synthetic examples -> Create a synthetic catalogue''.<br />
<br />
[[File:StartersGuideFigure1.png|thumb|center|600px| Figure 1: creating a set of synthetic tomograms with ''Dynamo'' Catalogue Manager]]<br />
<br />
''Dynamo'' will generate 3 tomograms of synthetic thermosomes and will put them in a new folder called <tt>testCatalogue</tt> along with the ground truth for particle locations and orientations. Tomograms contain small amount of noise and a missing wedge associated to rotation around Y-axis. Dynamo stores the information about location of particles in the tomograms in a catalogue called <tt>testCatalogue_withmodels</tt>. Existing catalogues from the current folder may be recalled with ''Catalogue -> look for local catalogues'' in the catalogue manager window.<br />
<br />
===Viewing tomograms===<br />
You can see the list of your tomograms and metadata in a table in the bottom of your [[dcm | catalogue manager window]]. You can add tomograms to the catalogue by ''Catalogue -> Browse for new volume''. You can get an overview of your tomograms in the catalogue by ''View all volumes -> create thumbnails'' and ''View all volumes -> show thumbnail gallery''. All these functionalities have their corresponding command line versions, accessible through the <tt>dcm</tt> command (short for <tt>dynamo_catalogue_manager</tt>), whose options and syntax can be consulted through <tt>>> doc dcm</tt>.<br />
''Dynamo'' has several viewers designed for various purposes. Select the first tomogram in the list with a secondary mouse click and go to ''View volume>Full tomogram file in tomoslice'' . <br />
<br />
[[ File:StartersSelectTomoSliceInCatalogue.png |thumb| center |600px| Figure 2: Selecting a volume in the catalogue manager to be inspected in <tt>dpreview</tt>]]<br />
<br />
The volume browser [[dtmslice | tomoslicer]] loads the entire tomograms into memory and allow making annotations to the regions of interest. The tomograms in this tutorial are small, and you can load them directly into memory. For real life tomograms, bear in mind that you will need to [[Prebinned tomograms| prebin]] the files before loading them into memory (or alternatively, load fragments of them using <tt>dpreview</tt>)<br />
<br />
Tomoslice has a simple set of controls, and is suitable for visualization tasks that require oblique sections through the tomogram. It uses the same tool as other ''Dynamo'' browsers to keep track of your annotations: a pool of models.<br />
<br />
Go to ''View volume -> load full tomogram into Tomoslice''. On the left control panels Scene turn off the 3d view radiobutton. You can navigate the z-height of the tomogram with a mouse wheel or with a slider in the tomoslice menu (1 to 256 in this case); you can also change the slice thickness from the default 10 pixels. Play with the controls and explore the tomogram. Turn 3d view back on.<br />
<br />
===Picking particles in tomograms===<br />
Coordinates of picked particles are represented by data types called models. In the tomoslicer window go to ''Model Pool -> Create new model in pool (choose type) -> General''. This is the simplest type of [[model]] where each clicked/model point corresponds to a single isolated particle. Now you can navigate up and down the tomogram, center the mouse on the particle and click “C” to add a new model point. (Note the ''Help -> Shortcut'' options that lists the different the actions of the different keystrokes). Backspace button deletes the last clicked point.<br />
<br />
[[File:StartersGuideFigure4.png|thumb|center|600px| Figure 4. Generating a new model]]<br />
<br />
After you are done clicking a dozen of particles go to ''Active model -> Edit active model''. This is a summary of the model with control elements for geometric transformations. Models of type general don’t allow any geometrical transforms, all you need to do here is to convert the clicked model points into “table points” that will be used for cropping. This is done by clicking the table-looking command icon (4th from the right in the top left part of the Editor of model parameters window). Alternatively, you could just click on ''Active model > Update Crop points'' in the tomoslice window.<br />
<br />
[[File:StartersGuideFigure5.png|thumb|center|600px| Figure 5. Updating crop points and extracting particles from the model editor window]]<br />
<br />
The model editor also gives you the possibility of changing the name of the model. You may want to give the models meaningful identifiers before storing them into disk. Also, you can open a GUI for extracting your particles (first icon from the right) to extract particles from only this tomogram, however we will do it later for all 3 tomograms. Make sure that the number of clicked points equals to the number of table points in the Information field and close the Editor window. Importantly, save the model into the catalogue by ''Active model -> Save active model into catalogue (disk)'' and close the slicer window. Pick particles for tomograms 2 and 3. In total, there are around 40 particles in the generated dataset.<br />
<br />
[[File:StartersGuideFigure6.png|thumb|center|600px| Figure 6. Remember to save your work: use <tt>Update Crop</tt> to create table points, and then save the model into the catalogue.]]<br />
<br />
Important: when you open a new tomogram, make sure that you delete the pool of models from memory (model pool > clear current pool from memory). This will have no effect onto the models stored in disk, and it is necessary in order to ensure that you are not mixing models from different tomograms.<br />
<br />
===An alternate visualization: orthogonal projections===<br />
<br />
Clicking on <tt>Projection</tt> you will get a screen where the x-y projection of the tomogram is shown. Use the secondary click on it to launch the orthogonal views of x-z and y-z planes that traverse that point. Clicking on those views determines a 3D point.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:orthoPickMain.png|thumb|upright|250px| click on the main projection determines two orthogonal slices]]<br />
| [[File:orthoPickX.png|thumb|upright|250px| Slice xz ]]<br />
| [[File:orthoPickY.png|thumb|upright|250px| Slice xy]]<br />
|}<br />
<br />
===Extracting particles from tomograms===<br />
In the catalogue manager window select the rows for tomograms from which you want to extract particles from. You can either select them one by one with a mouse click holding a CTRL button or click the “Select all” button.<br />
<br />
[[File:StartersGuideFigure8.png|thumb|center|600px| Figure 8. Extracting particles with DCM and selecting the models for particle extraction.]]<br />
<br />
Go to ''Crop Particles -> Open Volume List Manager'', all the models in the catalogue listed in the bottom of the window. Pick (by checking boxes) the models of type general – the ones that you have just clicked. Increase sidelength to 48-64 pixels. Vll-list is the format in which Dynamo stores directives for particle extraction; you can modify it with your own scripts if you have a lot of tomograms. Now click ''Create list'' and ''Crop particle''. Change output directory name to i.e. “particles” and click “start cropping”.<br />
<br />
<br />
[[File:StartersGuideFigure9.png|thumb|center|600px| Figure 9. Cropping particles from the set of selected models.]]<br />
<br />
After cropping is done explore your cropped particles by clicking ddbrowse under the output folder name. <tt>ddbrowse</tt> (Dynamo Data Browse) is a lightweight browser for particle visualization, just click “show” and make sure your particles are well centered and fit the box. If not - use pre-existing models to generate the vll-file and re-extract the particles or increase the sidelength. Please note that ''Dynamo'' generated a particles/ folder with sub-volumes formatted as particle_000XX.em.<br />
<br />
[[File:StartersGuideParticlesUnaligned.png|thumb|center|400px| Particles without alignment (<tt>table</tt> is switched off).]]<br />
<br />
==Subtomogram alignment and averaging==<br />
<br />
===Initial model generation===<br />
If you don’t have an initial model it is easy to generate it by manually aligning several particles and summing them up. ''Dynamo'' stores information about the particles in dynamo tables: for each particle a table contains shifts and rotations to bring the particle to the average (by default these values are initialized as 0s), particle ID (tag), orientation of the missing wedge and others. For full info type <tt>>> dthelp</tt> in Matlab. ''Dynamo'' catalogue generates an initial table during particle extraction, its location is <tt>particles/crop.tbl </tt><br />
<br />
==== Loading particle in <tt>dgallery</tt> ====<br />
<br />
For initial model generation we will use dynamo_gallery to generate initial orientations for some of our particles. Type<br />
<br />
<tt>>> dgallery(‘data’, ‘exampleData.Data’); </tt><br />
<br />
assuming you named <tt>exampleData</tt> your data folder while cropping.<br />
<br />
* Load particles into memory by clicking “load” in the Load from disk field on the left of the window. Note that at first, the scene will show only '''one''' particle.<br />
* Display some or all of the particles with the slider in the <tt>Shown</tt> particles field on the top of the window. <br />
* Toggle between X- Y- and Z-views in the View: orientation field.<br />
<br />
==== Manual alignment of particles in <tt>dgallery</tt> ====<br />
<br />
Now for each particle you can specify the center with a mouse click and the “C” button. Thermosomes are barrel-like particles with a long axis, you therefore may also specify the “top” of the particle by clicking along the long axis of the particle (but still within the box) with a mouse click and click “N” (stands for North). Do it for all the particles alternating the X- Y- and Z- views. Non-aligned particles are marked red, once the particle has been left-clicked it adds to the section in memory and turns blue. To remove from selection right click on the box. <br />
<br />
==== Selecting particles in <tt>dgallery</tt> ====<br />
<br />
You can save selected tags and a corresponding dynamo table by clicking the “quick save” button in the Particle Section field on top of the window. It saves a quickbuffer.tbl and quickbuffer.tags to the hard drive that you will use later.<br />
<br />
==== Averaging selected particles ====<br />
To generate the average you need to “apply” the table on the particles. For this click the “average” button in the Particle selection field. It opens dynamo_average_GUI with a lot of controls, we only need output filename in the Averaged volume field - my_average.em. Click “compute average” in the bottom of the window; wait till it is done. Right click to the output filename and select the [view] simple 3d depiction of all slices option and examine it from X- Y- and Z- views. If you are not satisfied with the result close the window and refine your manual alignment / add more particles to the average.<br />
<br />
===Alignment projects===<br />
Iterative alignment of subtomograms to the average is performed by dynamo projects that you can run in various high-performance computational environments. To have a project you need particles, an initial reference and a table which you have already generated. Start a [[dcp GUI| WIZARD]] in the ''Dynamo'' command window or type <br />
<br />
<tt>>> dcp</tt><br />
<br />
in Matlab (<tt>dcp</tt> stands for ''dynamo current project''). Type a project name “drun1” and press Enter, ''Dynamo'' will generate the auxiliary files in a folder called <tt>drun1</tt>. <br />
<br />
====Linked files====<br />
<br />
Provide input in the popup windows: folder with particles (and press OK), initial table <tt>particles/crop.tbl</tt>. You can also provide the <tt>quickbuffer.tbl</tt> which has the initial orientations from manual picking, however remember that this table contains only the selected particles. In cases when you don’t have a table you can generate a blank or a random tables that would be consistent with your particle folder but will lack the metadata i.e. real missing wedge values or source tomograms. Go to “template” and input <tt>my_average.em</tt>.<br />
Go to “Masks”, here you can provide several masks for different purposes, now we are only interested in the alignment mask. Use ellipsoid, you can specify semi-axis or go to “Mask editor” to make more sophisticated masks. Note that Dynamo by default uses Rossman correlation which eliminates the artefacts associated with hard (non-soft) mask. As usual you right click on the mask that you generate in order to open it with dview. For this type of simulated data “use default masks” will work.<br />
<br />
====Numerical parameters====<br />
Numerical parameters menu provides flexibility on how to run your alignment. You can specify different parameters for different rounds of refinement. You can select any value in the table with a mouse and click the “?” button on the top right of the parameter window, this will give you more info. For more parameters check boxes on the left. The most important parameters to consider are:<br />
<br />
* ''Number of iterations'' in each round. Make 2 rounds with 3-4 iterations in each. The first round will be global search with a course angular step, the second – constrained search with a finer step. If you decided to use a table with prealigned particles (quickbuffer.tbl) you may skip the first round.<br />
* ''Angular search ranges'': cone aperture is the scan range for the first two Euler angles with a step of cone sampling starting at the actual angles stored in the table. 360 is the full scan range. Azimuth rotation range defines rotation range around the new vertical axis of the particle<br />
* ''High- and lowpass values'' in Fourier voxels limit the used frequency range. It can be autotuned (below in this manual), here use some reasonable values like half-Nyquist.<br />
*''Particle dimensions'' – use the sidelength of your box; if you put lower value the particles will be downsampled for the particular round. This will speed up the process.<br />
* ''Refine'' after each angular scan the search step reduces refine factor times, this repeats refine times. I.e. if your cone sampling is 10 deg, refine and refine factor and 3 and 2 then 10, 5, 2.5 and 1.25 degrees will be sampled. This is the optimization of angular search space. We typically use 2 and 2.<br />
*''Shift limits'' limits translation of particles from the center of the box (if shifts limiting way is 1) or from the previously estimated center (if shifts limiting way is 2). The previous estimates for the shifts and rotations are taken from input tables and are updated at the end of each iteration.<br />
*''Symmetry'' – if you know the symmetry of your protein it will speed up the convergence and result in higher resolution. Here we don’t make any assumptions.<br />
*''Threshold parameters and modus'' specify which particles contribute to the average at the end of each iteration. Our typical values are [1,2] and [0.5,5].<br />
<br />
There are predefined parameter sets for Global and Local searches (''Predefined profiles -> Global search / Refinement''). After you set up the parameters press ''OK'' or close the window.<br />
<br />
[[File:StartersGuideFigure10.png|thumb|center|600px| Figure 10. Summary of numerical parameters used for sub tomogram averaging.]]<br />
<br />
Go to Computing environment in Wizard and select Matlab-CPU. Click “Info (nvidia)” it should output you a table to your Matlab window, your GPU identifiers are in the left column of the table. Put those into the GPU identifiers to the ‘identifiers” field.<br />
<br />
====Running the project====<br />
<br />
Go back to the Wizard window, click “check” and ‘unfold”, this will generate a runnable matlab script drun1.m. If you will modify your project and will want to run it again you need to re-unfold the project before each run. <br />
<br />
* In a Matlab session you can just click on “run”. <br />
* In a standalone project it's more convenient to open a new terminal, activate ''Dynamo'' on it and run the project just invoking the name of the project script that was generated by the unfolding step:<br />
<br />
<tt> ./drun1.exe</tt><br />
<br />
Open another Matlab window, activate dynamo, go to the same folder and monitor the progress of the execution by typing in >> dvstatus drun1. After several iterations you can look at intermediate results by typing:<br />
<br />
<tt> >> ddb drun1:a –v </tt><br />
<br />
or <br />
<br />
<tt>>>ddb drun1:a:ite=* -j c10</tt><br />
<br />
For more info on <tt>ddb</tt> type <tt> >> help ddb</tt>. In the end of your iterations you can monitor the results in the show results section of Wizard and should get something like this:<br />
<br />
[[File:StartersGuideResultsAlignment.png|thumb|center|400px| Results of alignment.]]<br />
<br />
<br />
==Subtomogram classification==<br />
First generate a dataset which has 2 populations of particles. For this, type:<br />
<br />
<tt>>> dynamo_tutorial('data_classification','M',8,'N',8);</tt><br />
<br />
in your Matlab window. It generates two sets of particles, 8 particles in each and stores their real orientations in the directory <tt>data_classification</tt>.<br />
<br />
===Multireference alignment===<br />
The idea behind [[Multireference Analysis|MRA]] is: iterative alignment of particles to several references and then each particle assigned to the reference where it fits the best. This way over iterative refinement similar particles will have higher correlation to similar references and will eventually group together. MRA is naturally implemented in ''Dynamo''. For classification into N classes we will need N initial references and N initially identical tables that ''Dynamo'' will generate automatically. The references should be slightly different to start particle differentiation, in ''Dynamo'' we add a small amount of white noise to the overall average of all particles.<br />
<br />
==== Creating MRA projects ====<br />
Create a new project by typing “drun2” in the project name of Wizard set number of references to 2, set ''Swap particles'' on, put in the particle data. <br />
<br />
==== Seeds for MRA projects ====<br />
* Go to “table” and type <tt>data_classification/real.tbl</tt> into the “clone this” field and press “copy”, press “OK”.<br />
* Go to “template” and clone data_classification /original_template.em with addition of some extra noise with amplitude 1. <br />
* Initialize masks with the defaults parameters (large button on the bottom). <br />
<br />
Go to “numerical parameters” and use Predefined profiles -> refinement. Increase angular search steps to 5, set a thresholing policy and press “OK”. Set computing environment to GPU (under Matlab), specify GPU identifiers. <br />
<br />
==== Execution of projects ====<br />
Check, unfold and run the project, this project will run twice longer as each particle will be aligned to two references. Monitor the progress of your project by <br />
<br />
<tt>>> dvstatus drun2</tt><br />
<br />
and visualize the results as<br />
<br />
<tt> >> ddb drun2:a:ref=* -j c5</tt><br />
<br />
You should get something like this – two averages with particle in one class larger than in the other.<br />
<br />
[[File:StartersGuideResultsClassification.png|thumb|center|600px| Results of classification.]]<br />
<br />
===Classification with PCA and k-means===<br />
An analytical method is described in a tutorial <tt>classification_tutorial.pdf</tt>, please refer to it with <tt>dhelp</tt>.<br />
<br />
==Adaptive bandpass filtering==<br />
Aligning all particles to one reference may introduce overfitting resulting in resolution overestimation and a risk of overinterpretation of the final map. This is illustrated by a small exercise in the <tt>noise_artifacts.pdf</tt> tutorial. For the generation of the final map it is therefore highly recommended to separate your particles into two halves and perform independent alignment and averaging, using an [[Adaptive bandpass filtering]] approach: at the end of each iteration you can detect the current resolution by [[Fourier Shell Correlation]] and only use your reliable frequencies for the alignment at the next steps.<br />
<br />
To perform independent half-dataset refinement load your initial project <tt>drun1</tt> to Wizard (type in the Project field and press Enter or use Ctrl+L). Go to ''Multireference -> Adaptive filtering -> Derive a project'', ''Dynamo'' will generate a new project <tt>drun1_eo</tt>, load it, inspect/update the parameters and run it. After it is done you can measure your resolution based on fsc plots from ''Multireference -> Adaptive filtering -> Plot'' attained resolution and inspect the average the usual way.<br />
</div>Stefano Scaramuzza