Difference between revisions of "Model"

From Dynamo
Jump to navigation Jump to search
 
(26 intermediate revisions by the same user not shown)
Line 12: Line 12:
 
#: Models are naturally integrated inside the [[catalogue]] system, making the process of extracting particles very easy.
 
#: Models are naturally integrated inside the [[catalogue]] system, making the process of extracting particles very easy.
  
For the first two functionalities, ''Dynamo''offers different types of models, adapted to the different geometries that you might encounter: filaments, vesicles, irregularly shaped membranes, isolated particles, pseudo-crystaline arrangements....
+
For the first two functionalities, ''Dynamo'' offers different types of models, adapted to the different geometries that you might encounter: filaments, vesicles, irregularly shaped membranes, isolated particles, pseudo-crystaline arrangements....
  
 
==Working with models==
 
==Working with models==
Line 22: Line 22:
 
In most models, this stage produces a matrix of points in the field {{t|points}} of the model.
 
In most models, this stage produces a matrix of points in the field {{t|points}} of the model.
 
   
 
   
 
 
===Computation stage===
 
===Computation stage===
 
Once the initial points that describe the geometry of interest have been introduces, the user can choose for different parameters that will determine the actual location of the parameters. For instance, for vesicles, the user can choose the separation of particles for an even distribution on the vesicle surface, for filaments the user can chose the distance along the axis between successive cropping points. This process ranges between the trivial (isolated particles without preassigned orientation) to more complex construction (particles evenly distributed on arbitrarily shaped membranes)
 
Once the initial points that describe the geometry of interest have been introduces, the user can choose for different parameters that will determine the actual location of the parameters. For instance, for vesicles, the user can choose the separation of particles for an even distribution on the vesicle surface, for filaments the user can chose the distance along the axis between successive cropping points. This process ranges between the trivial (isolated particles without preassigned orientation) to more complex construction (particles evenly distributed on arbitrarily shaped membranes)
Line 28: Line 27:
 
At the end of this stage, all models will contain (non empty) fields {{t|crop_points}} and {{t|crop_angles}}. ''Dynamo'' will use them internally if you crop particles with the catalogue (useful when several models and tomograms are present), or you can extract a single cropping table corresponding to this particular model   
 
At the end of this stage, all models will contain (non empty) fields {{t|crop_points}} and {{t|crop_angles}}. ''Dynamo'' will use them internally if you crop particles with the catalogue (useful when several models and tomograms are present), or you can extract a single cropping table corresponding to this particular model   
  
<tt>>>t = myModel.grepTable()}</tt>   
+
<tt>>> t = myModel.grepTable();</tt>   
  
 
==Basic operations with models==
 
==Basic operations with models==
 
  
 
===Model files and model objects===
 
===Model files and model objects===
Line 37: Line 35:
 
With the order  
 
With the order  
  
<tt> m = dread(myModel.omd);</tt>
+
<tt>>> m = dread('myModel.omd');</tt>
  
 
we are reading the contents of the file {{t|myModel.omd)}} and bringing it into memory as the variable arbitrarily named {{t|m}} in the current session.
 
we are reading the contents of the file {{t|myModel.omd)}} and bringing it into memory as the variable arbitrarily named {{t|m}} in the current session.
We can edit this variable and write it back to disk when we are done, to the same or to another file.
+
We can edit this variable and write it back to disk when we are done, to the same or to another file, for instance with.
{{t| dwrite(m,'someOtherModelFile.omd');}}
 
 
 
 
 
 
  
 +
<tt>>> dwrite(m,'someOtherModelFile.omd');</tt>
  
 
{{Technicality}}
 
{{Technicality}}
Line 52: Line 47:
  
 
==Types of models==
 
==Types of models==
 +
 +
There are different model ''classes''. A model class is indicated in {{t|fixed width}}, and defines completely the two functionality stages of a model. We talk loosely about "membrane models" or "filament models", but to refer precisely to the a model with a given set of functionalities, we need to name the class. {{t|filamentWithTorsion}}  and {{t|filamentRings}} are two classes of filament models, and {{t|membraneByLevels}} is the class normally used for defining [[Membrane models|Membrane models]] freely shaped membranes.
 +
 +
{{Technicality}}
 +
Model classes are defined inside the ''Dynamo'' package {{t|dmodels}}
  
 
===Isolated particles===
 
===Isolated particles===
  
In those modes, the Geometry computation stage
+
In those modes, [[#computation stage]] is trivial. User points are directly mapped into crop points. Optionally, one common orientation can be selected for the full set of particles in the model
  
 
===Filaments===
 
===Filaments===
{{main|Filament modesl|Vesicle model}}
+
{{main|Filament model|Filament models}}
Adequate for particles evenly distributed along filaments. Subomograms can be defined to include full sections of the filament (with an initial orientation coaxial with the filament) or to be defined as centered in the building subunits and with an initial direction orthogonal to the axis.
+
Adequate for particles evenly distributed along filaments. Subtomograms can be defined to include full sections of the filament (with an initial orientation coaxial with the filament) or to be defined as centered in the building subunits and with an initial direction orthogonal to the axis.
Filament models can also be used as support geometry for particles the decorate the filaments irregularly.  
+
Filament models can also be used as support geometry for particles the decorate the filaments irregularly.
 
 
  
 
===Membranes===
 
===Membranes===
{{main|Membrane models|Vesicle model}}
+
{{main|Membrane models|Membrane models}}
 
When particles are regularly distributed on a membrane (or a membrane fraction)  that cannot be approximated by a vesicle.
 
When particles are regularly distributed on a membrane (or a membrane fraction)  that cannot be approximated by a vesicle.
  
 
===Vesicles===
 
===Vesicles===
{{main|Vesicle models|Vesicle model}}
+
{{main|Vesicle models|Vesicle models}}
 
Adequate for particles evenly distributed on spheres or ellipsoids.
 
Adequate for particles evenly distributed on spheres or ellipsoids.
 +
 +
===Dipole sets===
 +
{{main|Dipole set models|Dipole set models}}
 +
Used to mark in a single model different groups of points. Typically used to annotate the orientations of isolated particles, or to annotate groups vesicles of different radius in a single tomogram.
  
 
==Archiving models==
 
==Archiving models==
After creating or working with a model, it is important to remember to save it into disk. This will save the model in an appropriate place of the catalogue folder, and also create some internal links inside the catalogue.
+
After creating or working with a model, it is important to remember to save it into disk. This will save the model in an appropriate place of the catalogue folder, and also create some internal links inside the catalogue.  
  
 +
=== Programmatically ===
 +
 +
When a model related to a catalogue is in the Matlab workspace as a variable, you can return it to the catalogue with the method <tt>saveIntoCatalogue</tt>.
 +
This is useful to extract archived models, operate on them, and put them back into the catalogue.
 +
 +
:''Example''
 +
: You want to change the distance between consecutive crop points in a model which is stored with the name  <tt>myFilename </tt>. The model is ''catalogued'', and you want to put it back into the catalogue after updating it;
 +
: <tt>m = dread('myFilename');</tt> # reads the model object into the matlab workspace with the arbitrary variable name <tt>m</tt>
 +
: <tt>m.subunits_dz = 10;</tt> # changes the parameter value
 +
: <tt>m.updateCrop(); </tt> # recompute the <tt>crop_points</tt> and <tt>crop_angles</tt>
 +
:<tt>m.saveIntoCatalogue()</tt>
  
 
==Importing models==
 
==Importing models==
Coordinates and angles can be defined in other programs and imported into ''Dynamo'' as models. When an external file is imported as a model, ''Dynamo'' will fill the {{t|points}} fields of the models (and possibly other geometric parameters depending on the model type), covering the ''Point input stage'' of the model. The ''geometry computation'' stage can then be performed inside ''Dynamo''.
+
{{main|Importing models|Importing models}}
 +
Coordinates and angles can be defined in other programs and imported into ''Dynamo'' as models. When an external file is imported as a model, ''Dynamo'' will fill the {{t|points}} fields of the models (and possibly other geometric parameters depending on the model type), covering the ''Point input stage'' of the model. The ''geometry computation'' stage can then be performed inside ''Dynamo''.
  
{{docfunction|dynamo_model_import|dmimport}}
+
The most important function for manual importing of models is {{docfunction|dynamo_model_import|dmimport}}
  
 
==Command line options==
 
==Command line options==
Line 84: Line 99:
  
 
===Adding a model to a catalogued tomogram===
 
===Adding a model to a catalogued tomogram===
A file representing a model created on a catalogued tomogram will lay in a predefined position. For instance, a model named {{t|someModel}} created by ''Dynamo'' in one of its catalogue-based  [[Viewing tomograms|tomogram viewers]] for the volume indexed as 2 in a catalogue named {{t|my Catalogue}} will be automatically stored in the position:
+
{{main|Adding models to catalogued volumes|Adding models to catalogued volumes}}
{{t| myCatalogue /volume_2/models/someModel.omd}}  
 
  
Now, if you have produced  a model object {{t|myModel}} independently of the catalogue system, and want to embed it and assign it to a catalogued tomogram, you might have the temptation of just saving it as
+
Models can be created, imported or edited in the workspace, using the command line tools of Matlab and/or ''Dynamo''. A model object that has been extracted from the catalogue can be returned to it after edition using the method {{t|save}}, i.e.
{{t| dwrite(myModel,'myCatalogue /volume_2/models/myModel.omd')}}.
 
This is not correct, as the catalogue needs to preserve some internal links
 
# Assign the model with a ''Dynamo'' command
 
#: {{t|dcm -c myCatalogue -l 2 -am myModel.omd}}
 
# Alternatively, you can tell the catalogue to relink all the files found in the models folder of one or more volumes.
 
#: {{t|dcm -c myCatalogue --relinkAllModelFiles}}  
 
  
If you have created a model  
+
 
 +
<nowiki>m = dread(<someModelFileInACatalogue>);
 +
...  your edition operations ...
 +
m.saveIntoCatalogue();</nowiki>
 +
When a model is generated from scratch in the workspace, you need to add it to a volume [[Adding models to catalogued volumes|explicitly]].
 +
 
 +
=== Extracting models from catalogued tomograms ===
 +
{{main|Extracting models from catalogued tomograms|Extracting models from catalogued tomograms}}
 +
Models in a catalogue are just regular files defined in well defined locations. A model named 'myModel' in the catalogue 'myCatalogue' will be stored in the file
 +
<tt>myCatalogue/volume1/models/myModel.omd</tt>
 +
 
 +
They can be accessed with the <tt>dcm</tt> command:
 +
* <tt>dcm -c myCatalogue -l m -ws o</tt>
 +
: will list  all the models in all volumes in the catalogue (as a cell array of files in the {{t|result}} property output variable <tt>o</tt>).
 +
* <tt>dcm -c myCatalogue -l m -i 4 -ws o</tt>
 +
: will list the all the models in the tomogram indexed as number 4  the catalogue (as a cell array of files in the output variable <tt>o</tt>).
 +
 
 +
There is a simplified syntax through the <tt>dynamo_catalogue_models</tt> command:
 +
* <tt> files = dcmodels(myCatalogue);</tt>
 +
: sets in file array {{t|files}} all model files found inside the catalogue.
 +
* <tt> files = dcmodels(myCatalogue,'i',2);</tt>
 +
: sets in file array {{t|files}} all model files found inside volume number 1.
 +
* <tt> files = dcmodels(myCatalogue,'nameContains','vesicle');</tt>
 +
: adds the condition that the file names contain the string {{t| 'vesicle'}}
  
 
===Plotting models ===
 
===Plotting models ===
An easy way to visualize the contents of a model object is with the {{t|ezplot}} method. Type symply:
+
All models have a set of  methods that  start with the string {{t|plot}}
{{t|>> myModel.ezplot}}
+
* <tt>plotPoints</tt>
 +
* <tt>plotTablePoints</tt>
 +
* <tt>plotTableSketch</tt>
 +
Additionally each model class offers its own set of plotting options. For instance, the model {{t|Vesicle}} can generate surface plots, and includes thus the methods <tt>plotSurface</tt> and <tt>plotMesh</tt>
 +
 
 +
An easy way to visualize the contents of a model object from the command line is with the {{t|ezplot}} method. Type simply:
 +
<tt>|>> myModel.ezplot}}</tt>|
 
to get a menu of plotting options.
 
to get a menu of plotting options.
 +
 +
===Extracting particles from a model===
 +
Normally, you want to generate a [[data folder]] containing cropped particles form several models simultaneously. But sometimes, you are interested in extracting the particles from just one model into a single [[data folder]] in order to start working on your data. This can be done while you are navigating the corresponding tomogram with [[dtmslice]], through the menu [Active model] -> [Crop particles from active model], which will open the<tt>dtcrop</tt> command in its  GUI form.
 +
 +
From the command line, you can just extract the volume and the [[Cropping table |table]] contained in the model and use the command line form of the <tt>dtcrop</tt> command.
 +
<nowiki>
 +
m = dread(myModelFile);
 +
t = m.grepTable(); # here you can check the contents of table t
 +
tomogram = m.cvolume.fullFileName();% this recovers the name of the original tomogram.
 +
dtcrop(tomogram,t,'targetFolder',64);</nowiki>

Latest revision as of 10:00, 2 May 2017


The model is the standard way to create and store annotations in a tomogram. The most usual application of a model is in particle picking and extraction.

A model has three basic functionalities, applied sequentially:

  1. Data input
    Controls how the user feeds data into the model. An appropriate model provides the user with tools to inspect the tomograms, locate manually or semi-automatically regions, structures or coordinates of interest and mark them.
  2. Geometrical computation stage.
    Converts the input into particle positions.
  3. Archiving the annotations.
    Models are naturally integrated inside the catalogue system, making the process of extracting particles very easy.

For the first two functionalities, Dynamo offers different types of models, adapted to the different geometries that you might encounter: filaments, vesicles, irregularly shaped membranes, isolated particles, pseudo-crystaline arrangements....

Working with models

Input stage

This part represents the interaction with the tomogram. Depending on the model, it might imply directly clicking particles, or clicking centers of vesicles, points on a membrane.

In most models, this stage produces a matrix of points in the field points of the model.

Computation stage

Once the initial points that describe the geometry of interest have been introduces, the user can choose for different parameters that will determine the actual location of the parameters. For instance, for vesicles, the user can choose the separation of particles for an even distribution on the vesicle surface, for filaments the user can chose the distance along the axis between successive cropping points. This process ranges between the trivial (isolated particles without preassigned orientation) to more complex construction (particles evenly distributed on arbitrarily shaped membranes)

At the end of this stage, all models will contain (non empty) fields crop_points and crop_angles. Dynamo will use them internally if you crop particles with the catalogue (useful when several models and tomograms are present), or you can extract a single cropping table corresponding to this particular model

>> t = myModel.grepTable();

Basic operations with models

Model files and model objects

A model file is simply a file containing a model. A model object is a model as seen in a current Matlab or Dynamo Standalone session. The model object is, so to speak, an "alive" entity that is brought to the RAM for edition and work. With the order

>> m = dread('myModel.omd');

we are reading the contents of the file myModel.omd) and bringing it into memory as the variable arbitrarily named m in the current session. We can edit this variable and write it back to disk when we are done, to the same or to another file, for instance with.

>> dwrite(m,'someOtherModelFile.omd');

Technical information

A model object has always a superclass of type model, and the class will depend on the particular geometry.

Types of models

There are different model classes. A model class is indicated in fixed width, and defines completely the two functionality stages of a model. We talk loosely about "membrane models" or "filament models", but to refer precisely to the a model with a given set of functionalities, we need to name the class. filamentWithTorsion and filamentRings are two classes of filament models, and membraneByLevels is the class normally used for defining Membrane models freely shaped membranes.

Technical information Model classes are defined inside the Dynamo package dmodels

Isolated particles

In those modes, #computation stage is trivial. User points are directly mapped into crop points. Optionally, one common orientation can be selected for the full set of particles in the model

Filaments

Main article: Filament models

Adequate for particles evenly distributed along filaments. Subtomograms can be defined to include full sections of the filament (with an initial orientation coaxial with the filament) or to be defined as centered in the building subunits and with an initial direction orthogonal to the axis. Filament models can also be used as support geometry for particles the decorate the filaments irregularly.

Membranes

Main article: Membrane models

When particles are regularly distributed on a membrane (or a membrane fraction) that cannot be approximated by a vesicle.

Vesicles

Main article: Vesicle models

Adequate for particles evenly distributed on spheres or ellipsoids.

Dipole sets

Main article: Dipole set models

Used to mark in a single model different groups of points. Typically used to annotate the orientations of isolated particles, or to annotate groups vesicles of different radius in a single tomogram.

Archiving models

After creating or working with a model, it is important to remember to save it into disk. This will save the model in an appropriate place of the catalogue folder, and also create some internal links inside the catalogue.

Programmatically

When a model related to a catalogue is in the Matlab workspace as a variable, you can return it to the catalogue with the method saveIntoCatalogue. This is useful to extract archived models, operate on them, and put them back into the catalogue.

Example
You want to change the distance between consecutive crop points in a model which is stored with the name myFilename . The model is catalogued, and you want to put it back into the catalogue after updating it;
m = dread('myFilename'); # reads the model object into the matlab workspace with the arbitrary variable name m
m.subunits_dz = 10; # changes the parameter value
m.updateCrop(); # recompute the crop_points and crop_angles
m.saveIntoCatalogue()

Importing models

Main article: Importing models

Coordinates and angles can be defined in other programs and imported into Dynamo as models. When an external file is imported as a model, Dynamo will fill the points fields of the models (and possibly other geometric parameters depending on the model type), covering the Point input stage of the model. The geometry computation stage can then be performed inside Dynamo.

The most important function for manual importing of models is dmimport

Command line options

Models can read an write normally using dread and dwrite. Basic models use the extension ".omd" (object model).

Adding a model to a catalogued tomogram

Main article: Adding models to catalogued volumes

Models can be created, imported or edited in the workspace, using the command line tools of Matlab and/or Dynamo. A model object that has been extracted from the catalogue can be returned to it after edition using the method save, i.e.


m = dread(<someModelFileInACatalogue>);
...  your edition operations ...
m.saveIntoCatalogue();

When a model is generated from scratch in the workspace, you need to add it to a volume explicitly.

Extracting models from catalogued tomograms

Main article: Extracting models from catalogued tomograms

Models in a catalogue are just regular files defined in well defined locations. A model named 'myModel' in the catalogue 'myCatalogue' will be stored in the file myCatalogue/volume1/models/myModel.omd

They can be accessed with the dcm command:

  • dcm -c myCatalogue -l m -ws o
will list all the models in all volumes in the catalogue (as a cell array of files in the result property output variable o).
  • dcm -c myCatalogue -l m -i 4 -ws o
will list the all the models in the tomogram indexed as number 4 the catalogue (as a cell array of files in the output variable o).

There is a simplified syntax through the dynamo_catalogue_models command:

  • files = dcmodels(myCatalogue);
sets in file array files all model files found inside the catalogue.
  • files = dcmodels(myCatalogue,'i',2);
sets in file array files all model files found inside volume number 1.
  • files = dcmodels(myCatalogue,'nameContains','vesicle');
adds the condition that the file names contain the string 'vesicle'

Plotting models

All models have a set of methods that start with the string plot

  • plotPoints
  • plotTablePoints
  • plotTableSketch

Additionally each model class offers its own set of plotting options. For instance, the model Vesicle can generate surface plots, and includes thus the methods plotSurface and plotMesh

An easy way to visualize the contents of a model object from the command line is with the ezplot method. Type simply: |>> myModel.ezplot}}| to get a menu of plotting options.

Extracting particles from a model

Normally, you want to generate a data folder containing cropped particles form several models simultaneously. But sometimes, you are interested in extracting the particles from just one model into a single data folder in order to start working on your data. This can be done while you are navigating the corresponding tomogram with dtmslice, through the menu [Active model] -> [Crop particles from active model], which will open thedtcrop command in its GUI form.

From the command line, you can just extract the volume and the table contained in the model and use the command line form of the dtcrop command.

m = dread(myModelFile);
t = m.grepTable(); # here you can check the contents of table t
tomogram = m.cvolume.fullFileName();% this recovers the name of the original tomogram.
dtcrop(tomogram,t,'targetFolder',64);