Difference between revisions of "Averaging of subtomograms"

From Dynamo
Jump to navigation Jump to search
 
(8 intermediate revisions by the same user not shown)
Line 6: Line 6:
  
 
== Through the command line ==
 
== Through the command line ==
To operate manually averages of particles (given an available [[table|metadata]]), use the {{t|daverage}} command, or its GUI version {{t|dvarage_GUI}}. {{docfunction|dynamo_average|daverage}}, with the basic syntax:
+
To operate manually averages of particles (given an available [[table|metadata]]), use the <tt>[[daverage]]</tt> command, or its GUI version {{t|daverage_GUI}}. {{docfunction|dynamo_average|daverage}}, with the basic syntax:
  
 
  <tt> o = daverage( data,'t',table); </tt>  
 
  <tt> o = daverage( data,'t',table); </tt>  
  
in Matlab-like syntax, or  
+
in Matlab-like syntax, or:
  
 
  <tt> o = daverage(data,'t',table);</tt>
 
  <tt> o = daverage(data,'t',table);</tt>
Line 17: Line 17:
 
* <tt>'average_raw'</tt>: the average of the particles, ''without'' any [[Fourier compensation during averaging | Fourier compensation.]]
 
* <tt>'average_raw'</tt>: the average of the particles, ''without'' any [[Fourier compensation during averaging | Fourier compensation.]]
 
* <tt>'fweight'</tt>:  count of contributions for each Fourier coefficient.
 
* <tt>'fweight'</tt>:  count of contributions for each Fourier coefficient.
*<tt>'average'</tt>:  <tt>'average_raw'</tt> compensated through <tt>'fweight'</tt>.
+
* <tt>'average'</tt>:  <tt>'average_raw'</tt> compensated through <tt>'fweight'</tt>.
  
 
The most important additional  flags are:
 
The most important additional  flags are:
  
 
'''General flags'''
 
'''General flags'''
*<tt>'nmask'</tt>:  if a mask is provided and the ''norm'' flag is tuned to 1 (as it is by default), each particle will be normalized to have zero mean and standard deviation 1 inside the passed mask before adding it to the average.
+
*<tt>'nmask'</tt>:  if a mask is provided and the ''norm'' flag is tuned to 1 (as it is by default), each particle will be normalized to have zero mean and standard deviation 1 inside this mask before adding it to the average. <br />- default value is empty (no normalization).
*<tt>'implicitRotationMasking'</tt>. If tuned to 1, the contribution of the particle outside an interior sphere will be discarded to minimize [[Border alignment artifact | rotation artifacts]] <br />default value is one.
+
*<tt>'implicitRotationMasking'</tt>. If tuned to 1, the contribution of the particle outside an interior sphere will be discarded to minimize [[Border alignment artifact | rotation artifacts]] <br />- default value is 1.
 
'''Fourier compensation flags'''
 
'''Fourier compensation flags'''
*<tt>'fcompensate'</tt> : if tuned to 1, a [[Fourier compensation during averaging | Fourier compensation step]] will be carried out.  
+
*<tt>'fcompensate'</tt> : if tuned to 1, a [[Fourier compensation during averaging | Fourier compensation step]] will be carried out. <br /> default value is zero. No missing wedge compensation.
 
*<tt>'fmin'</tt>  : minimum  number of particles that need to contribute to a given Fourier component.
 
*<tt>'fmin'</tt>  : minimum  number of particles that need to contribute to a given Fourier component.
*<tt>'fminFraction</tt>  : if switched to a value between 0 and 1, selects the minimum fraction of particles that should contribute to a fourier component to admit it during averaging. <br /> default: empty. Elimination of underrepresented fourier components is driven by the value of parameter <tt>'fmin'</tt>.
+
*<tt>'fminFraction</tt>  : if switched to a value between 0 and 1, selects the minimum fraction of particles that should contribute to a fourier component to admit it during averaging. <br />- default: empty. Elimination of underrepresented fourier components is driven by the value of parameter <tt>'fmin'</tt>.
 
'''Smooth masking for Fourier compensation'''
 
'''Smooth masking for Fourier compensation'''
  
 
This family of flags is provided for fine tuning of the behaviour of the Fourier compensation: they drive the imposition of a mask on the raw average before computing its Fourier transform for division with the <tt>'fweight'</tt>. Note that with the default values of the parameters, this step is ''not'' operated.
 
This family of flags is provided for fine tuning of the behaviour of the Fourier compensation: they drive the imposition of a mask on the raw average before computing its Fourier transform for division with the <tt>'fweight'</tt>. Note that with the default values of the parameters, this step is ''not'' operated.
*<tt>'fCompensationSmoothingMask'</tt>. If a file or cube is matrix, the raw average will be multiplied with this mask before Fourier compensation. <br /> -default: no mask.
+
*<tt>'fCompensationSmoothingMask'</tt>. If a file or cube is matrix, the raw average will be multiplied with this mask before Fourier compensation. <br /> - default: no mask.
*<tt>'fCompensationSmoothingRadius'</tt>. if a non-zero value is passed, it will generate a spherical mask that will be imposed onto the raw average.  <br /> -default: 0, no spherical mask.
+
*<tt>'fCompensationSmoothingRadius'</tt>. if a non-zero value is passed, it will generate a spherical mask that will be imposed onto the raw average.  <br /> - default: 0, no spherical mask.
*<tt>'fCompensationSmoothingDecay'</tt>. If a spherical mask is used (i.e., if <tt>'fCompensationSmoothingRadius'</tt> is not zero), this sphere will have a smooth gaussian decay of <tt>'fCompensationSmoothingDecay'</tt> pixels from 1 to 0.  <br /> -default value: 4 pixels.
+
*<tt>'fCompensationSmoothingDecay'</tt>. If a spherical mask is used (i.e., if <tt>'fCompensationSmoothingRadius'</tt> is not zero), this sphere will have a smooth gaussian decay of <tt>'fCompensationSmoothingDecay'</tt> pixels from 1 to 0.  <br /> - default value: 4 pixels.
+
 
 
== Inside an alignment project ==
 
== Inside an alignment project ==
  
 
At the end of each iteration, the {{t|refined table}} generated by ''Dynamo'' is used to produce an average, which will be used as starting template in the next iteration.
 
At the end of each iteration, the {{t|refined table}} generated by ''Dynamo'' is used to produce an average, which will be used as starting template in the next iteration.
 +
 +
By default, the project will use the ''alignment mask'' of the project as normalization mask during averaging (i.e., <tt>'nmask'</tt> flag inside the internally executed <tt>'daverage'</tt> command).
  
 
There are different project parameters that can be used to modify the default behaviour of the {{t|daverage}} command used inside an alignment project.
 
There are different project parameters that can be used to modify the default behaviour of the {{t|daverage}} command used inside an alignment project.
  
*<tt>'implicitRotationMask'</tt>  <br /> default value is zero: this will introduce sharp edges contributing to the average, but will allow checking the features outside of the spherical mask.
+
*<tt>'implicitRotationMaskings'</tt>  <br /> default value is zero: this will introduce sharp edges contributing to the average, but will allow checking the features outside of the spherical mask.
 
*<tt>'fourierMinimumFractionForAverage'</tt>  <br /> default value is -1: use all Fourier coefficients during [[Fourier compensation during averaging | missing wedge compensation]].
 
*<tt>'fourierMinimumFractionForAverage'</tt>  <br /> default value is -1: use all Fourier coefficients during [[Fourier compensation during averaging | missing wedge compensation]].
  
Line 54: Line 56:
 
  <tt> [o,flags] = dpkproject.pipeline.genericInput2Average(vpr,myTable,'ite',ite,'justFlags',1); </tt>
 
  <tt> [o,flags] = dpkproject.pipeline.genericInput2Average(vpr,myTable,'ite',ite,'justFlags',1); </tt>
  
===Parallelism===
+
===Parallelization===
 
It is possible to select the number of cores that will be used during the ''averaging''.  The project parameter is called <tt>matlab_workers_average</tt> (shortform <tt>mwa</tt>), and can be modified also in [[dcp GUI]] (in the section of environment variables).  
 
It is possible to select the number of cores that will be used during the ''averaging''.  The project parameter is called <tt>matlab_workers_average</tt> (shortform <tt>mwa</tt>), and can be modified also in [[dcp GUI]] (in the section of environment variables).  
  
 
This option is available for computations on a single server with several cores. It will not work for the current MPI implementation.
 
This option is available for computations on a single server with several cores. It will not work for the current MPI implementation.

Latest revision as of 17:09, 6 December 2017

Category:General steps

Averaging of subtomograms means applying an available set of alignment parameters onto a set of data particles and adding them together. Because of the missing wedge, a compensation in the Fourier space is operated during this step.

From the operative point of view, the particles are stored in a data folder, which are indexed by a table.

Through the command line

To operate manually averages of particles (given an available metadata), use the daverage command, or its GUI version daverage_GUI. daverage, with the basic syntax:

 o = daverage( data,'t',table);  

in Matlab-like syntax, or:

 o = daverage(data,'t',table);

in Linux-like syntax. In both cases, o is an output structure which includes as flags:

  • 'average_raw': the average of the particles, without any Fourier compensation.
  • 'fweight': count of contributions for each Fourier coefficient.
  • 'average': 'average_raw' compensated through 'fweight'.

The most important additional flags are:

General flags

  • 'nmask': if a mask is provided and the norm flag is tuned to 1 (as it is by default), each particle will be normalized to have zero mean and standard deviation 1 inside this mask before adding it to the average.
    - default value is empty (no normalization).
  • 'implicitRotationMasking'. If tuned to 1, the contribution of the particle outside an interior sphere will be discarded to minimize rotation artifacts
    - default value is 1.

Fourier compensation flags

  • 'fcompensate' : if tuned to 1, a Fourier compensation step will be carried out.
    default value is zero. No missing wedge compensation.
  • 'fmin' : minimum number of particles that need to contribute to a given Fourier component.
  • 'fminFraction : if switched to a value between 0 and 1, selects the minimum fraction of particles that should contribute to a fourier component to admit it during averaging.
    - default: empty. Elimination of underrepresented fourier components is driven by the value of parameter 'fmin'.

Smooth masking for Fourier compensation

This family of flags is provided for fine tuning of the behaviour of the Fourier compensation: they drive the imposition of a mask on the raw average before computing its Fourier transform for division with the 'fweight'. Note that with the default values of the parameters, this step is not operated.

  • 'fCompensationSmoothingMask'. If a file or cube is matrix, the raw average will be multiplied with this mask before Fourier compensation.
    - default: no mask.
  • 'fCompensationSmoothingRadius'. if a non-zero value is passed, it will generate a spherical mask that will be imposed onto the raw average.
    - default: 0, no spherical mask.
  • 'fCompensationSmoothingDecay'. If a spherical mask is used (i.e., if 'fCompensationSmoothingRadius' is not zero), this sphere will have a smooth gaussian decay of 'fCompensationSmoothingDecay' pixels from 1 to 0.
    - default value: 4 pixels.

Inside an alignment project

At the end of each iteration, the refined table generated by Dynamo is used to produce an average, which will be used as starting template in the next iteration.

By default, the project will use the alignment mask of the project as normalization mask during averaging (i.e., 'nmask' flag inside the internally executed 'daverage' command).

There are different project parameters that can be used to modify the default behaviour of the daverage command used inside an alignment project.

  • 'implicitRotationMaskings'
    default value is zero: this will introduce sharp edges contributing to the average, but will allow checking the features outside of the spherical mask.
  • 'fourierMinimumFractionForAverage'
    default value is -1: use all Fourier coefficients during missing wedge compensation.

Reproducing averaging steps of a project from the command line

It is possible to apply onto a set of particles exactly the same averaging flags that were using during the runtime of a project using the command dpkproject.pipeline.genericInput2Average.

o = dpkproject.pipeline.genericInput2Average(vpr,myTable,'ite',ite); 

Here, o would be the output of the daverage command as operated with the flags passed to it by in the project vpr. These flags can be obtained explicitely with a second left hand side:

 [o,flags] = dpkproject.pipeline.genericInput2Average(vpr,myTable,'ite',ite,'justFlags',1); 

Parallelization

It is possible to select the number of cores that will be used during the averaging. The project parameter is called matlab_workers_average (shortform mwa), and can be modified also in dcp GUI (in the section of environment variables).

This option is available for computations on a single server with several cores. It will not work for the current MPI implementation.