Difference between revisions of "Postprocessing plugins"

From Dynamo
Jump to navigation Jump to search
(Created page with "Postprocessing plugins are a tool for customization of the alignment procedure in ''Dynamo''. With this kind of plugin, the user can define an action that ''Dynamo'' will exec...")
 
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
Postprocessing plugins are a tool for customization of the alignment procedure in ''Dynamo''. With this kind of plugin, the user can define an action that ''Dynamo'' will execute after each iteration during a project alignment.  
 
Postprocessing plugins are a tool for customization of the alignment procedure in ''Dynamo''. With this kind of plugin, the user can define an action that ''Dynamo'' will execute after each iteration during a project alignment.  
  
 +
==Applications of plugins==
  
==Naming of a plugin script==
+
Users have devised the use of plugins in very different scenarios and with very different levels of complexity. We just mention below some some examples. In square bracket we describe the difficulty of the writing the interface between the plugin and ''Dynamo'', not the complexity of the plugin itself.
  
''Dynamo'' requires that the plugin script itself is named with the pattern
+
* [Simple] Online definition of mask, depending on the average computed in the iteration.
<tt>dynamo_postprocessign_plugin_SOMESTRING.m</tt>
+
* [Simple] Taylored definition of [[Example of post processing plugin with template update | symmetrization operator]].
 +
* [Simple] User-defined thresholding for subtomograms from different tomograms.
 +
* [Medium] Resolution dependent scanning of angular scan parameters.
 +
* [Medium] Customization of golden-standard policy.
 +
* [Advanced] Fine tuning of [[MRA]] policy by PCA classification after each iteration.
  
where <tt>SOMESTRING</tt> is the shortname of the plugin.   
+
== Naming of a plugin script==
 +
 
 +
=== Matlab plugin ===
 +
''Dynamo'' requires that the plugin script itself is a matlab <tt>.m</tt> function named with the pattern:
 +
<tt>dynamo_plugin_post_PLUGINNAME.m</tt>
 +
 
 +
where <tt>PLUGINNAME</tt> is the shortname of the plugin.   
  
 
The script should be in the execution path of Matlab (i.e., you should use <tt>addpath</tt> on the directory where you have defined your plugin).
 
The script should be in the execution path of Matlab (i.e., you should use <tt>addpath</tt> on the directory where you have defined your plugin).
 
We recommend '''not''' to save your plugin files inside the folders of a ''Dynamo'' installation, but in your own folders.
 
We recommend '''not''' to save your plugin files inside the folders of a ''Dynamo'' installation, but in your own folders.
 +
 +
=== System executables ===
 +
 +
If you put your plugin as a system executable, the same naming convention will be used. You just need to skip the file extension <tt>.m</tt>.
  
 
==Linking plugin and project==
 
==Linking plugin and project==
Line 17: Line 32:
 
  <tt>dvput ptest plugin_post_r1 1</tt>
 
  <tt>dvput ptest plugin_post_r1 1</tt>
 
Then, you pass the plugin itself:
 
Then, you pass the plugin itself:
  <tt>dvput ptest plugin_post_order_r1 dynamo_plugin_thisIsAnExample</tt>
+
  <tt>dvput ptest plugin_post_order_r1 myExample()</tt>.
if necessary with parameters.
+
This assumes that the plugin script has the name <tt>dynamo_plugin_post_myExample.m</tt>.
  <tt>dvput ptest plugin_post_order_r1 dynamo_plugin_thisIsAnExample parameterForPlugin1 parameterForPlugin2</tt>
+
If necessary, you can also pass parameters.
 +
  <tt>dvput ptest plugin_post_order_r1 myExample parameterForPlugin1 parameterForPlugin2</tt>.
 +
The parentheses are necessary to tell ''Dynamo'' that the plugin is a Matlab script. If you skip them
  
 
Do not forget to unfold again the project before running it.
 
Do not forget to unfold again the project before running it.
  
 
==Syntax of a plugin script==
 
==Syntax of a plugin script==
 +
  
 
===Input arguments===
 
===Input arguments===
 +
 +
The execution pipeline of ''Dynamo'' will deliver to the plugin function a text card with all the parameter values used in the project for the iteration that just got executed. It also identifies the iteration number currently on execution.
 +
 +
Thus, when creating your script, your first input variable should be reserved to accept this file.
 +
 +
<tt>function dynamo_plugin_post_example(iterationCard);
  
 
===Location of iteration and round ===
 
===Location of iteration and round ===
 +
 +
==== In Matlab ====
 +
In a Matlab-written plugin, you can use some tools in order to extract the information in the <tt>iterartionCard</tt> text file delievered to the plugin script in runtime
 +
 +
<nowiki>
 +
scard=dynamo_read(iterationCard);
 +
 +
%  1 from which iteration it is being executed:
 +
ite = scard.database_ite;
 +
 +
%  2 from which project it is being invoked
 +
name_projec t= scard.name_project;
 +
 +
% 3 we read the corresponding virtual project
 +
vpr=dynamo_vpr_load(name_project)
 +
</nowiki>
 +
 +
==== In a non-matlab executable ====
 +
 +
If your plugin is an executable, you''ll have to parse the input text file in order to find the parameters needed by the plugin.
  
 
===The project database===
 
===The project database===
  
 +
The execution pipeline of ''Dynamo'' defines a series of items in a database, corresponding to the elements produced and used during the averaging procedure.
 +
 +
The typical action of a plugin is typically to access some of these elements, analyze and operate of them, and possibly replace some of them with the results of an operation defined inside the plugin.
 +
 +
The most important items are:
 +
* <tt>refined_table</tt>
 +
* <tt>average</tt>
 +
 +
They represent the results of the iteration in course. Also, they are passed to the next iteration to produce the starting positions and reference for the next computation.
 +
 +
==== In Matlab====
 +
 +
Both <tt>refined_table</tt> and <tt>average</tt> are identified by iteration and reference number (in case you have a multireference project);
 +
 +
In order to get the respective files inside your script:
 +
 +
<nowiki>
 +
fileTable  =  dynamo_database_locate('refine_table',vpr,'ite',1,'ref',1);
 +
fileAverage =  dynamo_database_locate('average',vpr,'ite',1,'ref',1);
 +
</nowiki>
 +
 +
Nopte that you can get  the indices of the references that are still alive in the current iteraration by:
 +
<tt>currentRefs = dpkproject.pipeline.getSurvivingReferences(vpr,ite);</tt>
 +
 +
==== In a non-matlab executable ====
 +
 +
Working outside Matlab, the user will have two tasks:
 +
# providing the right paths for all project items
 +
# using their own IO tools to read and write map and table files.
 +
 +
====Other database items====
 +
 +
A complete list of database items and their identification methods can be created with <tt>ddhelp</tt>
  
 
==Example script==
 
==Example script==
 +
Here we show a plugin for a simple task: we want to tell ''Dynamo'' to ignore the average of an iteration and use a
 +
 +
 +
=== Script contents ===
 +
You can create a text file
 +
 +
<tt> dynamo_plugin_post_frozenTemplate</tt>
 +
 +
and put it inside your matlab path.
 +
 +
<nowiki>
 +
function dynamo_plugin_post_frozenTemplate(file_card);
 +
 +
% Reads the file
 +
scard=dynamo_read(file_card);
 +
 +
% Now the plugin locates:
 +
%  1 from which iteration it is being executed:
 +
ite=scard.database_ite;
 +
 +
%  2 from which project it is being invoked
 +
name_project=scard.name_project;
 +
 +
% We read the corresponding virtual project
 +
vpr=dynamo_vpr_load(name_project);
 +
 +
 +
% informs the user that this plugin has been enterered
 +
 +
disp(repmat('-',[1,60]));
 +
disp(' ');
 +
disp('  [multireference_tutorial] entering post processing plugin.');
 +
disp(' ');
 +
 +
disp(sprintf('This plugin is operating iteration: %d',ite));
 +
 +
% we check where the result of the average is located.
 +
averageFile = dynamo_database_locate('average',name,'ite',ite,'ref',1);
 +
 +
% as we are going to over
 +
backUpLocation=dynamo_database_locate('average',vpr,'ite',...
 +
        ite,'ref',ref,'prepend','OLD');
 +
 +
% nwo we set our own file in the database position for the result of the
 +
% averaging
 +
myReference = 'map.em';
 +
 +
% we put our map in the database location of the file
 +
copyfile(myReference,averageFile);
 +
 +
% that's it. We have "hijacked" the regular Dynamo pipeline.
 +
% The contents of the file 'map.em' will be read by next iteration
 +
% in order to produce its reference.
 +
 +
disp('--- end of plugin ---');
 +
</nowiki>

Latest revision as of 18:07, 9 March 2017

Postprocessing plugins are a tool for customization of the alignment procedure in Dynamo. With this kind of plugin, the user can define an action that Dynamo will execute after each iteration during a project alignment.

Applications of plugins

Users have devised the use of plugins in very different scenarios and with very different levels of complexity. We just mention below some some examples. In square bracket we describe the difficulty of the writing the interface between the plugin and Dynamo, not the complexity of the plugin itself.

  • [Simple] Online definition of mask, depending on the average computed in the iteration.
  • [Simple] Taylored definition of symmetrization operator.
  • [Simple] User-defined thresholding for subtomograms from different tomograms.
  • [Medium] Resolution dependent scanning of angular scan parameters.
  • [Medium] Customization of golden-standard policy.
  • [Advanced] Fine tuning of MRA policy by PCA classification after each iteration.

Naming of a plugin script

Matlab plugin

Dynamo requires that the plugin script itself is a matlab .m function named with the pattern:

dynamo_plugin_post_PLUGINNAME.m

where PLUGINNAME is the shortname of the plugin.

The script should be in the execution path of Matlab (i.e., you should use addpath on the directory where you have defined your plugin). We recommend not to save your plugin files inside the folders of a Dynamo installation, but in your own folders.

System executables

If you put your plugin as a system executable, the same naming convention will be used. You just need to skip the file extension .m.

Linking plugin and project

You need two orders. With the first one you tell Dynamo that round 1 (for instance) will look for a plugin after normal Dynamo execution. For a project called ptest, you would write:

dvput ptest plugin_post_r1 1

Then, you pass the plugin itself:

dvput ptest plugin_post_order_r1 myExample().

This assumes that the plugin script has the name dynamo_plugin_post_myExample.m. If necessary, you can also pass parameters.

dvput ptest plugin_post_order_r1 myExample parameterForPlugin1 parameterForPlugin2.

The parentheses are necessary to tell Dynamo that the plugin is a Matlab script. If you skip them

Do not forget to unfold again the project before running it.

Syntax of a plugin script

Input arguments

The execution pipeline of Dynamo will deliver to the plugin function a text card with all the parameter values used in the project for the iteration that just got executed. It also identifies the iteration number currently on execution.

Thus, when creating your script, your first input variable should be reserved to accept this file.

function dynamo_plugin_post_example(iterationCard);

Location of iteration and round

In Matlab

In a Matlab-written plugin, you can use some tools in order to extract the information in the iterartionCard text file delievered to the plugin script in runtime

scard=dynamo_read(iterationCard);

%   1 from which iteration it is being executed:
ite = scard.database_ite;

%   2 from which project it is being invoked
name_projec t= scard.name_project;

% 3 we read the corresponding virtual project 
vpr=dynamo_vpr_load(name_project)

In a non-matlab executable

If your plugin is an executable, youll have to parse the input text file in order to find the parameters needed by the plugin.

The project database

The execution pipeline of Dynamo defines a series of items in a database, corresponding to the elements produced and used during the averaging procedure.

The typical action of a plugin is typically to access some of these elements, analyze and operate of them, and possibly replace some of them with the results of an operation defined inside the plugin.

The most important items are:

  • refined_table
  • average

They represent the results of the iteration in course. Also, they are passed to the next iteration to produce the starting positions and reference for the next computation.

In Matlab

Both refined_table and average are identified by iteration and reference number (in case you have a multireference project);

In order to get the respective files inside your script:

fileTable   =  dynamo_database_locate('refine_table',vpr,'ite',1,'ref',1);
fileAverage =  dynamo_database_locate('average',vpr,'ite',1,'ref',1);

Nopte that you can get the indices of the references that are still alive in the current iteraration by:

currentRefs = dpkproject.pipeline.getSurvivingReferences(vpr,ite);

In a non-matlab executable

Working outside Matlab, the user will have two tasks:

  1. providing the right paths for all project items
  2. using their own IO tools to read and write map and table files.

Other database items

A complete list of database items and their identification methods can be created with ddhelp

Example script

Here we show a plugin for a simple task: we want to tell Dynamo to ignore the average of an iteration and use a


Script contents

You can create a text file

 dynamo_plugin_post_frozenTemplate

and put it inside your matlab path.

function dynamo_plugin_post_frozenTemplate(file_card);

% Reads the file
scard=dynamo_read(file_card);

% Now the plugin locates:
%   1 from which iteration it is being executed:
ite=scard.database_ite;

%   2 from which project it is being invoked
name_project=scard.name_project;

% We read the corresponding virtual project
vpr=dynamo_vpr_load(name_project);


% informs the user that this plugin has been enterered

disp(repmat('-',[1,60]));
disp(' ');
disp('   [multireference_tutorial] entering post processing plugin.');
disp(' ');

disp(sprintf('This plugin is operating iteration: %d',ite));

% we check where the result of the average is located.
averageFile = dynamo_database_locate('average',name,'ite',ite,'ref',1);

% as we are going to over
backUpLocation=dynamo_database_locate('average',vpr,'ite',...
        ite,'ref',ref,'prepend','OLD');

% nwo we set our own file in the database position for the result of the 
% averaging
myReference = 'map.em';

% we put our map in the database location of the file
copyfile(myReference,averageFile);

% that's it. We have "hijacked" the regular Dynamo pipeline.
% The contents of the file 'map.em' will be read by next iteration
% in order to produce its reference.

disp('--- end of plugin ---');