Tilt series alignment workflow folder

Article under construction, sorry

The folder .AWF contains different categories of files:

Workflow object

The workflow object itself appears as a file called object.mat. This is a binary file, which is used internally by Dynamo when performing alignments and reconstructions. Although it contains information about every aspect of the workflow (used parameters, location of input and output data, etc), regular users rarely need to explicitly manipulate it for standard operations.

When it is needed (see examples below), It can be called into memory with the generic dread command:

w = dread('object.mat')

Remember to use the whole path if you are not in the alignment workflow folder.

Raw stack

The raw data (the stack) does not need to be physically stored inside the workflow folder. The data linked to a workflow folder can be consulted through the workflow object. If you have the workflow in memory with the variable name w (for instance, variable name is arbitrary), you can get the filename in the io area of the object w as:

w.io.datagates.items.matrix.getFilename() 

Angles

Nominal tilt angles

The nominaltilt angles of the raw stack are given as a text file with the default name:

nominal.tlt

This file is normally generated by Dynamo when the alignment workflow is created. Note that these angles could also be stored in a different location and linked to the workflow, but this procedure is non-standard.

In any case, the text file should contain an angle (in degrees) for each micrograph in the data stack, and appear in the same order.

Explicitly rejected tilt angle indices

Some micrographs might be manually rejected because of visually obvious errors, like defocus or shift jumps. An explicit decision by the user on eliminating a given (set of) micrograph(s) can be taken a priori, or during the alignment procedure with the GUI. In both cases, the indices are referred to the nominal angles seen by the workflow, and are stored in a text file called discardedTiltIndicesManual.txt .

Tilts included in this file are not counted by Dynamo when computing chains of markers.

Tilts rejected a priori

The user has the option of manually creating the file manuallyDiscardedTiltIndices.txt before starting the workflow (as automatic process or through the GUI). This option is normally used for batch processing using unsupervised alignment. Before launching the automatic alignment process, the user quickly supervises the different tilt series in the batch and marks the tilts that are not suitable for processing (typically because of defocus or shift jumps ).

Tilts rejected during manual alignment

When operating the GUI, some micrographs can be identified as invalid. They can be marked on the fly, and the file discardedTiltIndicesManual.txt will be updated. Inversely, one can restore tilts that had been deselected a-priori.

Tilts rejected during automatic alignment procedure

Tilts can be rejected by the alignment algorithm if they don't contain enough markers, or if the error in the markers detected on the tilts is too big. The tilt indices that fall in this category are stored in the file discardedTiltIndicesComputational.txt

Reconstruction angles

The indices of the angles from nominal.tlt that survive both manual and computational rejection are kept in the file reconstructionTiltIndices.txt

Stacks

Raw binned stack

Dynamo uses a binned, unaligned matrix in order to speed up the alignment procedure. It is stored under the name workingMatrix.

Aligned binned stack

The aligned binned stack is stored in the file alignedBinnedStack.mrc. It is generated by applying the fitter generated by the finalWorkingMarkers to the raw binned stack.

This stack does notcontain the micrographs corresponding to tilts that have been rejected, both manually or computationally. Thus, the order an identity of micrographs in the stack corresponds to the order and identity of angles indexed by reconstructionTiltIndices

Aligned full stack

The aligned full stack is stored in alignedFullStack.mrc

Markers

Files representing markers contain (among further information) the x,y coordinates of the markers on each tilt micrograph where the marker is present.

Makers files are stored in .dms files, which are binary files that contain Dynamo-native markerSet objects. They can be converted into an indexed cloud (numerical matrix) by reading the .dms file into memory and then using getIndexedCloud();

m = dread('markers.dms'); 

and then

 ic = m.getIndexedCloud();

Here, the indexed cloud ic is a matrix with four columns, each row represents the coordinates of each observed gold bead in this order:

• x position in micrograph
• y position in micrograph
• index of the micrograph (referred to the order in the nominal tilt angles)
• index of the marker itself

Markers on binned stack

The process of locating gold beads is initially performed on the binned matrix.

Rough alignment

Those are tracks of markers found by comparing shifted versions of micrographs. They do not include any 3D model of the projection geometry, and are used internally by Dynamo to provide an initial track identity to the gold markers.

They are contained in the file roughMarkers.dms.

Final tracks

They are contained in the file finalWorkingMarkers.dms.

Marker on full stack

The positions on the full stack lay in the file markers.dms.

They are computed by fitting the positions in the finalWorkingMarkers (originally computed by locating the centers of binned gold beads) to full resolution gold beads.

Reconstructions

Metadata