Difference between revisions of "Tips for management of tomographic data sets"

From Dynamo
Jump to: navigation, search
Line 13: Line 13:
 
Here, <tt><repository></tt> is some location in your file system, probably a file share destined for massive storage (at least Tbs)
 
Here, <tt><repository></tt> is some location in your file system, probably a file share destined for massive storage (at least Tbs)
  
== Organizing the tilt series==
+
== Organizing the tilt series stacks==
  
We organize our tilt series in batches. Each batch represents customarily tilt series acquired in the same conditions or in the same session, but there is no strict rule. Each batch will have its own folder under the <tt>data</tt> repository folder. The convention for the batch folder is a 'b' character followed with an integer (zero-padded to three figures).
+
=== Batches ===
 +
We organize our tilt series in batches. Each batch represents customarily tilt series acquired in the same conditions or in the same session, but there is no strict rule. Each batch will have its own folder under the <tt>data</tt> repository folder. The convention for the batch folder is a <tt>'b'</tt> character followed with an integer (zero-padded to three figures).
 
  <nowiki><repository>/data/b001
 
  <nowiki><repository>/data/b001
 
<repository>/data/b002
 
<repository>/data/b002
Line 21: Line 22:
 
</nowiki>
 
</nowiki>
  
 +
==Tilt series folder===
 +
 +
Inside each batch folder we create a separate folder for each tilt series. The convention for the batch folder is a <tt>'ts'</tt> character followed with an integer (zero-padded to three figures).
 +
<nowiki><repository>/data/b001/ts012
 +
<repository>/data/b001/ts016
 +
</nowiki>
  
 
Our acquisition system Focus delivers for each tilt series:
 
Our acquisition system Focus delivers for each tilt series:

Revision as of 12:55, 28 April 2017

The guidelines sketched here are a walkthrough on how we organize our tomographic data sets in the BioEM lab of the University of Basel.

We use a series of conventions on the way we storage of the raw data (i.e.) tilt series that makes easy to keep track on the processing steps already performed (alignment, reconstruction), the location of the intermediate results, and their binding into a Dynamo catalogue.

The raw data is stored in a tree below a repository folder, which will contain all the data (tilt series and tomograms) related to the tomography projects. The repository folder is created with the structure:


<repository>/data
<repository>/ctf

Here, <repository> is some location in your file system, probably a file share destined for massive storage (at least Tbs)

Organizing the tilt series stacks

Batches

We organize our tilt series in batches. Each batch represents customarily tilt series acquired in the same conditions or in the same session, but there is no strict rule. Each batch will have its own folder under the data repository folder. The convention for the batch folder is a 'b' character followed with an integer (zero-padded to three figures).

<repository>/data/b001
<repository>/data/b002
<repository>/data/b015

Tilt series folder=

Inside each batch folder we create a separate folder for each tilt series. The convention for the batch folder is a 'ts' character followed with an integer (zero-padded to three figures).

<repository>/data/b001/ts012
<repository>/data/b001/ts016

Our acquisition system Focus delivers for each tilt series:

  • a tilt series file, i.e., a stack file in .mrc format.
  • a metadata file .star file.



Creating the reconstructions

Creating the first catalogue

Prebinning of catalogue volumes

Visualizing the tomograms