Difference between revisions of "EMBO workshop 2021"

From Dynamo
Jump to navigation Jump to search
m
 
(52 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30) as part of the EMBO workshop for Image processing in cryoEM.
+
The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30, London time) as part of the EMBO workshop for Image processing in cryoEM.
−
 
 
−
= Tutorials =
 
−
==Connection==
 
−
Please refer to your course material to access your '''two sets''' of credentials:
 
  
 +
=Connect to Guacamole and run Dynamo=
 +
=== Credentials ===
 +
Please refer to your personal course material to access your '''two sets''' of credentials:
 
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.
 
# A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.
−
# A set of course credentials to enter the actual machines where the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.
+
# A set of course credentials to enter the actual machines on which the course will be done: the '''EMBO21 UserID'''. These are unique to each participant.
−
 
 
  
−
=== Connecting to guacamole ===
+
=== Connect to Guacamole ===
−
* Go to:  
+
To connect to Guacamole, first go to the following link in your browser:  
 
  <tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt>
 
  <tt>https://login.cryst.bbk.ac.uk/guacamole/#/ </tt>
  
−
in your browser. '''At this stage you should maximise your browser''', otherwise you will have visualisation problems when opening Dynamo.
+
'''At this stage you should maximize your browser window''', otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.
−
* Enter your Guacamole credentials.
 
  
 
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]
 
[[File:embo_2021_1.png|thumb|center|400px|Entering guacamole server.]]
−
=== Connecting to your machine ===
+
=== Connect to your machine ===
−
Now you need to select a machine. Please then click on +em and then +Servers - Public and select server Zeus-0-14.
+
Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).
 
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]
 
[[File:embo_2021_2.png|thumb|center|300px|Select a machine.]]
 
When you select the machine, you will prompted to use your EMBO21 UserID
 
When you select the machine, you will prompted to use your EMBO21 UserID
Line 30: Line 27:
 
  <tt> /d/embo2021/u/emboXX</tt>
 
  <tt> /d/embo2021/u/emboXX</tt>
 
where XX stands for your actual workshop participant number.
 
where XX stands for your actual workshop participant number.
 +
 +
=== Data location ===
 +
 +
The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command:
 +
 +
<tt>cd prac-5</tt>
 +
 +
or
 +
 +
<tt>cd prac-6</tt>
 +
 +
respectively. Double-check if the data is present by typing
 +
 +
<tt>ls</tt>
 +
 +
You should see the following files in both directories:
 +
 +
[[File:embo_files.png|thumb|center|500px|files]]
 +
 +
During the ''Dynamo'' workshop, work '''only''' within these two directories.
  
 
=== Load and run Dynamo ===
 
=== Load and run Dynamo ===
  
−
First you load the standalone version of Dynamo:
+
To use ''Dynamo'', first load the '''standalone''' version of ''Dynamo'' with the command:
 
  <tt> module load dynamo/v1.1.524</tt>
 
  <tt> module load dynamo/v1.1.524</tt>
−
and then it can be run by simply typing:
+
Then it can be run by simply typing:
 
  <tt> dynamo</tt>
 
  <tt> dynamo</tt>
  
Line 42: Line 59:
 
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command
 
''Dynamo'' specific commands are then typed into this so called ''Dynamo console''. For example, try to type the ''Dynamo'' command
 
  <tt>dynamo_version</tt>
 
  <tt>dynamo_version</tt>
−
to see the current version that you are using. Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.
+
to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects):
  
−
== Dynamic of the course ==
+
<tt>mbparse.multicore.checkPhysicalCores</tt>
−
The course is divided into different modules, described in the time schedule below. Each module is structured in three stages:
 
  
−
# '''Demostration:''' All students are connected to the main room. The goals of the module will be explained by an instructor.
+
'''Note:''' Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.
−
# '''Individual work:''' Students go the assigned breakout rooms to work on the tutorials independently. An instructor will be assigned to each breakout room to help with questions.
 
−
# '''Common discussion:''' In the last 5 minutes of the module, we come back to the general breakout room to wrap up the module.
 
  
−
=== Data location ===
+
'''Note:''' We suggest to open a new tab in the terminal (''File -> Open Tab''), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit ''Dynamo'' every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the ''Dynamo'' console you exit ''Dynamo'' and lose your workspace.
−
Each module has one or several tutorials. The data of the tutorials is sometimes synthetic and created on the fly, and sometimes a data set is necessary. The data will be available in the filesystem. The general repository for the data is:
 
−
<tt>/d/embo2021/d/dynamo</tt>
 
  
−
== Time schedule ==
+
=== Chimera ===
 +
To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside ''Dynamo'').
  
−
This time schedule is tentative <br />
+
<tt>/usr/local/bin/chimera myFile.mrc</tt>
  
−
Day 1: Tomography
+
= Schedule =
 +
The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:
 +
# '''Demonstration:''' All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.
 +
# '''Individual work:''' Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.
 +
# '''Closure:''' During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.
  
 
{| class="wikitable"
 
{| class="wikitable"
−
|'''Time'''
+
! colspan="4" | '''Day 1'''
−
|'''Module'''
 
 
|-
 
|-
−
| 14:00
+
|'''Time (London)'''
 +
|'''Activity
 +
|'''Topic'''
 +
|'''Comment'''
 +
|-
 +
| 14:00 - 14:30
 +
| Presentation
 
| Introduction
 
| Introduction
 +
|
 +
|-
 +
| 14:30 - 15:30
 +
| Tutorial
 +
| [[Walkthrough_on_GUI_based_tilt_series_alignment_(EMBO2021)| Tilt series alignment with GUI]]
 +
|
 
|-
 
|-
−
| 14:30
+
| 15:30 - 15:45
−
| Tilt series alignment: manually
+
| Break
 +
|
 +
|
 
|-
 
|-
−
| 15:00
+
| 15:45 - 16:45
−
| Tilt series alignment: GUI
+
| Tutorial
 +
| [[Walkthrough_on_command_line_based_tilt_series_alignment_(EMBO2021) | Tilt series alignment with command line]]
 +
|
 
|-
 
|-
−
| 15:30
+
| 16:45 - 17:00
−
| Tilt series alignment: command line
+
| Break
 +
|
 +
|
 
|-
 
|-
−
| 16:00
+
| 17:00 - 17:30
−
| Break
+
| Tutorial
 +
| [[Walkthrough_on_filament_models | Models for filaments]]
 +
|
 
|-
 
|-
−
| 16:30
+
| 17:30 - 18:00
−
| Models: catalogue system
+
| Tutorial
 +
| [[Walkthrough_on_membrane_models | Models for surfaces]]
 +
|
 
|-
 
|-
−
| 17:00
+
| Optional: 18:00
−
| Models: helical symmetry 
+
| Tutorial
 +
| [[Walkthrough for template matching | Template matching]]
 +
|
 
|-
 
|-
−
| 17:30
+
! colspan="4" | '''Day 2'''
−
| Models: vesicles and surfaces
 
 
|-
 
|-
−
| 18:00
+
|'''Time (London)'''
−
| Template matching
+
|'''Activity
−
|}
+
|'''Topic'''
−
 
+
|'''Comment'''
−
 
 
−
Day 2: Subtomogram averaging
 
−
 
 
−
{| class="wikitable"
 
−
|'''Time'''
 
−
|'''Module'''
 
 
|-
 
|-
−
| 14:00
+
| 14:00 - 14:15
 +
| Presentation
 
| Introduction
 
| Introduction
 +
|
 
|-
 
|-
−
| 14:30
+
| 14:15 - 15:30
−
| Getting started: synthetic thermosomes
+
| Tutorial
 +
| [[Starters_guide_(EMBO2021) | Starters guide]]
 +
|
 
|-
 
|-
−
| 15:30
+
| 15:30 - 15:45
−
| Advanced tutorial: FHV
+
| Break
 +
|
 +
|
 
|-
 
|-
−
| 16:00
+
| 15:45 - 17:45
−
| Break
+
| Tutorial
 +
| [[Advanced_starters_guide_(EMBO2021) | Advanced starters guide]]
 +
| Take a 15 min break individually during this tutorial
 
|-
 
|-
−
| 16:30
+
| 17:45 - 18:00
−
| FHV: continuation
+
| Presentation
−
|-
+
| Closing remarks
−
| 17:00
+
|
−
| Seed oversampling
 
 
|-
 
|-
−
| 18:00
+
| Optional: 18:00
−
| High resolution pipelines
+
| Tutorials
−
|-
+
| [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | Virus like particles]] or [[Walkthrough for lattices on vesicles| Spherical geometries]]
−
| 18:20
+
|
−
| Closing remarks
 
 
|}
 
|}
−
 
−
= Modules =
 
−
 
−
== Alignment of tilt series==
 
−
We will align the same tilt series tilt series with two different approaches. We prepared the first tilt series from the EMPIAR entry 10164, depicting a set of virus like particles (VLP). Move to the directory that contains this data using the command:
 
−
 
−
  <tt>cd prac-5</tt>
 
−
 
−
Double-check by typing
 
−
 
−
  <tt>ls</tt>
 
−
 
−
to see all files in the directory. One file should be the following tilt series:
 
−
 
−
  <tt>b001ts001.mrc</tt>
 
−
 
−
This is a binned version (for quicker processing) with the pixel size is 2.7 Angstrom (original deposition is 1.35).
 
−
 
−
=== Automated alignment of tilt series through the GUI ===
 
−
[[Walkthrough on GUI based tilt series alignment ]]
 
−
 
−
=== Automated alignment of tilt series through the command line ===
 
−
[[Walkthrough on command line based tilt series alignment ]]
 
−
 
−
== Models ==
 
−
=== Catalogues ===
 
−
=== Helical models ===
 
−
[http://{{SERVERNAME}}/w/doc/misc/modelFilament.pdf tutorial ]
 
−
=== Vesicle and surface models ===
 
−
[http://{{SERVERNAME}}/w/doc/misc/modelMembrane.pdf tutorial on] membrane modeling with <tt> dmslice </tt>
 
−
=== Reusing models ===
 
−
Reusing model workflows ([[Walkthrough model worfklow reuse| walkthrough]])
 
−
== Template matching ==
 
−
[[Walkthrough for template matching | walkthrough]] for automated identification of proteosomes on a real tomogram through template matching. (~1 hour) <br />
 
−
The data is available at:
 
−
<tt>/d/embo2021/d/dynamo/t20s.mrc</tt>
 
−
 
−
An example of template and its mask are provided as 'average32.em' and 'maskTight32.em'.
 
−
 
−
== Subtomogram averaging ==
 
−
=== Introduction ===
 
−
Guided presentation:
 
−
 
−
* [http://{{SERVERNAME}}/w/doc/misc/conceptsSheet.pdf Basic ''Dynamo'' jargon]
 
−
* [http://{{SERVERNAME}}/w/doc/misc/introductionTablesAndData.pdf tutorial on] basic elements: help, data and metadata formats.
 
−
* [http://{{SERVERNAME}}/w/doc/misc/introductionAlignmentProjects.pdf tutorial on] the basic concept in ''Dynamo'' alignment: the ''project''.
 
−
 
−
Working on your own:
 
−
 
−
Basic [[Starters guide | walkthrough]]: creating a catalogue, picking particles, launching a project.
 
−
 
−
=== Advanced starters guide ===
 
−
Complete the  [[advanced starters guide]] (~2 hours)
 
−
The data can be found in
 
−
<tt>/d/embo2021/d/dynamo/crop.rec</tt>
 
−
To use chimera path you need in the tutorial is
 
−
<tt> TBI </tt>
 
−
 
−
=== Seed oversampling ===
 
−
 
−
Tutorial [[Walkthrough for lattices on vesicles| from densely packed spherical geometry]] (~1 hour). The data is available at:
 
−
<tt>/d/embo2021/d/dynamo/v17.rec</tt>
 
−
 
−
Since the final alignment project is run on the standalone version of Dynamo, the 'destination' parameter should be 'standalone' instead of 'matlab_parfor'
 
−
 
−
== High resolution pipelines ==
 
−
 
−
=== VLPs ===
 
−
Exercise to train a realistic case including all processing steps from catalogue creation to final structure using a reduced dataset. Start with the [[Getting a Structure from Multiple Tomograms of HIV Capsids | exercise]] and refer to the complete [[Getting_a_Structure_from_Multiple_Tomograms_of_HIV_Capsids_(walkthrough) | walkthrough]] for guidance or the solution.
 
−
 
−
=== Interconnectivity tools ===
 
−
Alister:
 
−
Not a tutorial per se: this is a presentation on a set of tools to interconnect Dynamo, M and other software.
 
−
 
−
A set of tools have been developed which provide a means to use the geometrical tools and alignment projects from Dynamo with data preprocessed in Warp.
 
−
These tools mean that particle picking and alignment can be performed in Dynamo prior to multi-particle refinement in M.
 

Latest revision as of 17:07, 11 September 2021

The tutorials listed here take place virtually on the 11th and 13th of September of 2021 (14:00-18:30, London time) as part of the EMBO workshop for Image processing in cryoEM.

Connect to Guacamole and run Dynamo

Credentials

Please refer to your personal course material to access your two sets of credentials:

  1. A set of credentials to enter Guacamole (the platform for connecting via browser). These are common to all participants.
  2. A set of course credentials to enter the actual machines on which the course will be done: the EMBO21 UserID. These are unique to each participant.

Connect to Guacamole

To connect to Guacamole, first go to the following link in your browser: 

https://login.cryst.bbk.ac.uk/guacamole/#/

At this stage you should maximize your browser window, otherwise you might have visualization problems when opening Dynamo. You should see the following window. Enter your Guacamole credentials.

Entering guacamole server.

Connect to your machine

Now you need to select a machine. Please then click on +em and then +Servers - Public and select your dedicated server (in this example it is Zeus-0-14).

Select a machine.

When you select the machine, you will prompted to use your EMBO21 UserID

Entering your personal course credentials.

This should access your virtual Linux XFCe desktop, which should look like this in your browser

Linux XFCe desktop

If you open a terminal (with the terminal icon) you can test by typing 

pwd

that you are in the correct path, which should be: 

 /d/embo2021/u/emboXX

where XX stands for your actual workshop participant number.

Data location

The data used in the tutorials is already available for you. Move to the directory of the corresponding practical using the command: 

cd prac-5

or 

cd prac-6

respectively. Double-check if the data is present by typing 

ls

You should see the following files in both directories:

files

During the Dynamo workshop, work only within these two directories.

Load and run Dynamo

To use Dynamo, first load the standalone version of Dynamo with the command: 

 module load dynamo/v1.1.524

Then it can be run by simply typing: 

 dynamo
Dynamo console started.

Dynamo specific commands are then typed into this so called Dynamo console. For example, try to type the Dynamo command 

dynamo_version

to see the current version that you are using. Type the following command to check how many CPUs your machine has. You might use this number later for tasks that require parallel processing (e.g., alignment projects): 

mbparse.multicore.checkPhysicalCores

Note: Copy paste of commands from your local desktop into the dynamo standalone console on the remote Linux XFCe desktop might not always work. We suggest to additionally open the tutorials that you will be working on in a browser inside the remote Linux XFCe desktop itself, in case you need to copy paste long commands.

Note: We suggest to open a new tab in the terminal (File -> Open Tab), in which you navigate to the same directory. You can then use this tab for normal linux commands so you do not have to quit Dynamo every time you want to do something else (e.g., running alignment projects or using chimera). Be aware that if you press ctrl+c within the Dynamo console you exit Dynamo and lose your workspace.

Chimera

To use chimera, use the following command (replace 'myFile.mrc' with your file) in the normal terminal (outside Dynamo). 

/usr/local/bin/chimera myFile.mrc

Schedule

The course is divided into short presentations and tutorials, as described in the time schedule below. For the presentations, we all meet in the main room. The tutorials are structured as follows:

  1. Demonstration: All students stay connected to the main room for the first minutes, where the goals of the tutorial are briefly explained.
  2. Individual work: Students go the assigned breakout rooms to work on the material independently. An instructor will be assigned to each breakout room to help with questions.
  3. Closure: During the last 5 minutes we come back to the general breakout room to wrap up the tutorial.
Day 1
Time (London) Activity Topic Comment
14:00 - 14:30 Presentation Introduction
14:30 - 15:30 Tutorial Tilt series alignment with GUI
15:30 - 15:45 Break
15:45 - 16:45 Tutorial Tilt series alignment with command line
16:45 - 17:00 Break
17:00 - 17:30 Tutorial Models for filaments
17:30 - 18:00 Tutorial Models for surfaces
Optional: 18:00 Tutorial Template matching
Day 2
Time (London) Activity Topic Comment
14:00 - 14:15 Presentation Introduction
14:15 - 15:30 Tutorial Starters guide
15:30 - 15:45 Break
15:45 - 17:45 Tutorial Advanced starters guide Take a 15 min break individually during this tutorial
17:45 - 18:00 Presentation Closing remarks
Optional: 18:00 Tutorials Virus like particles or Spherical geometries
Retrieved from "https://wiki.dynamo.biozentrum.unibas.ch/w/index.php?title=EMBO_workshop_2021&oldid=4223"