Shorthand tool for restricting a table according to several criteria
  simultaneously. 
  
  The restriction will produce a new subtable: a subset of the original
  table comprising the same number of columns (properties) but a possibly
  reduced set of particles.
 
  
 
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
   INPUT
 
    table:          as cell array, filename, Matlan numerical array
                    or database query.
 
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
    * Particle selection by Parameter/Value
 
 
    'tags':         direct selection
 
                    - a set of values
                    - a .tags file
                    - a code word:
                      'all'    : to force all particles listed in the table
                      '*'      : to force all particles listed in the table
                      'average': to include only particles marked in column 3
                                of the table.
 
                       - default action: 'average'
 
 
     EXAMPLE
 
     tableout = dtgrep('example.tbl','tags',1:10);
     dtgrep table -tags [1:10] -ws tableout;
 
 
    'exclude' :       direct exclusion of sets of tags
 
     EXAMPLE
 
     tableout = dtgrep('example.tbl','exclude',1:10);
     dtgrep table -exclude [1:10] -ws tableout;  
 
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
   'match_data','md'    : data folder
                          Selects the tags available in the given data
                          folder (or generic Dynamo data container)
    
   SYNTAX
   dtgrep t1.tbl -md myFolder -o t2.tbl 
  
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
   Different ways to further restrict the set of tags:
  
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
               Direct restriction operators:
               -----------------------------
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
   'table_restrict','tr' 
    
    This is probably the most usefule parameter flag when dealing
    interactively in a console (both Matlab or Dynamo consoles)
    
    A  'table_restrict' is just a string that accumulates restrictions
    to be operated on a table. 
    Each restriction expressed a limitation on the values acceptable for 
    a given 'property' or column of the table.
 
    Each "property" can be indicated:
       - by a string (as 'sref','cc','tilt',etc... )
                      see "table_help" for options)
       - by a column number preceded by "&"
 
    EXAMPLES:
 
    dtgrep example.tbl -tr cc<0.6 -o t.tbl;
 
    % or
 
    dtgrep example.tbl -tr &10<0.6 -o t.tbl;
 
    % selects the particles whos cross correlation is below 0.6
    % (and writes the resulting table in the file "t.tbl")
 
    dtgrep example.tbl -tr 'cc<0.6;40[top]   or  [bottom]
     where  can be given as column number preceded by & or by name:
 
     EXAMPLE
     
     dtplot example.tbl -tr (abs)tilt[top]10
 
     % selects the 25 particles with best scores for the cc column (column 10)
     dtgrep example.tbl -tr &10[top]25
 
     % If the table contains less than 25 particles, all the available ones
     % will be picked.
 
     
     The top/bottom notation is compatible with the OR notation:
 
     EXAMPLE
 
     dtplot example.tbl -tr (abs)tilt[bottom]30||(abs)tilt[top]30 
     
     % plots the 30 particles closest to the north direction
     % and the 30 particles closes to the south direction
 
 
     Fractional Top/bottom restrictions
     ----------------------------------
 
     Similar to the previous ones, but the passed number is understood
     as a fraction. 
 
     [top]     is replaced by [topf]
     [bottom]  is replaced by [bottomf]
    
     EXAMPLE
 
     dtplot example.tbl -tr cc[topf]0.33
 
     % selects the 33 PERCENT better particle
 
 
     Angular restrictions
     --------------------
 
     You can select particles close to a direction with the syntax:
     [ang]tdrot,tilt,narot,aperture.
 
     EXAMPLE
 
     dtplot example.tbl -tr '[ang]0,45,0,30';
 
     % Note the difference with
 
     dtplot example.tbl -tr '[mang]0,45,0,30';
   
     % which adds the two coned 
 
     Summary: Syntax of restrictions
     ------------------------------
 
     Summarizing: each simple restriction is of one of these types,
     "property" meaning:
      - a string defined in table_help (i.e. 'cc','sref','tilt','etc')
      - an integer preceded by & (i.e. &10 for cross correlation)
 
     * interval selection
       - mixmin
       - propertyproperty
 
     * set selection
       - property=value
       - property=
 
     * top/bottom seletion 
       - property[top]number
       - property[bottom]number
 
     * selection modus
       This indicates how the restriction of particles will be reflected
       in the resulting table. 
       Main options are 'crop' and 'mark'
       - 'crop','c' : only the selected tags will be kept in the generated 
                      table.
       - 'mark','m' : the generated table will have the same dimensionality 
                      as the input table, but only the selected tags will
                      be "marked": they will get the value "1" in the column 2
                      while all the other tags will be marked with the
                      value 0
        By default, the selection modus is of "crop" type. 
      
    
      EXAMPLES:
 
      % compare the dimensions of the two output files:
      dtgrep example.tbl -tr cc<0.6:m -o table_marked.tbl
      dtgrep example.tbl -tr cc<0.6:c -o table_cropped.tbl
 
     
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
               Restrictions in functional form
               -------------------------------
 
    Some other parameters reproduce and extend the functionality of 
    the "table_restrict" syntax. These parameters might be more
    flexible in scripted applications:
 
    - 'grep'   type
    - 'best'   type
    - 'bottom' type 
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
    GREP  
    
     grep operations and best operations are mutually excluding
  
     'grep_column'       indicates which column will provide the grep
                         selection criterion 
 
     'grep_selection'    selects the tags presenting this particular set of 
                         values in the column indicated with 'grep_colume'
 
     'grep_interval'     selects all tags in an interval
 
 
     EXAMPLE
 
     dtgrep example.tbl -grep_column 35 -grep_selection [1,5] -ws tableout;
   
     Examines the column 35 in the table (incidentally, the subreference
     label in the standard Dynamo convention) and extracts those particles
     whose value is 1 or 5;
 
 
    grep by category
    ---------------
 
     Some additional parameters are included as shortcut to grep a
     selection of values corresponding to columns particularly used:
 
    'grep_align'         greps according to inclusion of particles in the 
                         alignment (as registered in column 2)
 
    'grep_average'       greps according to inclusion of particles in the 
                         average (as registered in column 3)
 
    'grep_volume'/'grep_tomo'
                         greps according to  column 20
 
    'grep_class'         greps according to  column 22
 
    'grep_ref'           greps according to  column 34
 
    'grep_sref'          greps according to  column 35
 
    'grep_annotation'    greps according to  column 23
 
 
     EXAMPLE:
 
 
     dtgrep example.tbl -grep_sref 5 -ws table_out;
 
     % this order is a shortcut to quickly extract those particles
     % belonging to subreference 5, and puts the result into 
     % workspace variable 'table_out'
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
     exclude  
     -------
 
     'exclude'           Explicitely exclude a set of tags from the table
                         - array of numeric values
                         - filename
 
     EXAMPLES
 
     dtgrep example.tbl -exclude my_tags -ws table_out
 
     % excludes tags 1, 5 and 7 from numeric table called "big_table"
     % and puts the result in my_table
     my_table = dtgrep(big_table,'exclude',[1,5,7]);
  
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
   
      best (thresholding)
      -------------------
 
     'best'-type operations are carried after 'grep'-type and 'exclude' operations 
 
     'best'              Selects a number of tags
                         By default, the criterion is column 10 in the
                         passed table.
 
     'best_fraction'     Selects a fraction (between 0 and 1) of tag set.
 
     'best_threshold'    Select particles above this threshold.
                         
 
     'best_column'       Explicites a column different to 10 from the table
 
     'best_scores'       (rarely used!)
                         Explicites a set of scores that attaches a
                         quality criterion to each tag.
                         The use of this option overrides 'best_column'.
                         The value can be passed as:
                         - numeric array
                         - a file  (.txt, .em)
 
   EXAMPLE
 
   dtgrep  example.tbl  -best 10  -ws a;
   % the best 10 particles (attending to the CC criteria)
 
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
      Bottom
      ------
 
      'bottom'-type operators are applied after 'grep','exclude' and 'best'
      They mirror the functionality of 'best' operators:
 
      'bottom'           : Select a number of tags.
 
      'bottom_threshold' : Select particles below a threshold;
 
 
   EXAMPLE
   dtgrep  example.tbl  -bottom_threshold 0.25  -ws b;
   % particles scoring below 0.25
 
 
  
 
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 
 
 
   Output flags
   ------------ 
 
      'selection','s'    : states how the table selection manifests
                           in the output table.
                           This option corresponds to the actions of type
                           "selection modus" in "table_restrict"
                           style.
                           Thus, your options here are also 'crop' and 'mark'
                           - 'crop'   : only selected tags will appear.
                                        The output table is thus possibly smaller.
                           - 'mark'   : - selected tags will be marked for 
                                          alignment (1 in column 2).
                                        - non selected tags will be marked for 
                                          out of alignment (0 in column 2).
                                       The output table is thus of the same
                                       size as the original one.
                            - default: 'crop'
 
      EXAMPLE
 
       dtgrep example.tbl -best 10 -s mark -o thresholded_table.tbl
     
       % * selects the 10 best cc scores in table.tbl
       % * marks them for alignment
       % * marks the other tags not to participate in the alignment 
 
      'extract_column','ec' :
                           Returns the columns instead of the whole table.
                           You can indicate the wished columne as number or
                           property string)
                         
       EXAMPLE
 
       dtgrep example.tbl -tr cc>0.5 -ec cc -ws best_ccs;
 
       % Selects the cross-correlation coefficients bigger than 0.5 and 
       % puts them (as a vector) in the workspace variable best_ccs;
 
 
       dtgrep example.tbl -tr cc>0.5 -ec x|y|z -o positions.txt;
 
       % extracts the x,y,z positions marker in the table of the particles
       % with correlation above 0.5 and writes them into a file;
 
      'extract_row_number','er' :
                           Returns the row numbers of the selected
                           tags.
                           This options is mainly intended for scripting.
 
 
      'o'                : directs the output into a file.
 
      'otags'            : direct the surviving tags into a text file
                           (typically with extension .tags)
 
      'ws'               : stores the output table in a base workspace variable.
 
   
      EXAMPLE
 
       dtgrep example.tbl -best 10 -s mark -o thresholded_table.tbl
     
       % * selects the 10 best cc scores in table.tbl
       % * marks them for alignment
       % * marks the other tags not to participate in the alignment 
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    
  
   OUTPUT to command line
 
    table_out:          The table(s), restricted to the indicated selection
   
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
   EXAMPLES OF SYNTAX
 
   table = dynamo_table_random(100);
   table_reduced = dynamo_table_grep(table,'tags',1:30);
   table_reduced = dynamo_table_grep(table,'tags',1:30,'best_fraction',0.5);