DBox folder

The dBox folder is an alternate way to store particle files. When you have, say, 100K files, it is a bad idea to put all of them on the same folder.

Instead of leaving all particles in the same data folder, inside the dBox folder there is a subdirectory hierarchy that allows saving the particles in different subfolders. This happens in a way totally transparent for the user.

Dynamo uses a class called dBoxes to manage this kind of generic data container. Check the different command line options with:

help dBoxes

Convert a normal data folder into a dBox folder

d = dBoxes.convertSimpleData(<foldername>,<dBoxes folder name>);

Merge several data folders into a single dBox folder

If you cannot merge all the data folders independently into a single data folder, you'll have to enter all the particles one by one using dBoxes.enterParticle;

%
% create several tutorial folders
%

for i=1:10;
    testFolder =['testfolder',num2str(i)];
     dtutorial(testFolder,'M',100);
     dataFolder{i} = [testFolder,'/data'];
     tbl{i}      = [testFolder,'/real.tbl'];
end 

% create a dBoxes folder d = dBoxes('new', 'data.Boxes'); d.padding = 7; d.batch = 200; % each subfolder will have only 200 particles % updates the representation of the object in disk d.updateSettingsField('padding',7); d.updateSettingsField('batch',200); % creates a table that will talk to all elements in the final data folder globalTable = zeros(0,size(tbl{end},2)); disp('Merging folders.'); timeStart = clock(); for i=1:length(dataFolder) f = ddinfo(dataFolder{i},'v',0); tableForFolder = dread(tbl{i}); % ensures the individual tables are sorted in ascending order of the % tags tableForFolder = dynamo_table_sort(tableForFolder); for itag = 1:f.N % file of particle in original folder file = dynamo_tag2file(f.tags(itag),f,0); %disp(file); % the file is transferred as copy, without passing the % particle map into memory d.enterParticle(file,'directcopy',1); end % rearrange the tags in the table corresponding to a single folder tableForFolder(:,1) = [(size(globalTable,1)+1):(size(globalTable,1)+f.N)]'; globalTable = [globalTable;tableForFolder]; disp(sprintf('finished merging folder %d: %s',i,dataFolder{i})); end timeFinish = etime(clock(),timeStart); disp('done merging. Averaging step.'); %% o = daverage('data.Boxes','t',globalTable); disp(sprintf('Seconds for creation of dBoxes %f',timeFinish)); disp('Show average'); dview(o);