babs merge: Merge results and provenance

Command-Line Arguments

Merge results and provenance from all successfully finished jobs.

usage: babs merge [-h] [--chunk-size CHUNK_SIZE] [--trial-run] [PATH]

User arguments

PATH

Absolute path to the root of BABS project. For example, '/path/to/my_BABS_project/' (default is current working directory).

Default: /home/docs/checkouts/readthedocs.org/user_builds/pennlinc-babs/checkouts/0.5.0/docs

Developer arguments

Parameters for developers. Users should not use these.

--chunk-size, --chunk_size

Number of branches in a chunk when merging at a time. We recommend using default value.

Default: 2000

--trial-run, --trial_run

Whether to run as a trial run which won't push the merge back to output RIA. This option should only be used by developers for testing purpose. Users: please don't turn this on!

Default: False

Example commands

babs merge /path/to/my_BABS_project

It's usually not necessary to add the other two arguments (--chunk-size, --trial-run), as those two arguments are mainly for developers to use.

Detailed description

babs merge will generate a folder called merge_ds to perform merging. This folder is also a DataLad dataset and a sibling of output RIA.

What if babs merge fails?

The folder merge_ds generated by babs merge won't be automatically removed by babs merge, as users may want to check files regarding warnings saved in this folder. If you have fixed the issue based on the error message, and hope to rerun babs merge, you need to remove folder merge_ds first:

# 1. go to `merge_ds` folder:
cd /path/to/my_BABS_project/merge_ds

# 2. save any new files:
datalad save -m "save any untracked files generated by babs merge"

# 3. ask DataLad not to track here:
git annex dead here

# 4. remove `merge_ds` DataLad dataset:
cd ..
datalad remove -d merge_ds --reckless availability

What if babs merge takes a long time?

Usually babs merge won't take a long time. For example, merging FreeSurfer results (got from fMRIPrep) for about n=2300 successful jobs took around 10-15 minutes. However if you are concerned that babs merge takes a long time, you can:

  • Either use screen so that even you are disconnected from the cluster login node, babs merge command keeps running;

  • Or submit babs merge as a job.