clean up looper docs

Author: nsheff

docs/looper/advanced-guide/advanced-run-options.md

@@ -18,19 +18,19 @@ Let's introduce some of the more advanced capabilities of `looper run`.
 
 ## Grouping many jobs into one
 
-By default, `looper` will translate each row in your `sample_table` into a single job. But perhaps you are running a project with tens of thousands of rows, and each job only takes mere minutes to run; in this case, you'd rather just submit a single job to process many samples. `Looper` makes this easy with the `--lump` and `--lumpn` command line arguments.
+By default, `looper` will translate each row in your `sample_table` into a single job. But perhaps you are running a project with tens of thousands of rows, and each job only takes mere minutes to run; in this case, you'd rather just submit a single job to process many samples. `Looper` makes this easy with the `--lump` and `--lump-n` command line arguments.
 
-### Lumping jobs by job count: `--lumpn`
+### Lumping jobs by job count: `--lump-n`
 
-It's quite simple: if you want to run 100 samples in a single job submission script, just tell looper `--lumpn 100`.
+It's quite simple: if you want to run 100 samples in a single job submission script, just tell looper `--lump-n 100`.
 
 ### Lumping jobs by input file size: `--lump`
 
-But what if your samples are quite different in terms of input file size? For example, your project may include many small samples, which you'd like to lump together with 10 jobs to 1, but you also have a few control samples that are very large and should have their own dedicated job. If you just use `--lumpn` with 10 samples per job, you could end up lumping your control samples together, which would be terrible. To alleviate this problem, `looper` provides the `--lump` argument, which uses input file size to group samples together. By default, you specify an argument in number of gigabytes. Looper will go through your samples and accumulate them until the total input file size reaches your limit, at which point it finalizes and submits the job. This will keep larger files in independent runs and smaller files grouped together.
+But what if your samples are quite different in terms of input file size? For example, your project may include many small samples, which you'd like to lump together with 10 jobs to 1, but you also have a few control samples that are very large and should have their own dedicated job. If you just use `--lump-n` with 10 samples per job, you could end up lumping your control samples together, which would be terrible. To alleviate this problem, `looper` provides the `--lump` argument, which uses input file size to group samples together. By default, you specify an argument in number of gigabytes. Looper will go through your samples and accumulate them until the total input file size reaches your limit, at which point it finalizes and submits the job. This will keep larger files in independent runs and smaller files grouped together.
 
-### Lumping jobs by input file size: `--lumpj`
+### Lumping jobs by job count: `--lump-j`
 
-Or you can lump samples into number of jobs.
+If you want to split your samples across a specific number of jobs, use `--lump-j`. For example, `--lump-j 10` will distribute all your samples evenly across 10 jobs.
 
 
 ## Running project-level pipelines
@@ -251,22 +251,28 @@ For example, to choose only samples where the `species` attribute is `human`, `m
 
 ```console
 looper run \
-  --sel-attr species
+  --sel-attr species \
   --sel-incl human mouse fly
 ```
 
 Similarly, to submit only one sample, with `sample_name` as `sample`, you could use:
 
 ```console
 looper run \
-  --sel-attr sample_name
+  --sel-attr sample_name \
   --sel-incl sample1
 ```
 
 ### Sample selection by exclusion
 
-If more convenient to *exclude* samples by filter, you can use the analogous arguments `--sel-attr` with `--sel-excl`.
-This will 
+If it's more convenient to *exclude* samples by filter, you can use the analogous arguments `--sel-attr` with `--sel-excl`.
+This will exclude any samples matching the specified values. For example, to run all samples *except* those where `species` is `rat`:
+
+```console
+looper run \
+  --sel-attr species \
+  --sel-excl rat
+```
 
 ### Toggling sample jobs through the sample table
 

docs/looper/user-tutorial/initialize.md

@@ -87,36 +87,56 @@ Chiapas
 
 ## Step 3: Create a metadata table
 
-Looper needs a list of samples in the form of a sample metadata table, which names each sample and includes paths to the data files. Looper will accept a PEP, which we'll discuss more later, or just a simple CSV file, which is where we'll start. Create a new file called `metadata/sample_table.csv` and paste this content in it:
+Looper needs a list of samples in the form of a sample metadata table, which names each sample and includes paths to the data files. Looper will accept a PEP, which we'll discuss more later, or just a simple CSV file, which is where we'll start. Create a new file called `metadata/sample_table.csv` with this content:
 
-```csv  title="metadata/sample_table.csv"
-sample_name,area_type,file_path
-mexico,state,data/mexico.txt
-switzerland,canton,data/switzerland.txt
-canada,province,data/canada.txt
-```
+=== "View"
+
+    ```csv title="metadata/sample_table.csv"
+    sample_name,area_type,file_path
+    mexico,state,data/mexico.txt
+    switzerland,canton,data/switzerland.txt
+    canada,province,data/canada.txt
+    ```
+
+=== "Create"
+
+    ```sh
+    mkdir -p metadata && cat > metadata/sample_table.csv << 'EOF'
+    sample_name,area_type,file_path
+    mexico,state,data/mexico.txt
+    switzerland,canton,data/switzerland.txt
+    canada,province,data/canada.txt
+    EOF
+    ```
 
 Each row corresponds to a sample, with a unique identifier under `sample_name`, a pointer to its corresponding file in `file_path`, and any other information you want to include about the sample (in this case, `area_type`). These will be the different values available to pass to your pipeline.
 
 ## Step 4: Create the pipeline
 
-Our example pipeline is a shell script that counts the lines in an input file. Since our data has one line per province, this script will tell us how many provinces there are in each country. Create a file under `pipeline/count_lines.sh` with this content:
+Our example pipeline is a shell script that counts the lines in an input file. Since our data has one line per province, this script will tell us how many provinces there are in each country. Create a file at `pipeline/count_lines.sh` with this content:
 
-```sh title="pipeline/count_lines.sh"
-#!/bin/bash
-linecount=`wc -l $1 | sed -E 's/^[[:space:]]+//' | cut -f1 -d' '`
-echo "Number of lines: $linecount"
-```
+=== "View"
+
+    ```sh title="pipeline/count_lines.sh"
+    #!/bin/bash
+    linecount=`wc -l $1 | sed -E 's/^[[:space:]]+//' | cut -f1 -d' '`
+    echo "Number of lines: $linecount"
+    ```
 
+=== "Create"
+
+    ```sh
+    mkdir -p pipeline && cat > pipeline/count_lines.sh << 'EOF'
+    #!/bin/bash
+    linecount=`wc -l $1 | sed -E 's/^[[:space:]]+//' | cut -f1 -d' '`
+    echo "Number of lines: $linecount"
+    EOF
+    chmod 755 pipeline/count_lines.sh
+    ```
 
 All this script does is run the unix `wc` command, and then parse the output using `sed` and `cut`, and then print the result with `echo`.
 In a real workspace, your pipeline is more likely to be a powerful Python script or something else.
-The important thing for looper is just that there's a command you can run to execute the pipeline, and you can pass arguments on the command line. 
-Since looper will need to execute it, make sure your script has execute permission:
-
-```sh
-chmod 755 pipeline/count_lines.sh
-```
+The important thing for looper is just that there's a command you can run to execute the pipeline, and you can pass arguments on the command line.
 
 
 ## Step 5: Create a pipeline interface
@@ -125,14 +145,27 @@ In order to run a pipeline, looper needs to know how to construct the command, w
 A pipeline developer does this through the *pipeline interface*.
 Our `count_lines.sh` pipeline just takes a single argument, which is the file path, so the command would be `count_lines.sh path/to/file.txt`.
 Here's how to create the appropriate pipeline interface for this pipeline.
-Create a file in `pipeline/pipeline_interface.yaml` and paste this content into it:
+Create a file at `pipeline/pipeline_interface.yaml` with this content:
 
-```yaml  title="pipeline/pipeline_interface.yaml"
-pipeline_name: count_lines
-sample_interface:
-  command_template: >
-    pipeline/count_lines.sh {sample.file_path}
-```
+=== "View"
+
+    ```yaml title="pipeline/pipeline_interface.yaml"
+    pipeline_name: count_lines
+    sample_interface:
+      command_template: >
+        pipeline/count_lines.sh {sample.file_path}
+    ```
+
+=== "Create"
+
+    ```sh
+    cat > pipeline/pipeline_interface.yaml << 'EOF'
+    pipeline_name: count_lines
+    sample_interface:
+      command_template: >
+        pipeline/count_lines.sh {sample.file_path}
+    EOF
+    ```
 
 This is the pipeline interface file.
 It specifies the `pipeline_name`, which looper uses to keep track of pipelines in case it is running multiple of them.
@@ -144,8 +177,12 @@ For now, the important part is just that the `command_template` is telling loope
 
 Now you have all the components finished for your looper project.
 The last step is to create the looper configuration file.
-The easiest way to do this is to run `looper init` from within your workspace folder.
-This will walk you through a wizard that will build the file for you.
+The easiest way to do this is to run the *looper init* wizard from within your workspace folder.
+This wizard will walk you through building the file.
+
+```
+looper init  # Run looper initialization wizard
+```
 
 Here's how to answer the questions
 
@@ -159,12 +196,25 @@ Here's how to answer the questions
 
 This will create a file at `.looper.yaml`, with this content:
 
-```yaml  title=".looper.yaml"
-pep_config: metadata/sample_table.csv
-output_dir: results
-pipeline_interfaces:
-  - pipeline/pipeline_interface.yaml
-```
+=== "View"
+
+    ```yaml title=".looper.yaml"
+    pep_config: metadata/sample_table.csv
+    output_dir: results
+    pipeline_interfaces:
+      - pipeline/pipeline_interface.yaml
+    ```
+
+=== "Create"
+
+    ```sh
+    cat > .looper.yaml << 'EOF'
+    pep_config: metadata/sample_table.csv
+    output_dir: results
+    pipeline_interfaces:
+      - pipeline/pipeline_interface.yaml
+    EOF
+    ```
 
 Right now, this file is basically just 3 pointers.
 The first line points to the metadata table we just created.
@@ -196,7 +246,7 @@ For example, here are some looper arguments:
 In the upcoming tutorials, we'll cover these and other features in more detail.
 
 !!! tip "Summary"
-    - To use looper, you'll need to have data, metadata, a pipeline (with pipeline interface), and a looper configuration file. 
-    - Looper is best suited for data that is split into samples, where you're trying to run a pipeline independently on each sample.
-    - Once you've configured everything correctly, you run your samples with the command `looper run`.
+    - To use looper, you'll create a looper configuration file which points to: 1. data, 2. metadata, and 3. a pipeline with pipeline interface.
+    - Looper is best suited for data that is split into samples or jobs, where you're trying to run a pipeline independently on each sample.
+    - Once you've configured looper, you run jobs with the command `looper run`.
 

mkdocs.yml

@@ -29,10 +29,11 @@ theme:
     - navigation.sections
     # - navigation.expand
     - navigation.footer
-    - navigation.indexes 
+    - navigation.indexes
     - toc.follow
     - content.action.edit
     - content.action.view
+    - content.code.copy
     - navigation.tabs
     - navigation.top
 
@@ -42,6 +43,8 @@ markdown_extensions:
   - admonition
   - pymdownx.details
   - pymdownx.superfences
+  - pymdownx.tabbed:
+      alternate_style: true
   - pymdownx.highlight:
       use_pygments: true
 #  - pymdownx.emoji: