Add notebooks cell standards check (#1152)

Fixes #1143 .

### Description
- Add checks for "Setup environment", "Setup imports", "print_config()"
- Limit number of lines (100) in the output text
- Fix redundant printing during notebook nightly test
- Fix Auto3DSeg readme indentation.
- Refractor the python code as suggested in a [previous
comment](#1145 (comment))

### Checks
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Avoid including large-size files in the PR.
- [x] Clean up long text outputs from code cells in the notebook.
- [x] For security purposes, please check the contents and remove any
sensitive info such as user names and private key.
- [x] Ensure (1) hyperlinks and markdown anchors are working (2) use
relative paths for tutorial repo files (3) put figure and graphs in the
`./figure` folder
- [x] Notebook runs automatically `./runner.sh -t <path to .ipynb file>`

Signed-off-by: Mingxin Zheng <18563433+mingxin-zheng@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Wenqi Li <831580+wyli@users.noreply.github.com>

Author: 3 people

.github/contributing_templates/notebook/example_feature.ipynb

@@ -9,12 +9,12 @@
     "Licensed under the Apache License, Version 2.0 (the \"License\");  \n",
     "you may not use this file except in compliance with the License.  \n",
     "You may obtain a copy of the License at  \n",
-    "    http://www.apache.org/licenses/LICENSE-2.0  \n",
+    "    http://www.apache.org/licenses/LICENSE-2.0  \n",
     "Unless required by applicable law or agreed to in writing, software  \n",
     "distributed under the License is distributed on an \"AS IS\" BASIS,  \n",
     "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  \n",
     "See the License for the specific language governing permissions and  \n",
-    "limitations under the License.  \n",
+    "limitations under the License.\n",
     "\n",
     "# Your Tutorial Title"
    ]
@@ -214,7 +214,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "base",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -228,12 +228,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.13"
+   "version": "3.8.10 (default, Nov 14 2022, 12:59:47) \n[GCC 9.4.0]"
   },
   "orig_nbformat": 4,
   "vscode": {
    "interpreter": {
-    "hash": "d4d1e4263499bec80672ea0156c357c1ee493ec2b1c70f0acce89fc37c4a6abe"
+    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
    }
   }
  },

.github/workflows/guidelines.yml

@@ -0,0 +1,40 @@
+name: build
+
+on:
+  # quick tests for every pull request
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  # caching of these jobs:
+  #   - docker-20-03-py3-pip- (shared)
+  #   - ubuntu py37 pip-
+  #   - os-latest-pip- (shared)
+  guidelines:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python 3.8
+      uses: actions/setup-python@v3
+      with:
+        python-version: 3.8
+    - name: cache weekly timestamp
+      id: pip-cache
+      run: |
+        echo "::set-output name=datew::$(date '+%Y-%V')"
+    - name: cache for pip
+      uses: actions/cache@v3
+      id: cache
+      with:
+        path: ~/.cache/pip
+        key: ${{ runner.os }}-pip-${{ steps.pip-cache.outputs.datew }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip wheel
+        python -m pip install -r https://raw.githubusercontent.com/Project-MONAI/MONAI/dev/requirements-dev.txt
+        python -m pip install -r requirements.txt
+    - name: Guidelines notebook format check
+      run: |
+        $(pwd)/runner.sh --no-run --no-checks --cell-standard

CONTRIBUTING.md

@@ -59,23 +59,39 @@ Finally, the MONAI tutorial has a [README file](README.md) to communicate the im
 
 All source code and notebook files should include copyright information at the top of the file.
 
-NOTE: for Jupyter Notebook `.ipynb` files, the copyright information should appear at the top of the first markdown cell.
-There are extra two spaces at the end of each line to ensure no line auto-wrap in the markdown rendering in the display.
-
-```markdown
-Copyright (c) MONAI Consortium
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-    http://www.apache.org/licenses/LICENSE-2.0
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-
+NOTE: For script files (python, shell commands, and etc), please use the following header
+
+```python
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 ```
 
+NOTE: For Jupyter notebook `.ipynb` files, the copyright information should appear at the top of the first markdown cell.
+There are *two extra spaces* at the end of each line to ensure no line auto-wrap in the markdown rendering in the display, e.g. the first line of the copyright should be `"Copyright (c) MONAI Consortium  \n"` instead of `"Copyright (c) MONAI Consortium\n"` in the raw string format.
+Indentation in markdown is represented by consecutive 4 non-breaking space entities ` `  in the line.
+
+Please copy the following header with formats and paste it in the first markdown cell of a Jupyter notebook:
+<pre>
+Copyright (c) MONAI Consortium  <space>
+Licensed under the Apache License, Version 2.0 (the "License");  <space>
+you may not use this file except in compliance with the License.  <space>
+You may obtain a copy of the License at  <space>
+&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;http://www.apache.org/licenses/LICENSE-2.0  <space>
+Unless required by applicable law or agreed to in writing, software  <space>
+distributed under the License is distributed on an "AS IS" BASIS,  <space>
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  <space>
+See the License for the specific language governing permissions and  <space>
+limitations under the License.
+</pre>
+
 ### Create a notebook
 
 Jupyter Notebook is the preferred way of writing a new MONAI tutorial because we encourage contributors to visualize the outputs and keep the records with the code.

auto3dseg/README.md

@@ -44,7 +44,7 @@ We have demonstrated preliminary results of public datasets are described in the
 - [INSTANCE22](tasks/instance22/README.md)
 - [Beyond the Cranial Vault (BTCV) Abdomen Dataset](tasks/btcv/README.md)
 - Medical Segmentation Decathlon (MSD) Dataset
-        - [Task04 Task04_Hippocampus](tasks/msd/Task04_Hippocampus/README.md)
+	- [Task04 Task04_Hippocampus](tasks/msd/Task04_Hippocampus/README.md)
 	- [Task05 Prostate](tasks/msd/Task05_Prostate/README.md)
 	- [Task09 Spleen](tasks/msd/Task09_Spleen/README.md)
 

ci/nbtest.py

@@ -0,0 +1,94 @@
+# Copyright (c) MONAI Consortium
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#     http://www.apache.org/licenses/LICENSE-2.0
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import nbformat
+import re
+
+def define_parser(parser):
+    """Define the parser for commands"""
+    parser.add_argument("-f", "--filename", type=str, help="a .ipynb jupyter notebook", required=True)
+    group = parser.add_mutually_exclusive_group()
+    group.add_argument("-s", "--start", type=int, default=0, help="the starting index of target cells")
+    group.add_argument("-i", "--index", type=int, help="the index of a target cell")
+    parser.add_argument("-e", "--end", type=int, default=-1, help="the ending index of target cells")
+    parser.add_argument("-k", "--keyword", type=str, default="", help="regex keyword to search in the target cell")
+    parser.add_argument(
+        "--type",
+        type=str,
+        default="markdown",
+        help="the type of target cell (code/markdown). Default is markdown"
+    )
+    parser.add_argument(
+        "--field",
+        type=str,
+        default="source",
+        help="the type of target cell (code/markdown). Default is markdown"
+    )
+
+
+def count_matches(filename, start, end, keyword, cell_type="markdown", field="source"):
+    """Count the number of keyword matches from start index to end index"""
+    occurrences = []
+
+    with open(filename, "r") as f:
+        notebook = nbformat.reads(f.read(), as_version=4)
+        cells = notebook.cells[start:end]
+        if not cells:
+            raise ValueError(f"No cells extracted from index {start} to index {end}")
+        for cell in cells:
+            if cell.cell_type != cell_type:
+                occurrences.append(0)
+            else:
+                #  treat backslashes as literal characters
+                keyword_rawstring = repr(keyword)[1:-1]  # remove the single quotes before/after the word
+                occurrences.append(len(re.findall(keyword_rawstring, str(cell[field]))))  # value can be list of dict
+
+    return occurrences
+
+def print_verification_bool(*args, **kwargs):
+    """Print true/false for whether target cells match the keyword and the cell_type to interface with bash"""
+
+    result = "true" if sum(count_matches(*args, **kwargs)) else "false"
+    print(result)
+
+def print_count_array(*args, **kwargs):
+    """Print number array of the matches to interface with bash"""
+
+    print(" ".join(map(str, count_matches(*args, **kwargs))))
+
+def main():
+    parser = argparse.ArgumentParser()
+    # Create a "subparsers" object to hold the subcommands
+    subparsers = parser.add_subparsers(title="subcommands", dest="subcommand")
+
+    # Create a parser for the "verify" subcommand
+    parser_verify = subparsers.add_parser("verify")
+    define_parser(parser_verify)
+    # Create a parser for the "count" subcommand
+    parser_count = subparsers.add_parser("count")
+    define_parser(parser_count)
+
+    args = parser.parse_args()
+
+    if 'index' in args and args.index:
+        args.start = args.index
+        args.end = args.index + 1
+
+    if args.subcommand == "verify":
+        print_verification_bool(args.filename, args.start, args.end, args.keyword, args.type, args.field)
+    elif args.subcommand == "count":
+        print_count_array(args.filename, args.start, args.end, args.keyword, args.type, args.field)
+    else:
+        print("No subcommand provided. Please use -h for help")
+
+if __name__ == "__main__":
+    main()