Update outdated documentation sections

AyanSinhaMahapatra · AyanSinhaMahapatra · commit 2a1f596c0b60 · 2025-12-09T13:53:55.000+05:30
Also update documentation formatting and pages layout/sections.

Signed-off-by: Ayan Sinha Mahapatra &lt;asmahapatra@aboutcode.org&gt;
diff --git a/docs/source/reference/index.rst b/docs/source/reference/index.rst
@@ -11,5 +11,3 @@ Technical reference material.
    scancode-cli/index.rst
    scancode-plugins/index.rst
    scancode-supported-packages
-   extractcode-cli
-   scancode-reindex-licenses-cli
diff --git a/docs/source/reference/scancode-cli/cli-core-options.rst b/docs/source/reference/scancode-cli/cli-core-options.rst
@@ -18,6 +18,8 @@ Comparing progress message options
 
     **Default progress message**
 
+    By default a rolling progress bar and scanned file count is shown.
+
     .. code-block:: none
 
         Scanning files for: infos, licenses, copyrights, packages, emails, urls with 1 process(es)...
diff --git a/docs/source/reference/scancode-cli/cli-extractcode.rst b/docs/source/reference/scancode-cli/cli-extractcode.rst
@@ -11,6 +11,7 @@ Archives within an extracted archive are extracted **recursively** by default,
 with the extraction occurring in a directory named "-extract" alongside the original archive.
 
 **Quick Reference**
+-------------------
 
 --shallow   Do not extract recursively nested archives (e.g. Not
             archives in archives).
diff --git a/docs/source/reference/scancode-cli/cli-scancode-license-data.rst b/docs/source/reference/scancode-cli/cli-scancode-license-data.rst
@@ -0,0 +1,34 @@
+.. _cli-scancode-license-data:
+
+scancode license data CLI
+=========================
+
+Scancode includes the license data for all the licenses in detects
+as scancode-licensedb with the license texts and other reference data
+like license category, authors, and reference links.
+
+``scancode-license-data`` is a CLI command to provide this license data
+in a structured format that is API, and this includes:
+
+- all the license details, without license text, in one JSON/YAML file
+- all the individual licenses, with license text, in JSON/YAML files
+- locally built HTML page with a searchable table of all licenses
+- locally built HTML pages for all individual licenses with its text and details
+
+The data and HTML pages are dumped into a folder specified by the required
+CLI option ``--path``.
+
+The licensedb is also hosted at https://scancode-licensedb.aboutcode.org/ and is
+updated daily from the latest added licenses from the scancode ``develop`` branch
+using the ``scancode-license-data`` CLI option.
+
+Usage: ``scancode-license-data [OPTIONS]``
+
+**Quick Reference**
+-------------------
+
+--path DIR      Dump the license data in this directory in the LicenseDB format
+                and exit. Creates the directory if it does not exist.
+                [required]
+-h, --help      Show this message and exit.
+
diff --git a/docs/source/reference/scancode-cli/cli-scancode-reindex-licenses.rst b/docs/source/reference/scancode-cli/cli-scancode-reindex-licenses.rst
@@ -10,7 +10,7 @@ The ``scancode-reindex-licenses`` command rebuilds the license index.
 Usage: ``scancode-reindex-licenses [OPTIONS]``
 
 Quick Reference
-^^^^^^^^^^^^^^^
+---------------
 
   --all-languages             [EXPERIMENTAL] Rebuild the license index
                               including texts all languages (and not only
diff --git a/docs/source/reference/scancode-cli/index.rst b/docs/source/reference/scancode-cli/index.rst
@@ -3,72 +3,88 @@
 **ScanCode CLI**
 ================
 
-.. toctree::
-   :hidden:
-
-   cli-help-text-options
-   cli-basic-options
-   cli-core-options
-   cli-output-format-options
-   cli-output-control-and-filter-options
-   cli-pre-scan-options
-   cli-post-scan-options
-
 Below is a quick reference of all ScanCode options.
-For detailed descriptions and examples, please refer to the chapters in the table of contents.
+For detailed descriptions and examples, please refer to the chapters in the table of
+contents or the link to the detailed CLI options at the end of each section.
 
-The order of the sections and all their options is the same as in the :ref:`cli-help-option`,
-available in the command line.
+This has the same CLI options in the :ref:`cli-help-option`, available in the command line
+but with more details and additional references.
 
 ----
 
 .. _cli-scancode-basic-options:
 
-**Basic options**
-
 .. include::  /rst-snippets/cli-basic-options.rst
-   :start-line: 3
+
+For more details into the basic CLI options, see :ref:`cli-basic-options`.
 
 ----
 
 .. _cli-scancode-core-options:
 
-**Core options**
-
 .. include::  /rst-snippets/cli-core-options.rst
-   :start-line: 3
+
+For more details into the core CLI options, see :ref:`cli-core-options`.
+
 ----
 
 .. _cli-scancode-output-format-options:
 
-**Output formats**
-
 .. include::  /rst-snippets/cli-output-format-options.rst
-   :start-line: 3
+
+For more details into the output format CLI options, see :ref:`cli-output-format-options`.
 
 ----
 
 .. _cli-scancode-output-control-and-filter-options:
 
-**Controlling output and filters**
-
 .. include::  /rst-snippets/cli-output-control-and-filter-options.rst
-   :start-line: 3
+
+For more details into the output filter CLI options, see
+:ref:`cli-output-control-and-filter-options`.
 
 ----
 
 .. _cli-scancode-pre-scan-options:
 
-**Pre-scan options**
-
 .. include::  /rst-snippets/cli-pre-scan-options.rst
-   :start-line: 3
+
+For more details into the pre-scan CLI options, see :ref:`cli-pre-scan-options`.
 
 ----
 
 .. _cli-scancode-post-scan-options:
 
-**Post-scan options**
-
 .. include::  /rst-snippets/cli-post-scan-options.rst
-   :start-line: 3
+
+For more details into the post-scan CLI options, see :ref:`cli-post-scan-options`.
+
+----
+
+.. _cli-other-commands:
+
+.. include::  /rst-snippets/cli-other-commands.rst
+
+----
+
+.. _cli-doc-help-options:
+
+.. include::  /rst-snippets/cli-doc-help-options.rst
+
+----
+
+**Detailed reference for CLI options/commands**
+-----------------------------------------------
+
+.. toctree::
+
+   cli-help-text-options
+   cli-basic-options
+   cli-core-options
+   cli-output-format-options
+   cli-output-control-and-filter-options
+   cli-pre-scan-options
+   cli-post-scan-options
+   cli-extractcode
+   cli-scancode-reindex-licenses
+   cli-scancode-license-data
diff --git a/docs/source/rst-snippets/cli-basic-options.rst b/docs/source/rst-snippets/cli-basic-options.rst
@@ -1,5 +1,5 @@
-"basic" options
----------------
+**Basic options**
+-----------------
 
 Option lists are two-column lists of command-line options and descriptions,
 documenting a program's options. For example:
@@ -33,9 +33,11 @@ documenting a program's options. For example:
 --system-package             Scan ``<input>`` for installed system package
                              databases.
 
---package-only               Scan ``<input>`` for system and application
-                             only for package metadata, without license/
-                             copyright detection and package assembly.
+--package-only               Faster package scan, scanning ``<input>`` for
+                             system and application packages, only for package
+                             metadata. This option is skipping
+                             - license and copyright detection for package metadata
+                             - package assembly
 
 -e, --email                  Scan ``<input>`` for emails.
 
@@ -68,19 +70,26 @@ documenting a program's options. For example:
 --generated                  Classify automatically generated code files with a flag.
 
 --max-email INT              Report only up to INT emails found in a
-                             file. Use 0 for no limit.  [Default: 50]
+                             file. Use 0 for no limit.
+
+                             Default: ``50``
 
                              Sub-option of: ``--email``
 
 --max-url INT                Report only up to INT urls found in a
-                             file. Use 0 for no limit.  [Default: 50]
+                             file. Use 0 for no limit.
+
+                             Default: ``50``
 
                              Sub-option of: ``--url``
 
 --license-score INTEGER
 
           Do not return license matches with scores lower than this score.
-          A number between 0 and 100.  [Default: 0]
+          A number between 0 and 100.
+
+          Default: ``0`` (i.e. we return all license matches by default).
+
           Here, a bigger number means a better match, i.e. Setting a higher license score
           translates to a higher threshold (with equal or smaller number of matches).
 
@@ -101,7 +110,8 @@ documenting a program's options. For example:
           Set the template URL used for the license reference URLs.
 
           In a template URL, curly braces ({}) are replaced by the license key.
-          [Default: default: https://scancode-licensedb.aboutcode.org/{}]
+
+          Default: ``https://scancode-licensedb.aboutcode.org/{}``
 
           Sub-option of: ``--license``
 
diff --git a/docs/source/rst-snippets/cli-core-options.rst b/docs/source/rst-snippets/cli-core-options.rst
@@ -1,27 +1,37 @@
-"core" options
---------------
+**Core options**
+----------------
 
 -n, --processes INTEGER  Scan ``<input>`` using n parallel processes.
-                         [Default: (number of CPUs)-1]
+
+                         Default: ``(number of CPUs)-1``
 
 -v, --verbose            Print verbose file-by-file progress messages.
 
 -q, --quiet              Do not print summary or progress messages.
 
 --timeout FLOAT          Stop scanning a file if scanning takes longer
-                         than a timeout in seconds.  [Default: 120]
+                         than a timeout in seconds.
+
+                         Default: ``120``
+
+--from-json              Load codebase from one or more existing JSON scans to:
 
---from-json              Load codebase from one or more existing JSON scans.
+                         - apply post-scan options to do additional processing
+                           of scan results
+                         - merge multiple JSON scans into one.
 
 --max-in-memory INTEGER  Maximum number of files and directories scan
                          details kept in memory during a scan.
                          Additional files and directories scan details
                          above this number are cached on-disk rather
                          than in memory. Use 0 to use unlimited memory
                          and disable on-disk caching. Use -1 to use
-                         only on-disk caching.  [Default: 10000]
+                         only on-disk caching.
+
+                         Default: ``10000``
 
 --max-depth INTEGER      Descend at most INTEGER levels of directories
                          including and below the starting point. INTEGER
                          must be positive or zero for no limit.
-                         [Default: 0]
+
+                         Default: ``0``
diff --git a/docs/source/rst-snippets/cli-doc-help-options.rst b/docs/source/rst-snippets/cli-doc-help-options.rst
@@ -1,5 +1,5 @@
-Documentation/help options
---------------------------
+**Documentation/help options**
+------------------------------
 
 -h, --help               Show the :ref:`cli-help-option` and exit.
 
diff --git a/docs/source/rst-snippets/cli-other-commands.rst b/docs/source/rst-snippets/cli-other-commands.rst
@@ -0,0 +1,10 @@
+**Other CLI commands**
+----------------------
+
+-n, --processes INTEGER  Scan ``<input>`` using n parallel processes.
+
+                         Default: ``(number of CPUs)-1``
+
+-v, --verbose            Print verbose file-by-file progress messages.
+
+-q, --quiet              Do not print summary or progress messages.
diff --git a/docs/source/rst-snippets/cli-output-control-and-filter-options.rst b/docs/source/rst-snippets/cli-output-control-and-filter-options.rst
@@ -1,5 +1,5 @@
-"output control and filter" options
------------------------------------
+**Controlling output and filters**
+----------------------------------
 
 --strip-root           Strip the root directory segment of all paths.
 
diff --git a/docs/source/rst-snippets/cli-output-format-options.rst b/docs/source/rst-snippets/cli-output-format-options.rst
@@ -1,5 +1,5 @@
-"output" options
-----------------
+**Output formats**
+------------------
 
 --json FILE             Write scan output as compact JSON to FILE.
 
diff --git a/docs/source/rst-snippets/cli-post-scan-options.rst b/docs/source/rst-snippets/cli-post-scan-options.rst
@@ -1,5 +1,5 @@
-"post-scan" options
--------------------
+**Post-scan options**
+---------------------
 
 --mark-source              Set the "is_source" flag to true for directories that
                            contain over 90% of source files as direct children
@@ -34,10 +34,30 @@
 --license-references FILE  Returns reference data for all licenses and license
                            rules present in detections.
 
+--classify                 Classify files with flags telling if the
+                           file is a legal, or readme or test file,
+                           etc.
+
+                           Sub-options:
+
+                           - ``--license-clarity-score``
+                           - ``--tallies-key-files``
+                           - ``--summry``
+
 --summary                  Summarize scans by providing declared origin
                            information and other detected info at the
                            codebase attribute level.
 
+                           Sub-option of:
+
+                           - ``--classify``
+
+--todo                     Identify license and package detections which needs
+                           review as there are potential issues with the detections.
+                           Lists all the unique ambigious detections with references
+                           to the file path and line numbers, detection details and
+                           review comments to help explain the detection issue.
+
 --tallies                  Summarize license, copyright and other scans
                            at the codebase level with occurrence counts.
 
diff --git a/docs/source/rst-snippets/cli-pre-scan-options.rst b/docs/source/rst-snippets/cli-pre-scan-options.rst
@@ -1,19 +1,10 @@
-"pre-scan" options
-----------------------
+**Pre-scan options**
+--------------------
 
 --ignore <pattern>           Ignore files matching ``<pattern>``.
 
 --include <pattern>          Include files matching ``<pattern>``.
 
---classify                   Classify files with flags telling if the
-                             file is a legal, or readme or test file,
-                             etc.
-
-                             Sub-options:
-
-                             - ``--license-clarity-score``
-                             - ``--tallies-key-files``
-
 --facet <facet_pattern>      Here ``<facet_pattern>`` represents
                              ``<facet>=<pattern>``. Add the ``<facet>``
                              to files with a path matching ``<pattern>``.
diff --git a/docs/source/rst-snippets/improve-docs.rst b/docs/source/rst-snippets/improve-docs.rst
@@ -4,4 +4,5 @@ Something Missing?
 If something is missing in the documentation or if you found some part confusing, please file
 an `issue <https://github.com/aboutcode-org/scancode-toolkit/issues/new>`_ with your suggestions for
 improvement. Use the "Documentation Improvement" template.
+
 Your help makes ScanCode docs better, we love hearing from you!
