Merge pull request matplotlib#23621 from tacaswell/family-parsing

DOC: update and extend fonts explanation

Author: QuLogic

doc/users/explain/fonts.rst

@@ -1,23 +1,27 @@
 .. redirect-from:: /users/fonts
 
-Fonts in Matplotlib text engine
-===============================
+Fonts in Matplotlib
+===================
 
 Matplotlib needs fonts to work with its text engine, some of which are shipped
-alongside the installation. However, users can configure the default fonts, or
-even provide their own custom fonts! For more details, see :doc:`Customizing
-text properties </tutorials/text/text_props>`.
+alongside the installation.  The default font is `DejaVu Sans
+<https://dejavu-fonts.github.io>`_ which covers most European writing systems.
+However, users can configure the default fonts, and provide their own custom
+fonts.  See :doc:`Customizing text properties </tutorials/text/text_props>` for
+details and :ref:`font-nonlatin` in particular for glyphs not supported by
+DejaVu Sans.
 
-However, Matplotlib also provides an option to offload text rendering to a TeX
-engine (``usetex=True``),
-see :doc:`Text rendering with LaTeX </tutorials/text/usetex>`.
+Matplotlib also provides an option to offload text rendering to a TeX engine
+(``usetex=True``), see :doc:`Text rendering with LaTeX
+</tutorials/text/usetex>`.
 
-Font specifications
--------------------
-Fonts have a long and sometimes incompatible history in computing, leading to
-different platforms supporting different types of fonts. In practice, there are
-3 types of font specifications Matplotlib supports (in addition to 'core
-fonts', more about which is explained later in the guide):
+Fonts in PDF and PostScript
+---------------------------
+
+Fonts have a long (and sometimes incompatible) history in computing, leading to
+different platforms supporting different types of fonts.  In practice, there
+are 3 types of font specifications Matplotlib supports (in addition to 'core
+fonts' in pdf which is explained later in the guide):
 
 .. list-table:: Type of Fonts
    :header-rows: 1
@@ -37,20 +41,19 @@ fonts', more about which is explained later in the guide):
      - Hinting supported (virtual machine processes the "hints")
    * - Non-subsetted through Matplotlib
      - Subsetted via external module `ttconv <https://github.com/sandflow/ttconv>`_
-     - Subsetted via external module `fonttools <https://github.com/fonttools/fonttools>`_
+     - Subsetted via external module `fonttools <https://github.com/fonttools/fonttools>`__
 
 NOTE: Adobe will disable support for authoring with Type 1 fonts in
 January 2023. `Read more here. <https://helpx.adobe.com/fonts/kb/postscript-type-1-fonts-end-of-support.html>`_
 
-Special mentions
-^^^^^^^^^^^^^^^^
+
 Other font specifications which Matplotlib supports:
 
 - Type 42 fonts (PS):
 
   - PostScript wrapper around TrueType fonts
   - 42 is the `Answer to Life, the Universe, and Everything! <https://en.wikipedia.org/wiki/Answer_to_Life,_the_Universe,_and_Everything>`_
-  - Matplotlib uses an external library called `fonttools <https://github.com/fonttools/fonttools>`_
+  - Matplotlib uses an external library called `fonttools <https://github.com/fonttools/fonttools>`__
     to subset these types of fonts
 
 - OpenType fonts:
@@ -60,50 +63,37 @@ Other font specifications which Matplotlib supports:
   - Generally contain a much larger character set!
   - Limited Support with Matplotlib
 
-Subsetting
-----------
-Matplotlib is able to generate documents in multiple different formats. Some of
-those formats (for example, PDF, PS/EPS, SVG) allow embedding font data in such
-a way that when these documents are visually scaled, the text does not appear
-pixelated.
-
-This can be achieved by embedding the *whole* font file within the
-output document. However, this can lead to very large documents, as some
-fonts (for instance, CJK - Chinese/Japanese/Korean fonts) can contain a large
-number of glyphs, and thus their embedded size can be quite huge.
-
-Font Subsetting can be used before generating documents, to embed only the
-*required* glyphs within the documents. Fonts can be considered as a collection
-of glyphs, so ultimately the goal is to find out *which* glyphs are required
-for a certain array of characters, and embed only those within the output.
-
-.. note::
-  The role of subsetter really shines when we encounter characters like **ä**
-  (composed by calling subprograms for **a** and **¨**); since the subsetter
-  has to find out *all* such subprograms being called by every glyph included
-  in the subset, this is a generally difficult problem!
-
-Luckily, Matplotlib uses a fork of an external dependency called
-`ttconv <https://github.com/sandflow/ttconv>`_, which helps in embedding and
-subsetting font data. (however, recent versions have moved away from ttconv to
-pure Python for certain types: for more details visit
-`these <https://github.com/matplotlib/matplotlib/pull/18370>`_, `links <https://github.com/matplotlib/matplotlib/pull/18181>`_)
-
-| *Type 1 fonts are still non-subsetted* through Matplotlib. (though one will encounter these mostly via *usetex*/*dviread* in PDF backend)
-| **Type 3 and Type 42 fonts are subsetted**, with a fair amount of exceptions and bugs for the latter.
-
-What to use?
-------------
-Practically, most fonts that are readily available on most operating systems or
-are readily available on the internet to download include *TrueType fonts* and
-its "extensions" such as MacOS-resource fork fonts and the newer OpenType
-fonts.
+Font Subsetting
+~~~~~~~~~~~~~~~
+
+The PDF and PostScript formats support embedding fonts in files allowing the
+display program to correctly render the text, independent of what fonts are
+installed on the viewer's computer and without the need to pre-rasterize the text.
+This ensures that if the output is zoomed or resized the text does not become
+pixelated.  However, embedding full fonts in the file can lead to large output
+files, particularly with fonts with many glyphs such as those that support CJK
+(Chinese/Japanese/Korean).
+
+The solution to this problem is to subset the fonts used in the document and
+only embed the glyphs actually used.  This gets both vector text and small
+files sizes.  Computing the subset of the font required and writing the new
+(reduced) font are both complex problem and thus Matplotlib relies on
+`fontTools <https://fonttools.readthedocs.io/en/latest/>`__ and a vendored fork
+of `ttconv <https://github.com/sandflow/ttconv>`_.
+
+Currently Type 3, Type 42, and TrueType fonts are subseted.  Type 1 fonts are not.
+
+
+Core Fonts
+~~~~~~~~~~
 
-PS and PDF backends provide support for yet another type of fonts, which remove
-the need of subsetting altogether! These are called **Core Fonts**, and
-Matplotlib calls them via the keyword **AFM**; all that is supplied from
-Matplotlib to such documents are font metrics (specified in AFM format), and it
-is the job of the viewer applications to supply the glyph definitions.
+In addition to the ability to embed fonts, as part of the `PostScript
+<https://en.wikipedia.org/wiki/PostScript_fonts#Core_Font_Set>`_ and `PDF
+specification
+<https://docs.oracle.com/cd/E96927_01/TSG/FAQ/What%20are%20the%2014%20base%20fonts%20distributed%20with%20Acroba.html>`_
+there are 14 Core Font that compliant viewers must ensure are available.  If
+you restrict your document to only these fonts you do not have to embed any
+font information in the document but still get vector text.
 
 This is especially helpful to generate *really lightweight* documents.::
 
@@ -119,14 +109,89 @@ This is especially helpful to generate *really lightweight* documents.::
     fig.savefig("AFM_PDF.pdf", format="pdf")
     fig.savefig("AFM_PS.ps", format="ps)
 
-.. note::
-  These core fonts are limited to PDF and PS backends only; they can not be
-  rendered in other backends.
 
-  Another downside to this is that while the font metrics are standardized,
-  different PDF viewer applications will have different fonts to render these
-  metrics. In other words, the **output might look different on different
-  viewers**, as well as (let's say) Windows and Linux, if Linux tools included
-  free versions of the proprietary fonts.
+Fonts in SVG
+------------
+
+Text can output to SVG in two ways controlled by :rc:`svg.fonttype`:
+
+- as a path (``'path'``) in the SVG
+- as string in the SVG with font styling on the element (``'none'``)
+
+
+When saving via ``'path'`` Matplotlib will compute the path of the glyphs used
+as vector paths and write those to the output.  The advantage of this is that
+the SVG will look the same on all computers independent of what fonts are
+installed.  However the text will not be editable after the fact.
+In contrast saving with ``'none'`` will result in smaller files and the
+text will appear directly in the markup.  However, the appearance may vary
+based on the SVG viewer and what fonts are available.
+
+Fonts in Agg
+------------
 
-  This also violates the *what-you-see-is-what-you-get* feature of Matplotlib.
+To output text to raster formats via Agg, Matplotlib relies on `FreeType
+<https://www.freetype.org/>`_.  Because the exact rendering of the glyphs
+changes between FreeType versions we pin to a specific version for our image
+comparison tests.
+
+
+How Matplotlib selects fonts
+----------------------------
+
+Internally using a Font in Matplotlib is a three step process:
+
+1. a `.FontProperties` object is created (explicitly or implicitly)
+2. based on the `.FontProperties` object the methods on `.FontManager` are used
+   to select the closest "best" font Matplotlib is aware of (except for
+   ``'none'`` mode of SVG).
+3. the Python proxy for the font object is used by the backend code to render
+   the text -- the exact details depend on the backend via `.font_manager.get_font`.
+
+The algorithm to select the "best" font is a modified version of the algorithm
+specified by the `CSS1 Specifications
+<http://www.w3.org/TR/1998/REC-CSS2-19980512/>`_ which is used by web browsers.
+This algorithm takes into account the font family name (e.g. "Arial", "Noto
+Sans CJK", "Hack", ...), the size, style, and weight.  In addition to family
+names that map directly to fonts there are five "generic font family names" (
+serif, monospace, fantasy, cursive, and sans-serif) that will internally be
+mapped to any one of a set of fonts.
+
+Currently the public API for doing step 2 is `.FontManager.findfont` (and that
+method on the global `.FontManager` instance is aliased at the module level as
+`.font_manager.findfont`), which will only find a single font and return the absolute
+path to the font on the filesystem.
+
+Font Fallback
+-------------
+
+There is no font that covers the entire Unicode space thus it is possible for the
+users to require a mix of glyphs that can not be satisfied from a single font.
+While it has been possible to use multiple fonts within a Figure, on distinct
+`.Text` instances, it was not previous possible to use multiple fonts in the
+same `.Text` instance (as a web browser does).  As of Matplotlib 3.6 the Agg,
+SVG, PDF, and PS backends will "fallback" through multiple fonts in a single
+`.Text` instance:
+
+
+.. plot::
+   :include-source:
+   :caption: The string "There are 几个汉字 in between!" rendered with 2 fonts.
+
+   fig, ax = plt.subplots()
+   ax.text(
+       .5, .5, "There are 几个汉字 in between!",
+       family=['DejaVu Sans', 'WenQuanYi Zen Hei'],
+       ha='center'
+   )
+
+
+Internally this is implemented by setting The "font family" on
+`.FontProperties` objects to a list of font families.  A (currently)
+private API extracts a list of paths to all of the fonts found and then
+constructs a single `.ft2font.FT2Font` object that is aware of all of the fonts.
+Each glyph of the string is rendered using the first font in the list that
+contains that glyph.
+
+A majority of this work was done by Aitik Gupta supported by Google Summer of
+Code 2021.

tutorials/text/text_props.py

@@ -215,6 +215,8 @@
 #    matplotlib.rcParams['font.family'] = ['Family1', 'SerifFamily1', 'SerifFamily2', 'Family2']
 #
 #
+# .. _font-nonlatin:
+#
 # Text with non-latin glyphs
 # ==========================
 #