rustdoc_texmath

Summary

Add support for the de facto standard TeX-math-in-markdown syntax to Rustdoc. It’s currently implemented using the math-core library, which generates MathML Core and is restricted to the subset of LaTeX that can be implemented that way. If you use unsupported syntax, you get a compiler warning from rustdoc.

Motivation

It would be nice if we could write complex equations in our docs. We know that there's demand for this feature, first of all because people have asked for it, but mostly because of crates that did it themselves by loading katex.js with inline HTML.

As far as I know, the most popular ways of doing that are:

# Cargo.toml
[package.metadata.docs.rs]
rustdoc-args = ["--html-in-header", "katex-header.html", "--cfg", "docsrs"]

# .cargo/config.toml
[build]
rustdocflags = [ "--html-in-header", "./src/docs-header.html" ]

Because the first one is only read by docs.rs, and the second one is only read when cargo is run from within the project itself, neither of them work when you use cargo doc to browse the docs of your dependencies.

There is a way to make it work in cargo doc, but it seems to be less popular.

Providing an easy way to build self-contained docs that work for non-docs.rs readers is the main motivation for adding built-in support for math syntax to Rustdoc, but there are a few other quality of life improvements that come with this feature:

• We can report math syntax errors on the CLI, just like we do for intra-doc links.
• We can render math in the resulting web page without JavaScript. No flash of unstyled content or blocking scripts.
• Built-in TeX math doesn't require double-escaping, because the Markdown parser knows about math, and lets you backslash escape the dollar sign to disable it.
• Cross-crate inlining works.

Guide-level explanation

How to enable

To enable $\TeX$ math syntax in rustdoc, add this line to your crate root.

#![doc(syntax="+tex_math_dollars")]

In a future edition, we may enable it by default. If you need to turn it off, add this line to your crate root.

#![doc(syntax="-tex_math_dollars")]

The only supported values for doc(syntax) are "+tex_math_dollars" and "-tex_math_dollars", but we left the option open for future extensions.

When this feature is enabled, equations are wrapped in single or double $ dollar signs.

$$\sum_{i=0}^N x_i$$

The result looks like this:

$$\sum_{i=0}^N x_i$$

A detailed comparison between our syntax and KaTeX's can be found here.

Reference-level explanation

Disabling and enabling math syntax

The doc(syntax="+tex_math_dollars"|"-tex_math_dollars") attribute enables and disables support for parsing $-delimited TeX math in Rustdoc's Markdown.

Obviously, you can't set both of them at the same time on a single item. If neither of them are set, rustdoc will use the edition-specific default (which is currently to disable it).

This attribute can be set on any item that accepts doc comments. The syntax is inherited from the syntactic parent item. So, to list a few examples:

// lib.rs

//! This is *not* math syntax: $x$

#[doc(syntax="-tex_math_dollars")]
pub mod bar;
#[doc(syntax="+tex_math_dollars")]
pub mod baz;

#[doc(syntax="+tex_math_dollars")]
/// This *is* math syntax: $x$
pub struct Foo;

impl Foo {
    /// This is *not* math syntax: $x$
    pub fn foo() {}
}

#[doc(syntax="+tex_math_dollars")]
impl Foo {
    /// This *is* math syntax: $x$
    pub fn bar() {}
}

#[doc(syntax="+tex_math_dollars")]
/// <https://github.com/rust-lang/rust/pull/158514#issuecomment-4825510621>
pub fn wizzywig() {
    #[doc(syntax="-tex_math_dollars")]
    /// This is *not* math syntax: $x$
    pub struct WizzyWig;

    impl Foo {
        /// This *is* math syntax: $x$
        pub fn baz() {}
    }
}

// bar.rs
#![doc(syntax="+tex_math_dollars")] //~ ERROR only one doc syntax can be declared on a single item

// baz.rs
//! This *is* math syntax

Writing math code in markdown

Math expressions are wrapped in $ signs. One dollar sign means "inline" math, and two means "display" math.

Inline math cannot have any whitespace at the start or end of its contents, so $1$ is a math span, but $ 1 $ is not. Inline math spans also can't be empty.

Display math is allowed to have space at the start, so $$ 1 $$ is a display math span.

Unescaped curly braces within math spans must balance, and unescaped dollar signs can only appear between unescaped curly braces, so $$ 1 {$} 2 $$ is parsed as a display math span, but $$ 1 $ 2 $$ and $$ { $$ are not.

Math syntax

Within a math span, whitespaces are used for grouping and formatting. But you can't have more than one line break in a row within a math span, because that ends the paragraph that contains it.

Other characters are usually rendered literally, except for

• backslashes, \, which are the sigil for commands
• curly braces, { and }, which are used for command arguments
• dollar signs, $, which delimit math spans
• number signs, #, which are reserved for use in a future macro system
• ampersands, &, which are used for writing matrices and tables
• circumflex, ^, which is used for exponents
• underscore, _, which is used for subscript
• single quote, ', which becomes the prime symbol
• tilde, ~, which becomes a rendered, non-breaking space (since ordinary spaces are used for grouping)
• percent, %, which mark line comments
• NUL, which is not allowed

Commands are used to write things that can't easily be typed on a keyboard, and for complex layouts like fractions and matrices. The math-core parser that we use implements hundreds of commands.

Drawbacks

There is no such thing as invalid Markdown

Adding new syntax to Rustdoc's Markdown is rough, because it's so difficult to do without causing widespread breakage. As spelled out in the CommonMark spec, "any sequence of characters is a valid CommonMark document," so changing anything so that it acts like a metacharacter where it didn't used to changes the behavior of already-valid documents; a breaking change.

And, unlike when GitHub redesigned their Markdown as a CommonMark dialect, we can't run a one-time batch converter job over old crates.io crates.

This class of problem has come up when intra-doc links were designed, when pulldown-cmark was last updated, when hoedown was replaced with pulldown-cmark in the first place, and when anyone proposes replacing Markdown with something else that has a "principled extension" system.

Verbosity or breakage as side effect

From the perspective of 99% of doc authors who didn't want to write a math span in the first place, false positives that mangle their generated docs are a nasty papercut. Having to read English text without any spaces caused by failing to escape the dollar signs sucks (though not as bad as accidentally triggering a link refdef, as the result might still be legible). The LaTeX math syntax is forgiving enough that English prose is often syntactically valid LaTeX, so Rustdoc invalid LaTeX warnings can't catch every unescaped dollar sign.

But if we assume that every doc author adds the escapes that they need, this forces doc comments to have more escaped metacharacters than they used to. This makes doc comments less easily readable in their source form, imposing a cost on the 99% that don't want the feature in favor of the 1% who do.

This argument, if taken to its logical extreme, would imply that we should use plain text doc comments with no extra formatting features. The downside of doing that is similar to the downside of not offering TeX math: users who really want bold text deploy unicode crimes and pictures of text, which create accessibility problems.

Rationale and alternatives

Why LaTeX math in markdown, specifically?

I would like to avoid the annoying scenario where Rustdoc deploys a complicated, special purpose language, then the community moves on to some new, incompatible language, and we’re stuck maintaining it ourselves because of the stability promise.

There are a lot of special-purpose technical notations that we might theoretically want to support, but LaTeX-math-in-markdown is special, for two reasons:

• Lindy effect: LaTeX is an established standard that is not going anywhere any time soon.
• There is more than one implementation of the subset of LaTeX that we need.

The pull request I've been working on uses math-core, but, if that implementation turns out to be problematic, we could pivot to another one, like pulldown-latex, or katex run in quick-js ¹.

Why use a Rust library, specifically, for the implementation?

If your LaTeX syntax doesn't parse, you'll get a warning at Rustdoc compile time, just like you do with broken intra-doc links:

error: unknown command "\frobnicate"
  --> $DIR/basic.rs:6:5
   |
LL | //! $\frobnicate{2}$
   |     ^^^^^^^^^^^^^^^^
   |
note: the lint level is defined here
  --> $DIR/basic.rs:2:9
   |
LL | #![deny(rustdoc::invalid_math)]
   |         ^^^^^^^^^^^^^^^^^^^^^

error: aborting due to 1 previous error

That's harder if the engine is written in a different language than the compiler.

Why target MathML

MathML Core is supported by most browsers, but with a few known bugs that the math-core library tries to work around. It is a strict accessibility improvement over rendering SVG, for example.

Prior art

• https://github.com/cben/mathdown/wiki/math-in-markdown
• https://en.wikibooks.org/wiki/LaTeX/Mathematics
• The span parsing is based on the math spec for commonmark-hs, which is the parser used if you run pandoc in gfm mode.
• Span parsing is documented in more detail in the math spec for pulldown-cmark.

Unresolved questions

Avoiding Hyrum's Law

There are a lot of \commands in math-core, and some of them are known buggy, meaning they don't match LaTeX itself. We don't want authors to rely on those bugs, either accidentally or in a workaround.

Normally, we might "phase in" new commands by making them unstable first, letting more risk-tolerant authors try it out, then make it available to everyone else. But math-core doesn't have an API for that.

Font

Right now, math formulas default to Noto Sans Math.

This was chosen because it's inoffensive and fine. But it is a sans serif font face, that will usually be surrounded by serif text.

We should also look into subsetting the font.

Future possibilities

Extending doc(syntax)

The name tex_math_dollars is deliberately the same name and syntax used by Pandoc for this feature. If we add more syntactic features, we can follow the same pattern. An example of how to build on this:

(* if no language is supplied, the default is "rustdoc_markdown", so "+tex_math_dollars" is synonymous with "rustdoc_markdown+tex_math_dollars" *)
doc syntax = [ language ], { ( "+" | "-" ), extension } ;
(* "commonmark" enables no extensions (you can, of course, add them)
  "rustdoc_markdown" is synonymous with `commonmark+intra_doc_links+doctests+smart+pipe_tables+strikeout+footnotes+task_lists"
  "gfm" is synonymous with `commonmark+smart+tex_math_dollars+pipe_tables+strikeout+footnotes+task_lists+emoji+tex_math_gfm+alerts+autolink_bare_uris+yaml_metadata_block"
  "typst" doesn't support any extensions except "intra_doc_links" and "doctests", so, for example, `doc(syntax="typst+pipe_tables")` is an error *)
language = "commonmark" | "rustdoc_markdown" | "gfm" | "typst" ;
extension = 
   "intra_doc_links"
 | "doctests"
 | "smart"
 | "pipe_tables"
 | "strikeout"
 | "footnotes"
 | "task_lists"
 | "tex_math_dollars"
 | "emoji"
 | "tex_math_gfm"
 | "alerts"
 | "autolink_bare_uris"
 | "yaml_metadata_block" ;

If a parent and child element both have doc(syntax) attributes, they aren't merged. The child just overrides the parent.

Undelimited environments

It's a relatively rare feature, but Jupyter Notebook and a few others support LaTeX environments introduced with the \begin{foo} / \end{foo} syntax without wrapping dollar signs. Since backslashes in Markdown only have meaning when followed by punctuation, the false positives shouldn't be that common. Since we don't have to worry about false positives, we can treat it like a CommonMark block construct and allow blank lines in it.

/// Computes sum from `start` to `end` of the given function.
///
/// \begin{equation}
/// \sum_{i=start}^{end}{f(i)}
/// \end{equation}
fn sum(f: impl FnMut(usize) -> usize, start: usize, end: usize) -> usize {
    let mut result = 0;
    for i in start..=end {
        result += f(i);
    }
    result
}

Custom macros

Some implementations, such as Pandoc's latex_macros extension, let you write LaTeX macro definitions directly in Markdown. For example, these two doc comments are equivalent:

#[doc(syntax="+tex_math_dollars+latex_macros")]
/// \newcommand{\tuple}[1]{\langle #1 \rangle}
///
/// $\tuple{a, b, c}$
fn foo_bar() {}

#[doc(syntax="+tex_math_dollars+latex_macros")]
/// $\langle a, b, c \rangle$
fn foo_bar() {}

Existing users of rustdoc and katex already write custom functions. This isn't a niche feature.

Drawing and charting syntax

There are a lot of different chart formats we could try to support. The tough part is that we want to support it long-term, give error messages at compile time (if the language has a concept of errors), and, ideally, have a specification without much churn.

• PlantUML is pretty much exactly what we would want. But we don't want to bundle a JRE.
• The other obvious choice is Mermaid, because GitHub supports it. It implements special notations and layout algorithms for commonplace chart notations, so Mermaid syntax is terse and easy to edit. There is also Merman, which implements the same language in Rust. Like Typst, the biggest downside is that they're new and unproven: Mermaid 11.0.0, the most recent breaking change, came out in August 2024.
• Svgbob actually has a Rust implementation. Ironic, since Svgbob has no syntax errors, it's actually less important to have a Rust implementation than it is for the others, which have the possibility of an "invalid document" with errors that we would want to report at compile time. We could fall back to displaying the original ASCII, and the result would still be legible.
• GraphViz DOT has a Rust implementation, too. Like LaTeX, DOT is popular, stable, and has multiple third-party reimplementations. However, it is not terse unless its layout algorithm does what you need.

A simple sequence diagram, in each language

@startuml
Actor1 -> Actor2: Start
Actor2 -> Actor3: Do something
Actor3 -> Actor1: Return
@enduml

sequenceDiagram
  Actor1 ->> Actor2: Start
  Actor2 ->> Actor3: Do something
  Actor3 ->> Actor1: Return

+------+ +------+ +------+
|Actor1| |Actor2| |Actor3|
+------+ +------+ +------+
    :        :        :
    :------->:        : Start
    :        :        :
    :        :------->: Do something
    :        :        :
    :<----------------: Return

digraph SEQ_DIAGRAM {
    graph [overlap=true, splines=line, nodesep=1.0, ordering=out];
    node [shape=none, width=0, height=0, label=""];

    // Draw actors
    {
        rank=same;
        node[shape=rectangle, height=0.7, width=2];
        Actor1 [label="Actor1"];
        Actor2 [label="Actor1"];
        Actor3 [label="Actor1"];
    }
    // Draw vertical lines
    {
        edge [style=dashed, weight=6, arrowhead=none];
        Actor1 -> a1 -> a2 -> a3;
        Actor2 -> b1 -> b2 -> b3;
        Actor3 -> c1 -> c2 -> c3;
    }
    // Draws activations
    { rank=same; a1 -> b1 [label="Start", arrowhead=normal]; b1 -> c1 [style=invis]; }
    { rank=same; a2 -> b2 [style=invis]; b2 -> c2 [label="Do something", arrowhead=normal]; }
    { rank=same; a3 -> b3 [arrowhead=normal, dir=back, label="Return"]; b3 -> c3; }
}