Documentation Add-ons and Extensions Reference

[中文]

This documentation is created using Sphinx application that renders text source files in reStructuredText (.rst) format located in docs directory. For some more details on that process, please refer to section Documenting Code.

Besides Sphinx, there are several other applications that help to provide nicely formatted and easy to navigate documentation. These applications are listed in section Setup for building documentation locally with the installed version numbers provided in file docs/requirements.txt.

We build ESP-IDF documentation for two languages (English, Simplified Chinese) and for multiple chips. Therefore we don’t run sphinx directly, there is a wrapper Python program build_docs.py that runs Sphinx.

On top of that, we have created a couple of custom add-ons and extensions to help integrate documentation with underlining ESP-IDF repository and further improve navigation as well as maintenance of documentation.

The purpose of this section is to provide a quick reference to the add-ons and the extensions.

Documentation Folder Structure

  • The ESP-IDF repository contains a dedicated documentation folder docs in the root.

  • The docs folder contains localized documentation in docs/en (English) and docs/zh_CN (simplified Chinese) subfolders.

  • Graphics files and fonts common to localized documentation are contained in docs/_static subfolder.

  • Remaining files in the root of docs as well as docs/en and docs/zh_CN provide configuration and scripts used to automate documentation processing including the add-ons and extensions.

  • Sphinx extensions are provided in two directories, extensions and idf_extensions.

  • A _build directory is created in the docs folder by build_docs.py. This directory is not added to the ESP-IDF repository.

Add-ons and Extensions Reference

Config Files

docs/conf_common.py

This file contains configuration common to each localized documentation (e.g. English, Chinese). The contents of this file is imported to standard Sphinx configuration file conf.py located in respective language folders (e.g. docs/en, docs/zh_CN) during build for each language.

docs/sphinx-known-warnings.txt

There are couple of spurious Sphinx warnings that cannot be resolved without doing update to the Sphinx source code itself. For such specific cases, respective warnings are documented in sphinx-known-warnings.txt file, that is checked during documentation build, to ignore the spurious warnings.

Scripts

docs/build_docs.py

Top-level executable program which runs a Sphinx build for one or more language/target combinations. Run build_docs.py --help for full command line options.

When build_docs.py runs Sphinx it sets the idf_target configuration variable, sets a Sphinx tag with the same name as the configuration variable, and uses some environment variables to communicate paths to IDF-Specific Extensions.

docs/check_lang_folder_sync.sh

To reduce potential discrepancies when maintaining concurrent language version, the structure and filenames of language folders docs/en and docs/zh_CN folders should be kept identical. The script check_lang_folder_sync.sh is run on each documentation build to verify if this condition is met.

Note

If a new content is provided in e.g. English, and there is no any translation yet, then the corresponding file in zh_CN folder should contain an .. include:: directive pointing to the source file in English. This will automatically include the English version visible to Chinese readers. For example if a file docs/zh_CN/contribute/documenting-code.rst does not have a Chinese translation, then it should contain .. include:: ../../en/contribute/documenting-code.rst instead.

Non-Docs Scripts

These scripts are used to build docs but also used for other purposes:

tools/gen_esp_err_to_name.py

This script is traversing the ESP-IDF directory structure looking for error codes and messages in source code header files to generate an .inc file to include in documentation under Error Codes Reference.

tools/kconfig_new/confgen.py

Options to configure ESP-IDF’s components are contained in Kconfig files located inside directories of individual components, e.g. components/bt/Kconfig. This script is traversing the component directories to collect configuration options and generate an .inc file to include in documentation under Configuration Options Reference.

Generic Extensions

These are Sphinx extensions developed for IDF that don’t rely on any IDF-docs-specific behaviour or configuration:

docs/extensions/toctree_filter.py

Sphinx extensions overrides the :toctree: directive to allow filtering entries based on whether a tag is set, as :tagname: toctree_entry. See the Python file for a more complete description.

docs/extensions/list_filter.py

Sphinx extensions that provides a .. list:: directive that allows filtering of entries in lists based on whether a tag is set, as :tagname: - list content. See the Python file for a more complete description.

docs/extensions/html_redirects.py

During documentation lifetime, some source files are moved between folders or renamed. This Sphinx extension adds a mechanism to redirect documentation pages that have changed URL by generating in the Sphinx output static HTML redirect pages. The script is used together with a redirection list html_redirect_pages. conf_common.py builds this list from docs/page_redirects.txt.

Third Party Extensions

  • sphinxcontrib extensions for blockdiag, seqdiag, actdiag, nwdiag, rackdiag & packetdiag diagrams.

  • Sphinx selective exclude eager_only extension.

IDF-Specific Extensions

Build System Integration

docs/idf_extensions/build_system/

Python package implementing a Sphinx extension to pull IDF build system information into the docs build.

  • Creates a dummy CMake IDF project and runs CMake to generate metadata.

  • Registers some new configuration variables and emits a new Sphinx event, both for use by other extensions.

Configuration Variables

  • docs_root - The absolute path of the $IDF_PATH/docs directory

  • idf_path - The value of IDF_PATH variable, or the absolute path of IDF_PATH if environment unset

  • build_dir - The build directory passed in by build_docs.py, default will be like _build/<lang>/<target>

  • idf_target - The IDF_TARGET value. Expected that build_docs.py set this on the Sphinx command line

New Event

idf-info event is emitted early in the build, after the dummy project CMake run is complete.

Arguments are (app, project_description), where project_description is a dict containing the values parsed from project_description.json in the CMake build directory.

Other IDF-specific extensions subscribe to this event and use it to set up some docs parameters based on build system info.

Other Extensions

docs/idf_extensions/include_build_file.py

The include-build-file directive is like the built-in include-file directive, but file path is evaluated relative to build_dir.

docs/idf_extensions/kconfig_reference.py

Subscribes to idf-info event and uses confgen to generate kconfig.inc from the components included in the default project build. This file is then included into Project Configuration.

docs/idf_extensions/link_roles.py

This is an implementation of a custom Sphinx Roles to help linking from documentation to specific files and folders in ESP-IDF. For description of implemented roles, please see Linking Examples and Linking Language Versions.

docs/idf_extensions/esp_err_definitions.py

Small wrapper extension that calls gen_esp_err_to_name.py and updates the included .rst file if it has changed.

docs/idf_extensions/gen_toolchain_links.py

There couple of places in documentation that provide links to download the toolchain. To provide one source of this information and reduce effort to manually update several files, this script generates toolchain download links and toolchain unpacking code snippets based on information found in tools/toolchain_versions.mk.

docs/idf_extensions/gen_version_specific_includes.py

Another extension to automatically generate reStructuredText Text .inc snippets with version-based content for this ESP-IDF version.

docs/idf_extensions/util.py

A collection of utility functions useful primarily when building documentation locally (see Setup for building documentation locally) to reduce the time to generate documentation on a second and subsequent builds.

docs/idf_extensions/format_idf_target.py

An extension for replacing generic target related names with the idf_target passed to the Sphinx command line. This is a {IDF_TARGET_NAME}, with /{IDF_TARGET_PATH_NAME}/soc.c, compiled with {IDF_TARGET_TOOLCHAIN_PREFIX}-gcc with CONFIG_{IDF_TARGET_CFG_PREFIX}_MULTI_DOC will, if the backspaces are removed, render as This is a ESP32, with /esp32/soc.c, compiled with xtensa-esp32-elf-gcc with CONFIG_ESP32_MULTI_DOC.

Also supports markup for defining local (single .rst-file) substitions with the following syntax: {IDF_TARGET_TX_PIN:default=”IO3”,esp32=”IO4”,esp32s2=”IO5”}

This will define a replacement of the tag {IDF_TARGET_TX_PIN} in the current rst-file.

The extension also overrides the default .. include:: directive in order to format any included content using the same rules.

These replacements cannot be used inside markup that rely on alignment of characters, e.g. tables.

docs/idf_extensions/latex_builder.py

An extension for adding ESP-IDF specific functionality to the latex builder. Overrides the default Sphinx latex builder.

Creates and adds the espidf.sty latex package to the output directory, which contains some macros for run-time variables such as IDF-Target.

docs/idf_extensions/gen_defines.py

Sphinx extension to integrate defines from IDF into the Sphinx build, runs after the IDF dummy project has been built.

Parses defines and adds them as sphinx tags.

Emits the new ‘idf-defines-generated’ event which has a dictionary of raw text define values that other extensions can use to generate relevant data.

docs/idf_extensions/exclude_docs.py

Sphinx extension that updates the excluded documents according to the conditional_include_dict {tag:documents}. If the tag is set, then the list of documents will be included.

Also responsible for excluding documents when building with the config value docs_to_build set. In these cases all documents not listed in docs_to_build will be excluded.

Subscribes to idf-defines-generated as it relies on the sphinx tags to determine which documents to exclude

docs/idf_extensions/run_doxygen.py

Subscribes to idf-defines-generated event and runs Doxygen (docs/doxygen/Doxyfile_common) to generate XML files describing key headers, and then runs Breathe to convert these to .inc files which can be included directly into API reference pages.

Pushes a number of target-specific custom environment variables into Doxygen, including all macros defined in the project’s default sdkconfig.h file and all macros defined in all soc component xxx_caps.h headers. This means that public API headers can depend on target-specific configuration options or soc capabilities headers options as #ifdef & #if preprocessor selections in the header.

This means we can generate different Doxygen files, depending on the target we are building docs for.

Please refer to Documenting Code and API Documentation Template, section API Reference for additional details on this process.

Provide feedback about this document