docs: generate documentation for Kconfig options

This adds a simple script which parses Kconfig files using Kconfiglib
and emits ReST document with the list of all options. For each option
a link target is generated, to make it possible to link to any Kconfig
option from the rest of the documentation.
Since Kconfiglib is not on PyPI, the latest version (45f87b9d) is
bundled into the docs directory.

docs/api-guides/build-system.rst

@@ -568,6 +568,8 @@ is overridden then the component can instruct the linker to link other binaries
 .. _GNU Make Manual: https://www.gnu.org/software/make/manual/make.html
 
 
+.. _custom-sdkconfig-defaults:
+
 Custom sdkconfig defaults
 -------------------------
 

docs/api-reference/index.rst

@@ -12,4 +12,5 @@ API Reference
    Protocols <protocols/index>
    Storage <storage/index>
    System <system/index>
+   Configuration Options <kconfig>
 

docs/api-reference/kconfig.rst

@@ -0,0 +1,28 @@
+Configuration Options
+*********************
+
+Introduction
+============
+
+ESP-IDF uses Kconfig_ system to provide a compile-time configuration mechanism. Kconfig is based around options of several types: integer, string, boolean. Kconfig files specify dependencies between options, default values of the options, the way the options are grouped together, etc.
+
+Applications developers can use ``make menuconfig`` build target to edit components' configuration. This configuration is saved inside ``sdkconfig`` file in the project root directory. Based on ``sdkconfig``, application build targets will generate ``sdkconfig.h`` file in the build directory, and will make sdkconfig options available to component makefiles.
+
+Using sdkconfig.defaults
+========================
+
+When updating ESP-IDF version, it is not uncommon to find that new Kconfig options are introduced. When this happens, application build targets will offer an interactive prompt to select values for the new options. New values are then written into ``sdkconfig`` file. To supress interactive prompts, applications can either define ``BATCH_BUILD`` environment variable, which will cause all prompts to be suppressed. This is the same effect as that of ``V`` or ``VERBOSE`` variables. Alternatively, ``defconfig`` build target can be used to update configuration for all new variables to the default values.
+
+In some cases, such as when ``sdkconfig`` file is under revision control, the fact that ``sdkconfig`` file gets changed by the build system may be inconvenient. The build system offers a way to avoid this, in the form of ``sdkconfig.defaults`` file. This file is never touched by the build system, and must be created manually. It can contain all the options which matter for the given application. The format is the same as that of the ``sdkconfig`` file. Once ``sdkconfig.defaults`` is created, ``sdkconfig`` can be deleted and added to the ignore list of the revision control system (e.g. ``.gitignore`` file for git). Project build targets will automatically create ``sdkconfig`` file, populated with the settings from ``sdkconfig.defaults`` file, and the rest of the settings will be set to their default values. Note that when ``make defconfig`` is used, settings in sdkconfig will be overriden by the ones in ``sdkconfig.defaults``. For more information, see :ref:`custom-sdkconfig-defaults`.
+
+Configuration Options Reference
+===============================
+
+Subsequent sections contain the list of available ESP-IDF options, automatically generated from Kconfig files. Note that depending on the options selected, some options listed here may not be visible by default in the interface of menuconfig.
+
+By convention, all option names are upper case with underscores. When Kconfig generates sdkconfig and sdkconfig.h files, option names are prefixed with ``CONFIG_``. So if an option ``ENABLE_FOO`` is defined in a Kconfig file and selected in menuconfig, then sdkconfig and sdkconfig.h files will have ``CONFIG_ENABLE_FOO`` defined. In this reference, option names are also prefixed with ``CONFIG_``, same as in the source code.
+
+
+.. include:: /_build/inc/kconfig.inc
+
+.. _Kconfig: https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt

docs/conf.py

@@ -29,6 +29,8 @@ print "Calling Doxygen to generate latest XML files"
 call('doxygen')
 # Generate 'api_name.inc' files using the XML files by Doxygen
 os.system("python gen-dxd.py")
+# Generate 'kconfig.inc' file from components' Kconfig files
+os.system("python gen-kconfig-doc.py > _build/inc/kconfig.inc")
 
 # http://stackoverflow.com/questions/12772927/specifying-an-online-image-in-sphinx-restructuredtext-format
 # 

docs/gen-kconfig-doc.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# gen-kconfig-doc.py — generate Sphinx .rst file from Kconfig files
+#
+# This script iterates over Kconfig and Kconfig.projbuild files in
+# ESP-IDF component directories, and outputs documentation for these options
+# as ReST markup.
+# For each option in Kconfig file (e.g. 'FOO'), CONFIG_FOO link target is
+# generated, allowing options to be referenced in other documents 
+# (using :ref:`CONFIG_FOO`)
+#
+# This script uses kconfiglib library to do all the work of parsing Kconfig
+# files: https://github.com/ulfalizer/Kconfiglib
+#
+# Copyright 2017 Espressif Systems (Shanghai) PTE LTD
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http:#www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+import os
+import kconfiglib
+
+# Indentation to be used in the generated file
+INDENT = '    '
+
+# Characters used when underlining section heading
+HEADING_SYMBOLS = '#*=-^"+'
+
+# Keep the heading level in sync with api-reference/kconfig.rst
+INITIAL_HEADING_LEVEL = 2
+MAX_HEADING_LEVEL = 5
+OPTION_HEADING_LEVEL = 6
+
+
+def print_menu_contents(title, items, heading_level, breadcrumbs):
+    if title:
+        print_section_heading(title, heading_level)
+    for entry in items:
+        if entry.is_menu():
+            if len(breadcrumbs) > 0:
+                new_breadcrumbs = breadcrumbs + ' > ' + entry.get_title()
+            else:
+                new_breadcrumbs = entry.get_title()
+
+            print_menu_contents(entry.get_title(), entry.get_items(),
+                                min(heading_level + 1, MAX_HEADING_LEVEL),
+                                new_breadcrumbs)
+        elif entry.is_choice():
+            print_choice(entry, breadcrumbs)
+        else:
+            if len(entry.get_prompts()) == 0:
+                # Skip entries which can never be visible
+                continue
+            # Currently this does not handle 'menuconfig' entires in any special way,
+            # as Kconfglib offers no way of recognizing them automatically.
+            print_option(entry, breadcrumbs)
+        # Trailing newline after every option
+        print
+
+def print_choice(choice, breadcrumbs):
+    print_option(choice, breadcrumbs)
+    print
+    print '%sAvailable options:' % INDENT
+    for opt in choice.get_symbols():
+        # Format available options as a list
+        print '%s- %s' % (INDENT * 2, opt.name)
+
+def print_section_heading(title, heading_level):
+    print title
+    print HEADING_SYMBOLS[heading_level] * len(title)
+    print
+
+def print_option(opt, breadcrumbs):
+    # add link target so we can use :ref:`CONFIG_FOO` 
+    print '.. _CONFIG_%s:' % opt.name
+    print
+    print_section_heading(opt.name, OPTION_HEADING_LEVEL)
+    if len(opt.prompts) > 0:
+        print '%s%s' % (INDENT, opt.prompts[0][0])
+        print
+    print '%s:emphasis:`Found in: %s`' % (INDENT, breadcrumbs)
+    print
+    if opt.get_help() is not None:
+        # Help text normally contains newlines, but spaces at the beginning of
+        # each line are stripped by kconfiglib. We need to re-indent the text
+        # to produce valid ReST.
+        print '%s%s' % (INDENT, opt.get_help().replace('\n', '\n%s' % INDENT))
+
+def process_kconfig_file(kconfig_file, heading_level, breadcrumbs):
+    if os.path.exists(kconfig_file):
+        cfg = kconfiglib.Config(kconfig_file, print_warnings=True)
+        print_menu_contents(None, cfg.get_top_level_items(), heading_level, breadcrumbs)
+
+def print_all_components():
+    heading_level = INITIAL_HEADING_LEVEL
+    # Currently this works only for IDF components.
+    # TODO: figure out if this can be re-used for documenting applications?
+    components_path = os.path.join(os.path.curdir, '..', 'components')
+    for component_name in os.listdir(components_path):
+        if component_name.startswith('.'):
+            continue    # skip system thumbnail folders
+
+        kconfig_file_path = os.path.join(components_path, component_name, 'Kconfig')
+
+        process_kconfig_file(kconfig_file_path, heading_level, 'Component config')
+        process_kconfig_file(kconfig_file_path + '.projbuild', heading_level, '')
+
+if __name__ == '__main__':
+    print_all_components()