Merge branch 'doc/submodule_links' into 'master'

doc: roles for linking files now correctly links to submodule repositories

Closes IDF-1584

See merge request espressif/esp-idf!8405
This commit is contained in:
Krzysztof Budzynski 2020-04-22 01:49:18 +08:00
commit 0842aed2bc
3 changed files with 63 additions and 25 deletions

View file

@ -33,34 +33,35 @@ These third party libraries can be included into the application (firmware) prod
* `linenoise`_ line editing library Copyright (c) 2010-2014 Salvatore Sanfilippo, Copyright (c) 2010-2013 Pieter Noordhuis, licensed under 2-clause BSD license.
* `libcoap`_ COAP library Copyright (c) 2010-2017 Olaf Bergmann and others, is licensed under 2-clause BSD license.
* `libcoap`_ COAP library Copyright (c) 2010-2017 Olaf Bergmann and others, is licensed under 2-clause BSD license as described in :component_file:`LICENSE file <coap/libcoap/LICENSE>` and :component_file:`COPYING file <coap/libcoap/COPYING>` .
* `libexpat`_ XML parsing library Copyright (c) 1998-2000 Thai Open Source Software Center Ltd and Clark Cooper, Copyright (c) 2001-2017 Expat maintainers, is licensed under MIT license.
* `libexpat`_ XML parsing library Copyright (c) 1998-2000 Thai Open Source Software Center Ltd and Clark Cooper, Copyright (c) 2001-2017 Expat maintainers, is licensed under MIT license as described in :component_file:`COPYING file <expat/expat/expat/COPYING>`.
* `FatFS`_ library, Copyright (C) 2017 ChaN, is licensed under :component_file:`a BSD-style license <fatfs/src/ff.h#L1-L18>`.
* `cJSON`_ library, Copyright (c) 2009-2017 Dave Gamble and cJSON contributors, is licensed under MIT license.
* `cJSON`_ library, Copyright (c) 2009-2017 Dave Gamble and cJSON contributors, is licensed under MIT license as described in :component_file:`LICENSE file <json/cJSON/LICENSE>`.
* `libsodium`_ library, Copyright (c) 2013-2018 Frank Denis, is licensed under ISC license.
* `libsodium`_ library, Copyright (c) 2013-2018 Frank Denis, is licensed under ISC license as described in :component_file:`LICENSE file <libsodium/libsodium/LICENSE>`.
* `micro-ecc`_ library, Copyright (c) 2014 Kenneth MacKay, is licensed under 2-clause BSD license.
* `nghttp2`_ library, Copyright (c) 2012, 2014, 2015, 2016 Tatsuhiro Tsujikawa, Copyright (c) 2012, 2014, 2015, 2016 nghttp2 contributors, is licensed under MIT license.
* `nghttp2`_ library, Copyright (c) 2012, 2014, 2015, 2016 Tatsuhiro Tsujikawa, Copyright (c) 2012, 2014, 2015, 2016 nghttp2 contributors, is licensed under MIT license as described in :component_file:`COPYING file <nghttp/nghttp2/COPYING>`.
* `Mbed TLS`_ library, Copyright (C) 2006-2018 ARM Limited, is licensed under Apache License 2.0.
* `Mbed TLS`_ library, Copyright (C) 2006-2018 ARM Limited, is licensed under Apache License 2.0 as described in :component_file:`LICENSE file <mbedtls/mbedtls/LICENSE>`.
* `SPIFFS`_ library, Copyright (c) 2013-2017 Peter Andersson, is licensed under MIT license.
* `SPIFFS`_ library, Copyright (c) 2013-2017 Peter Andersson, is licensed under MIT license as described in :component_file:`LICENSE file <spiffs/spiffs/LICENSE>`.
* `TinyCBOR`_ library, Copyright (c) 2017 Intel Corporation, is licensed under MIT License.
* `TinyCBOR`_ library, Copyright (c) 2017 Intel Corporation, is licensed under MIT License as described in :component_file:`LICENSE file <cbor/tinycbor/LICENSE>`.
* :component_file:`SD/MMC driver <sdmmc/sdmmc_cmd.c>` is derived from `OpenBSD SD/MMC driver`_, Copyright (c) 2006 Uwe Stuehler, and is licensed under BSD license.
* :component:`Asio <asio>`, Copyright (c) 2003-2018 Christopher M. Kohlhoff is licensed under the Boost Software License.
* :component:`Asio <asio>`, Copyright (c) 2003-2018 Christopher M. Kohlhoff is licensed under the Boost Software License as described in :component_file:`COPYING file<asio/asio/asio/COPYING>`.
* :component:`ESP-MQTT <mqtt>` MQTT Package (contiki-mqtt) - Copyright (c) 2014, Stephen Robinson, MQTT-ESP - Tuan PM <tuanpm at live dot com> is licensed under Apache License 2,0 as described in :component_file:`LICENSE file <mqtt/esp-mqtt/LICENSE>`.
* :component:`ESP-MQTT <mqtt>` MQTT Package (contiki-mqtt) - Copyright (c) 2014, Stephen Robinson, MQTT-ESP - Tuan PM <tuanpm at live dot com> is licensed under Apache License 2.0.
* :component:`BLE Mesh <bt/esp_ble_mesh>` is adapted from Zephyr Project, Copyright (c) 2017-2018 Intel Corporation and licensed under Apache License 2.0
* `mynewt-nimble`_ Apache Mynewt NimBLE, Copyright 2015-2018, The Apache Software Foundation, is licensed under Apache License 2.0.
* `mynewt-nimble`_ Apache Mynewt NimBLE, Copyright 2015-2018, The Apache Software Foundation, is licensed under Apache License 2.0 as described in :component_file:`LICENSE file<bt/host/nimble/nimble/LICENSE>`.
Build Tools

View file

@ -141,7 +141,7 @@ There is couple of tips, how you can make your documentation even better and mor
Linking Examples
----------------
When linking to examples on GitHub do not use absolute / hardcoded URLs. Instead, use docutils custom roles that will generate links for you. These auto-generated links point to the tree or blob for the git commit ID (or tag) of the repository. This is needed to ensure that links do not get broken when files in master branch are moved around or deleted.
When linking to examples on GitHub do not use absolute / hardcoded URLs. Instead, use docutils custom roles that will generate links for you. These auto-generated links point to the tree or blob for the git commit ID (or tag) of the repository. This is needed to ensure that links do not get broken when files in master branch are moved around or deleted. The roles will transparently handle files that are located in submodules and will link to the submodule's repository with the correct commit ID.
The following roles are provided:

View file

@ -6,6 +6,7 @@ import re
import os
import subprocess
from docutils import nodes
from collections import namedtuple
def get_github_rev():
@ -21,6 +22,30 @@ def get_github_rev():
return path
# Creates a dict of all submodules with the format {submodule_path : (url relative to git root), commit)}
def get_submodules():
git_root = subprocess.check_output(['git', 'rev-parse', '--show-toplevel']).strip().decode('utf-8')
gitmodules_file = os.path.join(git_root, '.gitmodules')
submodules = subprocess.check_output(['git', 'submodule', 'status']).strip().decode('utf-8').split('\n')
submodule_dict = {}
Submodule = namedtuple('Submodule', 'url rev')
for sub in submodules:
sub_info = sub.lstrip().split(' ')
# Get short hash, 7 digits
rev = sub_info[0].lstrip('-')[0:7]
path = sub_info[1].lstrip('./')
config_key_arg = "submodule.{}.url".format(path)
rel_url = subprocess.check_output(['git', 'config', '--file', gitmodules_file, '--get', config_key_arg]).decode('utf-8').lstrip('./').rstrip('\n')
submodule_dict[path] = Submodule(rel_url, rev)
return submodule_dict
def url_join(*url_parts):
""" Make a URL out of multiple components, assume first part is the https:// part and
anything else is a path component """
@ -31,34 +56,43 @@ def url_join(*url_parts):
def setup(app):
rev = get_github_rev()
submods = get_submodules()
# links to files or folders on the GitHub
app.add_role('idf', github_link('tree', rev, '/', app.config))
app.add_role('idf_file', github_link('blob', rev, '/', app.config))
app.add_role('idf_raw', github_link('raw', rev, '/', app.config))
app.add_role('component', github_link('tree', rev, '/components/', app.config))
app.add_role('component_file', github_link('blob', rev, '/components/', app.config))
app.add_role('component_raw', github_link('raw', rev, '/components/', app.config))
app.add_role('example', github_link('tree', rev, '/examples/', app.config))
app.add_role('example_file', github_link('blob', rev, '/examples/', app.config))
app.add_role('example_raw', github_link('raw', rev, '/examples/', app.config))
app.add_role('idf', github_link('tree', rev, submods, '/', app.config))
app.add_role('idf_file', github_link('blob', rev, submods, '/', app.config))
app.add_role('idf_raw', github_link('raw', rev, submods, '/', app.config))
app.add_role('component', github_link('tree', rev, submods, '/components/', app.config))
app.add_role('component_file', github_link('blob', rev, submods, '/components/', app.config))
app.add_role('component_raw', github_link('raw', rev, submods, '/components/', app.config))
app.add_role('example', github_link('tree', rev, submods, '/examples/', app.config))
app.add_role('example_file', github_link('blob', rev, submods, '/examples/', app.config))
app.add_role('example_raw', github_link('raw', rev, submods, '/examples/', app.config))
# link to the current documentation file in specific language version
app.add_role('link_to_translation', link_to_translation(app.config))
return {'parallel_read_safe': True, 'parallel_write_safe': True, 'version': '0.3'}
return {'parallel_read_safe': True, 'parallel_write_safe': True, 'version': '0.4'}
def github_link(link_type, rev, root_path, app_config):
def github_link(link_type, idf_rev, submods, root_path, app_config):
def role(name, rawtext, text, lineno, inliner, options={}, content=[]):
msgs = []
BASE_URL = 'https://github.com/'
IDF_REPO = "espressif/esp-idf"
def warning(msg):
system_msg = inliner.reporter.warning(msg)
system_msg.line = lineno
msgs.append(system_msg)
BASE_URL = 'https://github.com/espressif/esp-idf'
# Redirects to submodule repo if path is a submodule, else default to IDF repo
def redirect_submodule(path, submods, rev):
for key, value in submods.items():
if path.lstrip('/').startswith(key):
return value.url.replace('.git', ''), value.rev, re.sub('^/{}/'.format(key), '', path)
return IDF_REPO, rev, path
# search for a named link (:label<path>) with descriptive label vs a plain URL
m = re.search(r'(.*)\s*<(.*)>', text)
@ -71,8 +105,11 @@ def github_link(link_type, rev, root_path, app_config):
rel_path = root_path + link
abs_path = os.path.join(app_config.idf_path, rel_path.lstrip('/'))
repo, repo_rev, rel_path = redirect_submodule(rel_path, submods, idf_rev)
line_no = None
url = url_join(BASE_URL, link_type, rev, rel_path)
url = url_join(BASE_URL, repo, link_type, repo_rev, rel_path)
if '#L' in abs_path:
# drop any URL line number from the file, line numbers take the form #Lnnn or #Lnnn-Lnnn for a range