Merge branch 'feature/readme_checklinks' into 'master'

CI: add script for checking README links

Closes IDF-1846

See merge request espressif/esp-idf!9188
This commit is contained in:
Anton Maklakov 2020-07-08 12:03:14 +08:00
commit 9496f9bf46
19 changed files with 269 additions and 101 deletions

View file

@ -6,11 +6,11 @@ ESP-IDF Blufi demo
This is the demo for bluetooth config wifi connection to ap.
To test this demo, you need to prepare a mobile phone with blufi application installed. You can download the blufi application from [Android version](https://github.com/EspressifApp/EspBlufi) and [iOS version](https://itunes.apple.com/cn/app/espblufi/id1450614082?mt=8).
To test this demo, you need to prepare a mobile phone with blufi application installed. You can download the blufi application from [Android version](https://github.com/EspressifApp/EspBlufi) and [iOS version](https://itunes.apple.com/cn/app/espblufi/id1450614082?mt=8).
Blufi is completely open source, here is the download link:
* [blufi source code](https://github.com/espressif/esp-idf/tree/master/examples/bluetooth/blufi)
* [blufi source code](https://github.com/espressif/esp-idf/tree/master/examples/bluetooth/bluedroid/ble/blufi)
* [BluFi protocol](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/blufi.html?highlight=blufi#the-frame-formats-defined-in-blufi)

View file

@ -1,6 +1,5 @@
================= =====
Supported Targets ESP32
================= =====
| Supported Targets | ESP32 |
| ----------------- | ----- |
ESP-IDF BT-INQUIRY demo
======================
@ -9,10 +8,10 @@ Demo of Classic Bluetooth Device and Service Discovery
This is the demo for user to use ESP_APIs to perform inquiry to search for a target device and then performs service search via SDP.
Options choose step:
1. idf.py menuconfig.
2. enter menuconfig "Component config", choose "Bluetooth"
3. enter menu Bluetooth, choose "Classic Bluetooth" and do not choose "Release DRAM from Classic BT controller"
4. choose your options.
Options choose step:`
1. `idf.py menuconfig`
2. enter menuconfig `Component config`, choose `Bluetooth`
3. enter menu Bluetooth, choose `Classic Bluetooth` and do not choose `Release DRAM from Classic BT controller`
4. choose your options.
After the program started, the device will start inquiry to search for a device with Major device type "PHONE" in the Class of Device Field. Then it will cancel the inquiry and started to perform service discovering on this remote device.

View file

@ -0,0 +1,19 @@
| Supported Targets | ESP32 |
| ----------------- | ----- |
ESP-IDF BT-SPP-ACCEPTOR demo
======================
Demo of SPP acceptor role
This is the demo for user to use ESP_APIs to create a SPP acceptor.
Options choose step:
1. `idf.py menuconfig`
2. enter menuconfig `Component config`, choose `Bluetooth`
3. enter menu Bluetooth, choose `Classic Bluetooth" and "SPP Profile`
4. choose your options.
Then set `SPP_SHOW_MODE` as `SPP_SHOW_DATA` or `SPP_SHOW_SPEED` in code(should be same with bt_spp_initator).
After the program started, bt_spp_initator will connect it and send data.

View file

@ -1,20 +0,0 @@
================= =====
Supported Targets ESP32
================= =====
ESP-IDF BT-SPP-ACCEPTOR demo
======================
Demo of SPP acceptor role
This is the demo for user to use ESP_APIs to create a SPP acceptor.
Options choose step:
1. idf.py menuconfig.
2. enter menuconfig "Component config", choose "Bluetooth"
3. enter menu Bluetooth, choose "Classic Bluetooth" and "SPP Profile"
4. choose your options.
Then set SPP_SHOW_MODE as SPP_SHOW_DATA or SPP_SHOW_SPEED in code(should be same with bt_spp_initator).
After the program started, bt_spp_initator will connect it and send data.

View file

@ -0,0 +1,20 @@
| Supported Targets | ESP32 |
| ----------------- | ----- |
ESP-IDF BT-SPP-INITATOR demo
======================
Demo of SPP initator role
This is the demo for user to use ESP_APIs to create a SPP initator.
Options choose step:
1. `idf.py menuconfig`
2. enter menuconfig `Component config`, choose `Bluetooth`
3. enter menu Bluetooth, choose `Classic Bluetooth` and `SPP Profile`
4. choose your options.
Then set `SPP_SHOW_MODE` as `SPP_SHOW_DATA` or `SPP_SHOW_SPEED` in code(should be same with bt_spp_acceptor).
After the program started, It will connect to bt_spp_acceptor and send data.

View file

@ -1,20 +0,0 @@
================= =====
Supported Targets ESP32
================= =====
ESP-IDF BT-SPP-INITATOR demo
======================
Demo of SPP initator role
This is the demo for user to use ESP_APIs to create a SPP initator.
Options choose step:
1. idf.py menuconfig.
2. enter menuconfig "Component config", choose "Bluetooth"
3. enter menu Bluetooth, choose "Classic Bluetooth" and "SPP Profile"
4. choose your options.
Then set SPP_SHOW_MODE as SPP_SHOW_DATA or SPP_SHOW_SPEED in code(should be same with bt_spp_acceptor).
After the program started, It will connect to bt_spp_acceptor and send data.

View file

@ -0,0 +1,17 @@
| Supported Targets | ESP32 |
| ----------------- | ----- |
ESP-IDF BT-SPP-VFS-ACCEPTOR demo
======================
Demo of SPP acceptor role
This is the demo for user to use ESP_APIs to create a SPP acceptor.
Options choose step:
1. `idf.py menuconfig`
2. enter menuconfig `Component config`, choose `Bluetooth`
3. enter menu Bluetooth, choose `Classic Bluetooth` and `SPP Profile`
4. choose your options.
After the program started, bt_spp_vfs_initator will connect it and send data.

View file

@ -1,18 +0,0 @@
================= =====
Supported Targets ESP32
================= =====
ESP-IDF BT-SPP-VFS-ACCEPTOR demo
======================
Demo of SPP acceptor role
This is the demo for user to use ESP_APIs to create a SPP acceptor.
Options choose step:
1. idf.py menuconfig.
2. enter menuconfig "Component config", choose "Bluetooth"
3. enter menu Bluetooth, choose "Classic Bluetooth" and "SPP Profile"
4. choose your options.
After the program started, bt_spp_vfs_initator will connect it and send data.

View file

@ -0,0 +1,17 @@
| Supported Targets | ESP32 |
| ----------------- | ----- |
ESP-IDF BT-SPP-VFS-INITATOR demo
======================
Demo of SPP initator role
This is the demo for user to use ESP_APIs to create a SPP initator.
Options choose step:
1. `idf.py menuconfig`
2. enter menuconfig `Component config`, choose `Bluetooth`
3. enter menu Bluetooth, choose `Classic Bluetooth` and `SPP Profile`
4. choose your options.
After the program started, It will connect to bt_spp_vfs_acceptor and send data.

View file

@ -1,18 +0,0 @@
================= =====
Supported Targets ESP32
================= =====
ESP-IDF BT-SPP-VFS-INITATOR demo
======================
Demo of SPP initator role
This is the demo for user to use ESP_APIs to create a SPP initator.
Options choose step:
1. idf.py menuconfig.
2. enter menuconfig "Component config", choose "Bluetooth"
3. enter menu Bluetooth, choose "Classic Bluetooth" and "SPP Profile"
4. choose your options.
After the program started, It will connect to bt_spp_vfs_acceptor and send data.

View file

@ -8,10 +8,10 @@ This example demonstrates the coexistence of gattc and gatts.
This example creates a GATT service and starts ADV. The ADV name is `ESP_GATTS_DEMO`, then waits to be connected. At the same time, a gatt client is created, the ADV name is `ESP_GATTS_DEMO`, the device is connected, and the data is exchanged. If the device is not found within 120 seconds, the example will stop scanning.
ESP-IDF also allows users to create a GATT service via an attribute table, rather than add attributes one by one. And it is recommended for users to use. For more information about this method, please refer to [gatt_server_service_table_demo](../gatt_server_service_table).
ESP-IDF also allows users to create a GATT service via an attribute table, rather than add attributes one by one. And it is recommended for users to use. For more information about this method, please refer to [gatt_server_service_table_demo](../../ble/gatt_server_service_table).
To test this example, you can run the [gatt_client_demo](../gatt_client), which can scan for and connect to this example automatically, and run [gatt_server_demo](../gatt_server), Waiting to be connected. They will start exchanging data once the GATT client has enabled the notification function of the GATT server.
To test this example, you can run the [gatt_client_demo](../../ble/gatt_client), which can scan for and connect to this example automatically, and run [gatt_server_demo](../../ble/gatt_server), Waiting to be connected. They will start exchanging data once the GATT client has enabled the notification function of the GATT server.
Please check the [tutorial](tutorial/Gatt_Server_Example_Walkthrough.md) for more information about the gatts part of this example.
Please check the [tutorial](tutorial/Gatt_Client_Example_Walkthrough.md) for more information about the gattc part of this example.
Please check the [tutorial](../../ble/gatt_server/tutorial/Gatt_Server_Example_Walkthrough.md) for more information about the gatts part of this example.
Please check the [tutorial](../../ble/gatt_client/tutorial/Gatt_Client_Example_Walkthrough.md) for more information about the gattc part of this example.

View file

@ -20,4 +20,4 @@ WiFi的Iperf协议使用方法请参考[WiFi Iperf README](../../../wifi/iperf/R
- Generic OnOff Server Model此Model通过暴露自己的OnOff State从而实现LED 灯的开关功能
- Generic OnOff Client Model 使用此Model可以实现开关功能控制别的node 的LED灯的开关
- Fast Provision Server Model 此Model是为了进行快速配网而实现的自定义Model通过此Model当节点作为临时provisioner进行配网成功后需要将生成的Element地址通过此Model进行传给provisioner
- Fast Provision Client Model此Model和Fast Provision Server Model是配合使用的
- Fast Provision Client Model此Model和Fast Provision Server Model是配合使用的

View file

@ -14,7 +14,7 @@ When PPP connection has been established, the IP packet flow from application si
### Hardware Required
To run this example, you need an ESP32 dev board (e.g. ESP32-WROVER Kit) or ESP32 core board (e.g. ESP32-DevKitC).
For test purpose, you also need a cellular modem module. Here we take the [SIM800L](http://www.simcom.com/product/showproduct.php?lang=en&id=277) and [BG96](https://www.quectel.com/product/bg96.htm) as an example.
For test purpose, you also need a cellular modem module. Here we take the [SIM800L](https://www.simcom.com/product/SIM800.html) and [BG96](https://www.quectel.com/product/bg96.htm) as an example.
You can also try other modules as long as they embedded PPP protocol.
**Note:** Since SIM800L only support **2G** which will **not** work in some countries. And also keep in mind that in some other countries it will stop working soon (many remaining 2G networks will be switched off in the next 2-3 years). So you should **check with your local providers for further details** if you try this example with any 2G modules.

View file

@ -89,13 +89,13 @@ I (6965) tcp_client_multiple: HTTP/1.1 200 OK
Date: Thu, 23 Apr 2020 07:02:58 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;
Content-Type: text/html;
I (6965) tcp_client_multiple: "example_connect: sta" Received Data 127 bytes
I (6985) tcp_client_multiple: HTTP/1.1 200 OK
Date: Thu, 23 Apr 2020 07:02:58 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;
Content-Type: text/html;
I (7675) tcp_client_multiple: "example_connect: eth" Socket created
I (7675) tcp_client_multiple: "example_connect: eth" Successfully connected
I (7695) tcp_client_multiple: "example_connect: sta" Socket created
@ -105,13 +105,13 @@ I (7735) tcp_client_multiple: HTTP/1.1 200 OK
Date: Thu, 23 Apr 2020 07:02:59 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;
Content-Type: text/html;
I (7955) tcp_client_multiple: "example_connect: sta" Received Data 127 bytes
I (7955) tcp_client_multiple: HTTP/1.1 200 OK
Date: Thu, 23 Apr 2020 07:02:59 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;
Content-Type: text/html;
I (8445) tcp_client_multiple: "example_connect: eth" Socket created
I (8445) tcp_client_multiple: "example_connect: eth" Successfully connected
I (8505) tcp_client_multiple: "example_connect: eth" Received Data 127 bytes
@ -119,7 +119,7 @@ I (8505) tcp_client_multiple: HTTP/1.1 200 OK
Date: Thu, 23 Apr 2020 07:03:00 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;
Content-Type: text/html;
I (8675) tcp_client_multiple: "example_connect: sta" Socket created
```
@ -127,7 +127,7 @@ I (8675) tcp_client_multiple: "example_connect: sta" Socket created
* When connecting using Ethernet, please consult troubleshooting described in [Ethernet common readme](../../../ethernet/README.md)
or [Ethernet documentation](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_eth.html).
If using Ethernet for the first time, it is recommended to start with the [Ethernet example readme](../../../ethernet/basic/README.md), which contains instructions for connecting and configuring the PHY.
If using Ethernet for the first time, it is recommended to start with the [Ethernet example readme](../../../ethernet/basic/README.md), which contains instructions for connecting and configuring the PHY.
Once Ethernet example obtains IP address successfully, proceed to this example.
* When connecting using Wi-Fi, please refer to the WiFi examples in [examples/wifi/getting_started/](../wifi/getting_started).
* When connecting using Wi-Fi, please refer to the WiFi examples in [examples/wifi/getting_started/](../../../wifi/getting_started).

View file

@ -1,6 +1,6 @@
# OTA Tool Example
This example demonstrates common operations the OTA tool [otatool.py](../../../components/app_update/otatool.py) allows the user to perform:
This example demonstrates common operations the OTA tool [otatool.py](../../../../components/app_update/otatool.py) allows the user to perform:
- reading, writing and erasing OTA partitions,
- switching boot partitions, and
@ -44,7 +44,7 @@ Shell script:
./otatool_example.sh
```
The script searches for valid target devices connected to the host and performs the operations on the first one it finds. This could present problems if there
The script searches for valid target devices connected to the host and performs the operations on the first one it finds. This could present problems if there
are multiple viable target devices attached to the host. To perform the operations on a specific device, specify the port it is attached to during script invocation ("/dev/ttyUSB2" for example):
Python script:

161
tools/ci/check_readme_links.py Executable file
View file

@ -0,0 +1,161 @@
#!/usr/bin/env python
#
# Checks that all links in the readme markdown files are valid
#
# Copyright 2020 Espressif Systems (Shanghai) PTE LTD
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import os
import re
import os.path
import urllib.request
import urllib.error
import concurrent.futures
import argparse
from pathlib import Path
from collections import namedtuple, defaultdict
EXCLUDE_DOCS_LIST = ['examples/peripherals/secure_element/atecc608_ecdsa/components/esp-cryptoauthlib/cryptoauthlib/**']
# The apple apps links are not accessible from the company network for some reason
EXCLUDE_URL_LIST = ['https://apps.apple.com/in/app/esp-ble-provisioning/id1473590141', 'https://apps.apple.com/in/app/esp-softap-provisioning/id1474040630']
Link = namedtuple('Link', ['file', 'url'])
class ReadmeLinkError(Exception):
def __init__(self, file, url):
self.file = file
self.url = url
class RelativeLinkError(ReadmeLinkError):
def __str__(self):
return "Relative link error, file - {} not found, linked from {}".format(self.url, self.file)
class UrlLinkError(ReadmeLinkError):
def __init__(self, file, url, error_code):
self.error_code = error_code
super().__init__(file, url)
def __str__(self):
files = [str(f) for f in self.file]
return "URL error, url - {} in files - {} is not accessible, request returned {}".format(self.url, ", ".join(files), self.error_code)
# we do not want a failed test just due to bad network conditions, for non 404 errors we simply print a warning
def check_url(url, files, timeout):
try:
with urllib.request.urlopen(url, timeout=timeout):
return
except urllib.error.HTTPError as e:
if e.code == 404:
raise UrlLinkError(files, url, str(e))
else:
print("Unable to access {}, err = {}".format(url, str(e)))
except Exception as e:
print("Unable to access {}, err = {}".format(url, str(e)))
def check_web_links(web_links):
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
errors = []
future_to_url = {executor.submit(check_url, url, files, timeout=30): (url, files) for url, files in web_links.items()}
for future in concurrent.futures.as_completed(future_to_url):
try:
future.result()
except UrlLinkError as e:
errors.append(e)
return errors
def check_file_links(file_links):
errors = []
for link in file_links:
link_path = link.file.parent / link.url
if not Path.exists(link_path):
errors.append(RelativeLinkError(link.file, link.url))
print("Found {} errors with relative links".format(len(errors)))
return errors
def get_md_links(folder):
MD_LINK_RE = r"\[.+?\]\((.+?)(#.+)?\)"
idf_path = Path(os.getenv('IDF_PATH'))
links = []
for path in (idf_path / folder).rglob('*.md'):
if any([path.relative_to(idf_path).match(exclude_doc) for exclude_doc in EXCLUDE_DOCS_LIST]):
print("{} - excluded".format(path))
continue
with path.open(encoding='utf8') as f:
content = f.read()
for url in re.findall(MD_LINK_RE, content):
link = Link(path, url[0].lstrip())
# Ignore "local" links
if not link.url.startswith('#'):
links.append(link)
return links
def check_readme_links(args):
links = get_md_links('examples')
print("Found {} links".format(len(links)))
errors = []
web_links = defaultdict(list)
file_links = []
# Sort links into file and web links
for link in links:
if link.url.startswith('http'):
web_links[link.url].append(link.file)
else:
file_links.append(link)
for url in EXCLUDE_URL_LIST:
del web_links[url]
errors.extend(check_file_links(file_links))
if not args.skip_weburl:
errors.extend(check_web_links(web_links))
print("Found {} errors:".format(len(errors)))
for e in errors:
print(e)
if errors:
raise e
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='check_readme_links.py: Checks for dead links in example READMEs', prog='check_readme_links.py')
parser.add_argument("--skip-weburl", "-w", action='store_true', help="Skip checking of web URLs, only check links to local files")
args = parser.parse_args()
check_readme_links(args)

View file

@ -312,3 +312,4 @@ test_mkdfu:
script:
- cd ${IDF_PATH}/tools/test_mkdfu
- ${IDF_PATH}/tools/ci/multirun_with_pyenv.sh ./test_mkdfu.py

View file

@ -221,3 +221,12 @@ scan_tests:
- python $CI_SCAN_TESTS_PY example_test -b make $EXAMPLE_TEST_DIR --exclude examples/build_system/idf_as_lib -c $TEST_CONFIG_FILE -o $EXAMPLE_TEST_OUTPUT_DIR
- python $CI_SCAN_TESTS_PY example_test -b cmake $EXAMPLE_TEST_DIR --exclude examples/build_system/idf_as_lib -c $TEST_CONFIG_FILE -o $EXAMPLE_TEST_OUTPUT_DIR
- python $CI_SCAN_TESTS_PY test_apps $TEST_APPS_TEST_DIR -c $TEST_CONFIG_FILE -o $TEST_APPS_OUTPUT_DIR
check_readme_links:
extends: .check_job_template
tags: [ "amd64", "deploy", "internet" ]
allow_failure: true
variables:
PYTHON_VER: 3
script:
- python ${IDF_PATH}/tools/ci/check_readme_links.py

View file

@ -44,6 +44,7 @@ tools/ci/check_deprecated_kconfigs.py
tools/ci/check_examples_cmake_make.sh
tools/ci/check_examples_rom_header.sh
tools/ci/check_idf_version.sh
tools/ci/check_readme_links.py
tools/ci/check_rom_apis.sh
tools/ci/check_ut_cmake_make.sh
tools/ci/checkout_project_ref.py