We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).
Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.
In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
We had some problems:
simultaneous compiling of kconfig in the same tree;
attempt to use menuconfig in some examples and ut in batch mode (w/o interactive console)
Also increase debug abilities in CI:
force non-interactive building;
add variable DEBUG_SHELL to toggle verbosity of scripts
tools/gen_esp_err_to_name.py generates
components/esp32/esp_err_to_name.c for lookup of the error codes from
all of the IDF project and easily identify all codes in one place
* Fixes issue in !1250 when IDF_PATH changes on build runner
* Allows checking document warnings before CI run
* Will be robust to line number changes in document files
The CI uses the mirror-submodule-update.sh for non-permanent reassignment
of 3rdparty sub-modules to local mirrors (only for 'non-master' branches).
List of the mirrors in mirror-list.txt
You can use the script locally to speed up data retrieval on network problems
check_submodule_sync job uses 'GIT_STRATEGY: clone' to check the availability of public sources
CI: auto generate configs for test jobs
Currently CI test jobs are static configured by several pre-generated config files.
This approach have several disadvantages:
1. not flexible to select test cases to run, which is important feature of @bot
2. difficult to update test as we need to pre-generate quite a lot files
3. need to maintain extra config files in IDF
4. not flexible to support new test apps or chips, can't use some new features of test bench
Therefore we'll add assign_test stage between build and test, to generate configs for test jobs.
See merge request !738
IDF built app download config may change with some modification.
save the download configs for built APP in build stage.
then we can use the correct download config in test stage.
wear levelling
This MR adds wear levelling component. It presents an interface similar to the interface of spi_flash and esp_partition. Inside, it stores data inside a partition to in a way that reduces worst case number of erase cycles for any given sector.
Also included are APIs similar to the ones provided for SDMMC to mount FAT filesystem on top of the wear levelling partition. A simple example shows how this API can be used.
Ref TW10338.
See merge request !567
CI support for testing & deploying release tags & branches
For CI testing & deployment of release branches:
* Test release tags as well as branches
* Swap gitlab submodule URLs to github when testing release tags or branches.
These changes are already pushed to the release/v2.0 branch.
See merge request !604
CI: support running CI test on release branches
Patches will be cherry-picked to release branches after it's released. Therefore we need to run test jobs with the same strategy as master.
See merge request !551
Previous design was put test env config on local runners. It's not easy
to manage as test runners count growing. Now we'll put config files for
test runners to a Gitlab repository. Test runners will get its config
from Gitlab every time before running.
ci: deploy code and docs for tags
Currently deploy happens only for the master branch.
This change is needed to support tags in release branches.
See merge request !458
docs: use custom roles to generate GitHub links
This change replaces direct links to GitHub master branch with
auto-generated links using docutils custom roles.
These auto-generated links point to the tree or blob for the git commit
ID (or tag) of the repository. This is needed to ensure that links don’t
become broken when files in master branch are moved around or deleted.
The following roles are introduced:
```
- :idf:`path` - points to directory inside ESP-IDF
- :idf_blob:`path` - points to file inside ESP-IDF
- :idf_raw:`path` - points to raw view of the file inside ESP-IDF
- :component:`path` - points to directory inside ESP-IDF components dir
- :component_blob:`path` - points to file inside ESP-IDF components dir
- :component_raw:`path` - points to raw view of the file inside ESP-IDF
components dir
- :example:`path` - points to directory inside ESP-IDF examples dir
- :example_blob:`path` - points to file inside ESP-IDF examples dir
- :example_raw:`path` - points to raw view of the file inside ESP-IDF
examples dir
```
A check is added to the CI build script, which searches RST files for
presence of hard-coded links (identified by tree/master, blob/master,
or raw/master part of the URL).
This check can be run manually: cd docs && make gh-linkcheck
Additionally, Sphinx linkcheck build type is used to create new CI test,
which check for broken links. This test has to be triggered explicitly,
because including it in normal build process (when the commit is not yet
deployed to Github) will not work. It can be triggered in a regular
fashion using a combination of cron and Curl, similar to stress tests.
See merge request !455
This change replaces direct links to GitHub master branch with
auto-generated links using docutils custom roles.
These auto-generated links point to the tree or blob for the git commit
ID (or tag) of the repository. This is needed to ensure that links don’t
become broken when files in master branch are moved around or deleted.
The following roles are introduced:
- :idf:`path` - points to directory inside ESP-IDF
- :idf_blob:`path` - points to file inside ESP-IDF
- :idf_raw:`path` - points to raw view of the file inside ESP-IDF
- :component:`path` - points to directory inside ESP-IDF components dir
- :component_blob:`path` - points to file inside ESP-IDF components dir
- :component_raw:`path` - points to raw view of the file inside ESP-IDF
components dir
- :example:`path` - points to directory inside ESP-IDF examples dir
- :example_blob:`path` - points to file inside ESP-IDF examples dir
- :example_raw:`path` - points to raw view of the file inside ESP-IDF
examples dir
A check is added to the CI build script, which searches RST files for
presence of hard-coded links (identified by tree/master, blob/master,
or raw/master part of the URL).
This check can be run manually: cd docs && make gh-linkcheck
Additionally, Sphinx linkcheck build type is used to create new CI test,
which check for broken links. This test has to be triggered explicitly,
because including it in normal build process (when the commit is not yet
deployed to Github) will not work. It can be triggered in a regular
fashion using a combination of cron and Curl, similar to stress tests.
Add python script that parses list of unit test cases for CI from component test folder
Modify .gitlab-ci.yml to run this script as part of build unit tests stage
1. config git user name before commit
2. continue committing test result for failed jobs
3. update test result repository path
4. change escape key word in branch name, use '___' to escape key word '/' in report file name
system_* APIs cleanup
This changeset is mostly about cleaning up `system_*` APIs and moving them from libcore. It has dragged a few more changes along, however.
- `system_deep_sleep` deprecated, new name is `esp_deep_sleep`
- `system_efuse_read_mac` deprecated, new name is `esp_efuse_read_mac`
- `system_read/write_rtc_mem` removed (reason: we use linker to place code/data into RTC memory now)
- `system_get_sdk_version` deprecated. Returns "master" for now. Support for compile-time and runtime version detection will be done in a separate MR.
- `system_get_time` and variations have been deprecated. `gettimeofday` should be used instead.
- `system_restart` deprecated, new name is `esp_restart`. Rewritten code to work correctly from any of the cores.
- `system_restore` deprecated, `esp_wifi_restore` should be used instead.
- `system_get_rst_info` removed (it wasn't in a header file)
- RTC library has been updated to latest version
- Logging output from `libfoo.a` goes through `foo_printf` function now, where `foo ` is any of the library names. For libraries other than rtc and phy, this is a temporary measure, until we finish switching to esp_log library and clean up all uses of printf/ets_printf.
- ESP-IDF build will fail if any of the libraries have printf/ets_printf reference.
- Added `esp_random` function which gets values from HW RNG. I have added code which makes sure the value is not polled too often (allows at least 16 APB cycles). Also added a header file for HW RNG (just one register).
- Moved random number generation functions used by wpa library into wpa_supplicant component.
See merge request !234
rename nvs host test folder, modify .gitlab-ci.yml
remove unit-test-app build
re-format unit test files
remove extra newlines in project.mk
some refactoring for unit test part in project.mk
add build files of unit-test-app in gitignore
add README.md for unit test app
correct headings in README.md
remove files and make minor tweaks in unit test app
update .gitlab-ci.yml to use unit test app
delete unused lines in component_wrapper.mk
delete periph_i2s.h and lcd test
add text floating point in components/esp32/test/Kconfig
correct idf test build paths in .gitlab-ci.yml
support build multiple bin for SSC
We need to build multiple SSC bin with different feature configs to do test.
For example, we need different SSC bins to test socket API blocking/non-blocking usage with same test cases.
SSC has a same branch to support build all pre-configured bins using `./gen_misc_ng.sh`. It will copy all bins to `./SSC/ssc_bin/`.
This MR will change the artifacts path for `build_ssc` job and the `BIN_PATH` for test jobs using SSC.
See merge request !219
save logs as test_report artifacts
save all logs as test report artifacts.
Testers only need to download artifacts for test report to analysis test result.
See merge request !217
fix bug that deploy when test failed:
test report will be a single stage. The result of test report will be calculated from the result of all test jobs in test stage. So it will only deploy when all test job passed.
See merge request !140
test report will be a single stage. The result of test report will be calculated from the result of all test jobs in test stage. So it will only deploy when all test job passed.
1. test will be categorized by test level
2. add test level as attribute to test cases
3. will select TestCaseScript by the test cases added to CI (currently no test case uses test case script)
4. adding test level to test jobs
5. update .gitlab-ci.yml, each job need to set its test case file path
6. update .gitlab-ci.yml, test case path for test report is changed to idf_test
1. test report job should be put to deploy stage, otherwise it can't get logs from test stage
2. allow test fail so that test report job will be executed for failed test
1. night jobs should exit without error if not triggered
2. remove get wifi connect status cases from IDF
3. use Env tag to check if test environment is special
1. add night job define (need to set variable in trigger to run night jobs)
2. move auto generated part to the end of file
3. add auto generated CI jobs
