Commit graph

1180 commits

Author SHA1 Message Date
Nachiket Kukade 1fc54cfad8 Increase example cmake parallel jobs number 2020-05-06 10:21:29 +05:30
Roland Dobai 7f4b651434 tools: Make Unicode from subprocess result in the CMake convert script
Fixes an issue with Python 3 in MSYS where it fails while trying to join
paths where one part is Unicode (default string on Python3) and the
second part are bytes (returned by the subprocess call).

Closes https://github.com/espressif/esp-idf/issues/5189
2020-04-28 15:22:27 +02:00
Roland Dobai c4af9d3e24 CI: Add EOL to sdkconfig.defaults before adding sdkconfig.ci 2020-04-23 16:54:33 +02:00
Roland Dobai 2d13bfb8fa CI: Fix partition table detection if "partition" is in the project name 2020-04-22 10:05:18 +02:00
Mahavir Jain 2cc9d28a35 Merge branch 'fix/wifi_provisioning_uuid_v4.0' into 'release/v4.0'
Wifi_provisioning: Change service_uuid to non standard 128 bit UUID and add retries in test script (backport v4.0)

See merge request espressif/esp-idf!8163
2020-04-01 13:04:26 +08:00
Angus Gratton a3f3c7bdc3 Merge branch 'bugfix/monitor_encrypted_target_v4.0' into 'release/v4.0'
tools: Fix flashing encrypted binaries from IDF Monitor (v4.0)

See merge request espressif/esp-idf!8128
2020-03-31 13:11:02 +08:00
Angus Gratton 0b822a9ec7 Merge branch 'bugfix/kconfiglib_open_encoding_v4.0' into 'release/v4.0'
tools: Avoid Unicode error in kconfiglib while opening files

See merge request espressif/esp-idf!8082
2020-03-31 11:52:16 +08:00
Prasad Alatkar 1e2ca78aac Wifi_provisioning_ble: Change service_uuid to non standard 128 bit UUID (v4.0)
- Modifies UUIDs in scheme_ble, manager example and esp_prov tool to custom 128
  bit UUID
2020-03-30 11:22:32 +05:30
Roland Dobai 13f4656d2a tools: Fix flashing encrypted binaries from IDF Monitor 2020-03-27 10:31:37 +01:00
Roland Dobai e9fdc4c7b2 tools: Avoid Unicode error in kconfiglib while opening files
This fix is for IDF v4.0 and earlier versions. The upstream kconfiglib
included since IDF v4.1 doesn't have this issue.

Closes https://github.com/espressif/esp-idf/issues/4977
2020-03-23 11:36:10 +01:00
Renz Christian Bagaporo 35839232c8 cmake: utility to create a failing target 2020-03-22 21:09:23 +08:00
Renz Bagaporo 7663928e49 ci: add test for partition_table target 2020-03-22 21:04:13 +08:00
Shubham Kulkarni 8785ac4c2a Bugfix for failing OTA example
example_test.py is added to test advanced_https_ota_example and native ota_example.

Closes https://github.com/espressif/esp-idf/issues/4394
2020-03-18 08:19:56 +00:00
He Yin Ling 0ad0507f88 ci: add find job id argument job status:
if we retry failed job, we will have 2 jobs with same name in one pipeline. we need to use status to find the retried job instead of failed job.
2020-03-12 09:59:31 +08:00
Michael (XIAO Xufeng) 1b08b7c5fb Merge branch 'bugfix/spiflash_read_psram_v4.0' into 'release/v4.0'
spi_flash: Fix over-allocation and OOM crash when reading from SPI flash to PSRAM buffers (v4.0)

See merge request espressif/esp-idf!7878
2020-03-09 11:10:52 +08:00
Angus Gratton c38c3ff3f0 spi_flash: Remove 16KB free internal heap limit for esp_flash_read() into PSRAM
Allocation of the temporary internal buffer will now repeat until a small enough buffer can be
allocated, and only fail if less than a 256 byte block of internal RAM is free.

Adds unit test for the same, and generic test utility for creating memory pressure.
2020-03-09 09:59:43 +11:00
Angus Gratton 3d0ecfdc70 cmake: Add warnings that convert_to_cmake.py doesn't calculate component requirements
Any component which requires another component will need this manually
added to its CMakeLists.txt file.
2020-03-05 17:06:38 +11:00
Angus Gratton e85c4a65a4 cmake: convert_to_cmake: Fix possible whitespace issues
As reported on forum:
https://esp32.com/viewtopic.php?f=2&t=13565&p=53476#p53453

split() with no arg will match any whitespace sequence not just
a single space, so takes care of case where two spaces are
inserted in the variable value.
2020-03-05 17:06:38 +11:00
Angus Gratton a2af518069 Merge branch 'bugfix/ci_weekend_test_update_v4.0' into 'release/v4.0'
ci: a build_system_test(_cmake) fix and some mqtt fixes (backport v4.0)

See merge request espressif/esp-idf!7773
2020-03-02 12:43:34 +08:00
Roland Dobai 2a4173e9a6 Docs: Encourage to used Python 3 2020-02-26 09:01:42 +01:00
David Cermak ee8ab79696 ci: fix test build system when using template repository from a
different branch then master
2020-02-25 18:32:22 +07:00
David Cermak a7f573c3a1 ci: fix weekend test confguration update per latest refactoring of
grouping tests
2020-02-25 18:32:11 +07:00
Angus Gratton a8fe083017 cmake: Set uninitialized variable warnings in ULP & bootloader subprojects
Fixes issue where PYTHON was not being expanded when running ulp_mapgen.py,
causing Windows launch setting to be used - reported here:
https://esp32.com/viewtopic.php?f=13&t=12640&p=50283#p50283

Closes https://github.com/espressif/esp-idf/issues/4785
2020-02-25 17:16:42 +11:00
Roland Dobai 6b6d9d7321 confgen.py: Escape special characters for cmake
Closes https://github.com/espressif/esp-idf/issues/4751
2020-02-18 09:55:53 +01:00
michael 8c148ede22 mcpwm: enable some unit tests again 2020-02-15 18:39:27 +08:00
Ivan Grokhotkov 45454cd477 ci: re-add the lost PEDANTIC_{C,CXX}FLAGS in CMake example builds 2020-02-13 19:01:43 +01:00
Ivan Grokhotkov 2309313103 Merge branch 'bugfix/ci_no_attempts_no_custom_toolchain_url_v4.0' into 'release/v4.0'
ci: no stage attempts variables, no custom toolchain url (backport v4.0)

See merge request espressif/esp-idf!7542
2020-02-12 17:55:17 +08:00
Ivan Grokhotkov 8cb3cc5154 Merge branch 'bugfix/idf_tools_virtualenv_v4.0' into 'release/v4.0'
idf_tools.py: fix install-python-env failure with virtualenv 20.0.0 (backport v4.0)

See merge request espressif/esp-idf!7553
2020-02-12 17:32:20 +08:00
Ivan Grokhotkov 7ccf6ebada idf_tools.py: fix install-python-env failure with virtualenv 20.0.0
--no-site-packages is the default behavior since virtualenv 1.7
(released in Nov 2011). This option was removed in virtualenv 20.0.
2020-02-11 09:46:03 +01:00
Anton Maklakov f368f69922 idf_tools.py: fix downloading for tools which have "on_request" options for some platforms.
'install' and 'download' options can be used for 'required' or 'all' tools
2020-02-10 20:10:26 +01:00
Anton Maklakov c5d20275dd ci: remove using IDF_XTENSA_TOOLCHAIN_URL because there are idf_tools.json and multi-target support 2020-02-10 18:38:23 +07:00
Ivan Grokhotkov c3961fdacb Merge branch 'bugfix/idf_monitor_usbser_sys_v4.0' into 'release/v4.0'
tools: work around usbser.sys bug in IDF monitor (v4.0)

See merge request espressif/esp-idf!7470
2020-02-10 17:34:00 +08:00
Ivan Grokhotkov c8215756fa Merge branch 'feature/idfpy_add_baud_parameter_to_monitor_v4.0' into 'release/v4.0'
idf.py: add monitor-baud option to monitor command (v4.0)

See merge request espressif/esp-idf!7461
2020-02-10 17:33:32 +08:00
Ivan Grokhotkov 97ef8fd057 Merge branch 'bugfix/idf_tools_v4.0' into 'release/v4.0'
idf_tools.py bug fixes (backport v4.0)

See merge request espressif/esp-idf!7479
2020-02-10 16:29:27 +08:00
Angus Gratton c2bce897b8 idf.py: Add environment variable to enable ccache by default 2020-01-30 12:14:54 +01:00
Ivan Grokhotkov 5140eea3d1 tools/docker: enable ccache by default 2020-01-30 12:10:31 +01:00
Ivan Grokhotkov 1b0473494f tools/docker: install CMake version provided in tools.json
Closes https://github.com/espressif/esp-idf/issues/4644
Closes IDFGH-2559
2020-01-30 12:10:31 +01:00
Ivan Grokhotkov 97242031e7 idf_tools.py: fix typo in __PYVENV_LAUNCHER__ env variable name
Fixes the workaround added in 9279b72a6.
2020-01-30 11:59:29 +01:00
Ivan Grokhotkov 5ab36dc2ec idf_tools.py: fix virtualenv issue for macOS with homebrew 2020-01-30 11:59:16 +01:00
Ivan Grokhotkov 09dd8ca3e9 idf_tools.py: when guessing IDF_PATH, set it in the enviroment
This allows the processes launched from idf_tools.py to use the value
of IDF_PATH. One such example is the installation of Python packages,
when requirements.txt uses IDF_PATH to refer to the esp-windows-curses
package.

Closes https://github.com/espressif/esp-idf/issues/4341
2020-01-30 11:57:51 +01:00
Rosa Elena Veiga Otero 02596a7efa tools: work around usbser.sys bug in idf_monitor
implemented workaround in idf_monitor for known usbser.sys bug which
causes changes in RTS signal to only be sent if DTR signal also changes.
Works by forcing a dummy change in DTR signal (re-assigning its current
value, as it has previously been done in the flashing tool)

Closes https://github.com/espressif/esp-idf/pull/4676
2020-01-29 10:36:43 +01:00
Sergei Silnov ead0e6da36 idf.py: add monitor-baud option to monitor command 2020-01-27 18:54:37 +01:00
Anton Maklakov ef34e6f6f7 ci: fix tags for internal deploy jobs 2020-01-10 18:33:11 +07:00
Shivani Tipnis b7a191a996 ci: Fix target variable default value v4.0 2020-01-07 20:29:06 +05:30
Renz Christian Bagaporo 989dac1e64 cmake: implement utility to import prebuilt libraries 2020-01-06 09:39:49 +08:00
Ivan Grokhotkov cb1d440cb8 Merge branch 'bugfix/fix_python3_errors_for_ttfw_v4.0' into 'release/v4.0'
test: fix python3 errors for ttfw (backport v4.0)

See merge request espressif/esp-idf!7146
2019-12-31 17:56:37 +08:00
Roland Dobai e8b454e802 CI: Adjust the test parameters of IDF Monitor 2019-12-30 14:28:30 +01:00
He Yin Ling dae4da07c3 CI: reduce number of unit test jobs 2019-12-30 10:59:47 +08:00
He Yin Ling ca3b2a8909 test: fix multiple devices cases fail on python3 2019-12-30 10:55:57 +08:00
He Yin Ling 95f0553f40 test: let unit test jobs use python3 2019-12-30 10:55:46 +08:00
He Yin Ling 09935510bd ttfw: fix incorrect relative import for python3 2019-12-30 10:55:32 +08:00
Angus Gratton 83813f830d Merge branch 'bugfix/menuconfig_confirm_v4.0' into 'release/v4.0'
Menuconfig: Don't ask to save configuration when nothing has changed (backport v4.0)

See merge request espressif/esp-idf!6813
2019-12-20 13:11:00 +08:00
Jiang Jiang Jian 482d2f06c3 Merge branch 'ci/artifact_expiry_v4.0' into 'release/v4.0'
ci: Keep build artifacts for 4 days (v4.0)

See merge request espressif/esp-idf!6994
2019-12-16 14:23:44 +08:00
Angus Gratton 3ba9eae86c Merge branch 'feature/ci_optimize_artifact_download_v4.0' into 'release/v4.0'
CI: optimize example build and artifact download (backport v4.0)

See merge request espressif/esp-idf!6940
2019-12-13 11:36:04 +08:00
Angus Gratton a8e01de0e4 ci: Keep build artifacts for 4 days 2019-12-12 06:30:13 +00:00
Angus Gratton 0104a24a36 Merge branch 'bugfix/server_down_message_v4.0' into 'release/v4.0'
tools: Produce user friendly error during install when the server is down (v4.0)

See merge request espressif/esp-idf!6958
2019-12-12 08:12:36 +08:00
Angus Gratton 30ac846112 Merge branch 'bugfix/cmake_secure_boot_v4.0' into 'release/v4.0'
secure boot: CMake bug fixes (v4.0)

See merge request espressif/esp-idf!6630
2019-12-12 06:30:37 +08:00
He Yin Ling 45a50ed701 test: fix incorrect error type in xunit result 2019-12-11 15:57:49 +11:00
He Yin Ling fd481d9cdd ci: fix fetch submodule error on python3 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov ec8e057e4c ci: remove dependencies from link_check job 2019-12-11 15:57:49 +11:00
He Yin Ling fbdedee4cb ci: reduce number of unit test cases 2019-12-11 15:57:49 +11:00
He Yin Ling 48786c1da0 CI: remove not exist file from executable-list.txt 2019-12-11 15:57:49 +11:00
He Yin Ling e9be102224 CI: build system do not check submodule for CI 2019-12-11 15:57:49 +11:00
He Yin Ling 3691ebc273 CI: modify fetch submodule method:
download archive for submodules instead of clone
2019-12-11 15:57:49 +11:00
He Yin Ling 31b9329d36 CI: raise import error instead of exit for ble tools 2019-12-11 15:57:49 +11:00
He Yin Ling d00acce7b2 CI: support only download artifacts by request:
use CI dependencies could waste a lot bandwidth for target test jobs, as
example binary artifacts are very large. Now we will parse required
artifacts first, then use API to download required files in artifacts.
2019-12-11 15:57:49 +11:00
He Yin Ling 6e05a79536 test: update example and unit tests with new import roles:
tiny_test_fw is a python package now. import it using normal way.
2019-12-11 15:57:49 +11:00
He Yin Ling 978bf09f83 CI: add utility gitlab_api 2019-12-11 15:57:49 +11:00
He Yin Ling 7a5d17e1b7 tiny-test-fw: move to tools/esp_python_packages:
make `tiny_test_fw` as a package and move to root path of idf python
packages
2019-12-11 15:57:49 +11:00
He Yin Ling d3e0301aee tiny-test-fw: only load module from the same file one time:
we should only load one module once.
if we load one module twice, python will regard the same object loaded in the first time and second time as different objects.
it will lead to strange errors like `isinstance(object, type_of_this_object)` return False
2019-12-11 15:57:49 +11:00
He Yin Ling 69c0e6243e unit-test-app: fix test script error when handling multiple unit test configs in one config file:
If we have multiple configs, we need to flash DUT with different binaries. But if we don't close DUT before apply new config, the old DUT will be reused, so new config name will not be applied.
2019-12-11 15:57:49 +11:00
He Yin Ling cffe7bf3a1 ci: use more flexible assign for unit test cases 2019-12-11 15:57:49 +11:00
He Yin Ling 1041bc800c unit-test-app: print config name in test script 2019-12-11 15:57:49 +11:00
He Yin Ling dd74a331c7 tiny-test-fw: print useful debug info when assign test:
* total requirements (number of required jobs vs tags)
* unused CI jobs
* missing jobs (number of missing jobs vs tags)
2019-12-11 15:57:49 +11:00
He Yin Ling 6ee7492512 ci: optimize unit test assignment:
Currently we use config and test function as filter when assign cases to one CI job. It's not necessary as the runner can run test with different configs / test functions. Now we will try to assign as many cases to a job as possible, to reduce the amount of jobs required.
2019-12-11 15:57:49 +11:00
Ivan Grokhotkov 3fcecdd12e ci: don't add build_examples_make as a target_test dependency 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov f3c8907cef ci: reimplement build_examples_cmake using {find,build}_apps.py 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov 7db756fbd9 tiny-test-fw: add build config and target options 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov 1d72766f1a tools: add build warnings checking script 2019-12-11 15:57:49 +11:00
Ivan Grokhotkov 5d03ae7428 tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.

The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
   to be built.
3. It filters out the apps and configurations which are not compatible
   with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
   format is a list of lines, each line a JSON string. In the future,
   the tool can be updated to output YAML files.

The lists of builds can be concatenated and processed with standard
command line tools, like sed.

The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.

These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).

Closes IDF-641
2019-12-11 15:57:49 +11:00
Roland Dobai bb7e52a4a8 tools: Produce user friendly error during install when the server is down
Closes https://github.com/espressif/esp-idf/issues/4329
2019-12-09 15:51:38 +01:00
Angus Gratton 4d3c650b49 Merge branch 'bugfix/macos_link_toomany_open_files_v4.0' into 'release/v4.0'
build system: explicitly disable LTO plugin to reduce the number of simultaneously open files (backport v4.0)

See merge request espressif/esp-idf!6867
2019-12-09 06:19:30 +08:00
Ivan Grokhotkov afeed3cc6f build system: explicitly disable LTO plugin
... to reduce the number of simultaneously open files at link time.

When plugin support is enabled in the linker, BFD's (and the
corresponding file handles) are cached for the plugin to use. This
results in quite a large number of simultaneously open files, which
hits the default limit on macOS (256 files).

Since we aren't using LTO now, disable it explicitly when invoking the
linker.

Closes IDF-923
Closes IDFGH-1764
Closes https://github.com/espressif/esp-idf/issues/3989
2019-11-28 11:02:18 +01:00
Roland Dobai c45830d03f tools: Don't use git work-tree in idf_tools.py
Closes https://github.com/espressif/esp-idf/issues/4355
2019-11-26 08:56:34 +01:00
Roland Dobai 97488fe806 Menuconfig: Don't ask to save configuration when nothing has changed
Closes https://github.com/espressif/esp-idf/issues/4303
2019-11-25 11:29:58 +01:00
Michael (XIAO Xufeng) 337b1df430 esp_flash: add unit test for external flash and QE toggling
Tests for external flash chips used to controlled by macros, one bin for
one chip. And tests are done manually. This commit refactored the test
so that all 3 chips can all run in single test.
2019-11-21 12:26:15 +08:00
Angus Gratton 668c0dfb9b Merge branch 'bugfix/idfpy_before_flash_option_v4.0' into 'release/v4.0'
idf.py: Add --before flag to esptool.py call (v4.0)

See merge request espressif/esp-idf!6625
2019-11-21 06:40:14 +08:00
Angus Gratton 0b0f8d6d6c Merge branch 'feature/windows_ccache_enable_v4.0' into 'release/v4.0'
tools: export IDF_CCACHE_ENABLE on Windows by default (backport v4.0)

See merge request espressif/esp-idf!6664
2019-11-21 06:39:27 +08:00
Jiang Jiang Jian 3b879ce8c3 Merge branch 'bugfix/put_more_rx_code_to_iram_v4.0' into 'release/v4.0'
Put some rx code to iram (backport v4.0)

See merge request espressif/esp-idf!6492
2019-11-20 11:19:20 +08:00
Angus Gratton 99fb9a3f7c Merge branch 'bugfix/miscellaneous_cmake_fixes' into 'release/v4.0'
CMake bugfixes

See merge request espressif/esp-idf!6654
2019-11-19 06:52:07 +08:00
xiehang e5773cc2fe esp_wifi: Put some rx code to iram and update phy4180 2019-11-18 02:54:46 +00:00
Angus Gratton 5bb0b39700 ci: Add test configs for 32kHz crystals 2019-11-16 17:06:14 +08:00
Ivan Grokhotkov 68ad6ad63a tools/docker: use correct branch and commit of IDF when building 2019-11-12 09:57:15 +01:00
Ivan Grokhotkov 981b2496cb tools: export IDF_CCACHE_ENABLE on Windows by default 2019-11-12 09:54:57 +01:00
Renz Christian Bagaporo e4137cc6ca cmake: support git worktree
Use rev-parse to get the HEAD directory instead of manually looking for
it. This method works in the main repository, worktrees and submodules.

Closes https://github.com/espressif/esp-idf/issues/4136
2019-11-11 18:09:15 +08:00
Renz Christian Bagaporo 308e4fcce1 cmake: do not check if sdkconfig exists 2019-11-11 18:06:48 +08:00
Renz Christian Bagaporo 95b5745c3c cmake: treat unregistered components as unresolved 2019-11-11 18:03:26 +08:00
Renz Christian Bagaporo ecce2bc926 cmake: introduce BUILD_COMPONENT_ALIASES
This commit makes it so that BUILD_COMPONENT holds only the component,
and a new property BUILD_COMPONENT_ALIASES hold the full name of the
component.

This also removes erroneous check for duplicate components, as this can
never happen:

(1) if two components have the same name but different prefixes,
the internal names are still unique between them

(2)if two components happen to have the same name and same prefix, the
latter would override the former
2019-11-11 18:01:00 +08:00
Renz Christian Bagaporo b69898257a cmake: make build components available before immediately
Previous implementation only builds list of components included in the
build during component registration.

Since the build components is known as the requirements expansion is
ongoing, update the list here instead.
2019-11-11 18:00:50 +08:00
Renz Christian Bagaporo 22548d0ea4 cmake: hide internal targets
Use imported library, which does not create additional rules, but still
allows attaching arbitraty properties instead of custom targets. This
allows the targets to not appear in the target list of IDEs such as
CLion.
2019-11-11 17:58:02 +08:00