1. replace xunitgen with junit_xml, which can log more info
2. allow test cases to handle junit test report by them own
3. allow test cases to log some info into report via `sysout` tag
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
There's still a small chance that can't process command issue still
happen on some runners. Now we change the way to detect if the runner
reset finished: input `-` to check test history for a few times until it
succeeded. If DUT can process check history command, then it should be
able to process other test commands.
Gitlab CI now allow us to define if a job need to be created by
varialbes. This commit add label to CI jobs, so we can make some jobs
not created in certain scenarios when we trigger pipeline with @bot.
Previous version assumed having winpty on the Path meant that it would be needed,
but some tools (like Atlassian Sourcetree) permanently add winpty to the path.
Symptom is arrow keys not working in menuconfig dialog.
Uninstaller is still unsigned, as currently building and signing in 'wine'
and running Linux osslsigncode inside Inno Setup inside wine is awkward.
Closes https://github.com/espressif/esp-idf/issues/1909
TW20810
* BLE and softAP provisioning examples added along with tests.
* An application specific example added for demonstrating custom provisioning scheme.
* tools/esp_prov/esp_prov.py can be used as a provisioning client for SoftAP, BLE and CLI based transports.
Co-Authored-By: Amey Inamdar <amey@espressif.com>
Co-Authored-By: Anurag Kar <anurag.kar@espressif.com>
set shorter time to save some space on server. For those artifacts need
to be stored for long time, we can manually press "keep" button in job page.
Also add a CI job to check if all artifacts have set expire time.
gen_esp_err_to_name.py includes files like
components/lwip/include/apps/esp_ping.h as "apps/esp_ping.h" because all
subdirectories of "include" are used. If "components/lwip/include/apps"
is in the path of the compiler instead of "components/lwip/include" then
include "apps/esp_ping.h" will fail. This fix allows to process lines
like:
'// gen_esp_err_to_name.py: some text "esp_ping.h" some text'
in header files for overwriting the include.
Changes:
KConfig: The option CONFIG_ESP32_RTCDATA_IN_FAST_MEM is added in Kconfig file for esp32 component.
esp32.common.ld: added support of RTC_DATA_ATTR, RTC_RODATA_ATTR data placement into appropriate segment according to Kconfig option.
esp32.ld: linker script is modified to set alias for memory segment selected by Kconfig option to place data. The segments for force placement are added for RTC_FAST_ATTR, RTC_SLOW_ATTR attributes.
esp_attr.h: added new attributes RTC_FAST_ATTR, RTC_SLOW_ATTR for force placement into fest/slow memory.
test_rtc_fast.c: Added unit test cases to check data placement into appropriate memory segment.
Updated documentation for RTC_DATA_ATTR, RTC_FAST_ATTR, RTC_SLOW_ATTR in deep_sleep_stub.rst file.
TW#18001
Closes https://github.com/espressif/esp-idf/issues/1553
added docs and tests for mqtt library, small fixes (removed warnings, option for custom outbox, websocket bug fixed for longer transports). refactored to use common tcp_transport component, support for CMake build system.
Closes#2108
Using 'sudo pip install' can cause a lot of problems because usually
there are a couple of Python interpreters installed. This fix encourages
developers to use `pip install --user` instead which is safer.
Guarantees that a component's project_include.cmake will be called after its dependent components'
project_include.cmake. Because of cycles in the dependency graph, this is less useful than you'd
think but it gives a strong guarantee for any component which is not part of a cycle.
Also applies deterministic ordering (ordering is initialised as COMPONENT_REQUIRES_COMMON then all
COMPONENTS in alphabetical order, but then the sorting by dependencies is applied.)
1. Add the 100 times test when the private key is generated by the random number;
2. Add the bt components to the unit-test-app/config directory.
3. Added the bt unit test case to CI.
Add a command `make erase_ota` - erases otadata if it there is in the
partition table otherwise error 'Partition table does not have ota_data partition'.
Closes https://github.com/espressif/esp-idf/issues/1821
If project.mk is included twice in recursive invocation of Make, some
variables defined on the first pass will not be redefined on the
second pass. Rather than cleaning up these variables before calling
Make recursively, don’t include IDF project.mk at all, if one of the
ut- targets is requested.
We found some SD card on Raspberry Pi could have very bad performance.
It could take seconds to save small amount of data.
If the DUT receives data and save it as log, then it stops receiving data until log is saved.
This could lead to expect timeout. As an workaround to this issue,
``BaseDUT`` class will create a thread to save logs.
Then data will be passed to ``expect`` as soon as received.
51e42d8e92
introduced filtering options which handles the last (unterminated) line
with a delay. This introduced poor performance for console applications
when the user interacts with the ESP32 device in the same line because
there is an artificial delay for each key-press.
bb152030a0
decreased the delay but there are still delays for each key-presses.
The current fix makes only one delay per line instead of each
key-presses. When an unterminated line is detected then no more
"delayed" print is used for the given line.
Fixes https://github.com/espressif/esp-idf/issues/2054
This change allows readonly FATFS to be mounted without wear levelling
support. This will provide the customers a simple way to mount FATFS images
generated on host and flashed onto the chip during factory provisioning.
Since NVS encryption is not supported yet and NVS entry size is limited,
the change will provide an easy alternative for securing the provisioning data
by just marking FATFS parition as encrypted.
The IDF monitor waits for one second before printing the last
unterminated line. This is necessary for the correct filtering.
However, this causes poor performance, for example, in the case of the
system/console example.
This fix reduces the waiting time which improves the response time of
the console example and doesn't seem to influence the monitor filter.
Closes https://github.com/espressif/esp-idf/issues/2054
For some test configurations, not all tests need to be run. This
change allows adding a comment in the configuration file of the
following form:
unit-test-app makefile will use it to determine which components to
build. If the comment is not present, all components will be built
(TESTS_ALL=1).
Also add test configuration for libsodium. It is a separate test
configuration due to the large binary size generated when building
libsodium tests.
Allows you to move the partition table, it gives more space for the bootloader.
Added a new utility - parttool.py. This utility can search for the offset and/or size of the partitions by name and type/subtype. Use for getting APP_OFFSET and PHY_DATA_OFFSET.
The linker(esp32.bootloader.ld) made changes that allow you to write a custom bootloader code more.
TW14125
Before entering the deep sleep, the RTC and FRC counters are synchronized. Updating the boot_time.
Added a unit test for this case.
Fixed warnings for MULTIPLE_STAGES
Closes https://github.com/espressif/esp-idf/issues/1840
These versions of conf/mconf are forked from the upstream, and the CMake system allows mconf-idf to
be installed externally on the path. So the best option is to rename.
Includes changes to Windows installer for v1.1
Don't show the "Python 2 interpreter" option in menuconfig when using CMake.
This is a stop-gap until we support Python 2 & 3 together in ESP-IDF (soon).
Closes https://github.com/espressif/esp-idf/issues/1924
Will give hint when any job failed to assign:
```
Please add the following jobs to .gitlab-ci.yml with specific tags:
* Add job with: psram, ESP32_IDF, UT_T1_1
* Add job with: psram, ESP32_IDF, UT_T1_1
```
We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).
Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.
In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
1. configs are not functional
* decorator will be executed when search case, need to set default config before search case.
2. fix DUT encode/decode errors
* python3 serial don't support write string/unicode, need to convert to bytes first.
* python2 string could failed to encode/decode non-acsii chars
3. fix bug that log folder not created
4. fix bug that test suite name is not correctly passed:
* the keyward arg should be `test_suite_name` not `test_name`
5. fix bug that test stopped on failed case
6. fix DUT `read` don't return all data cache
* `_DataCache.get_data` should first copy all data from queue to data cache and then return to user.
7. fix bug that `expect_all` failed even all expected item passed
8. optimize error info for expect
* print pattern for regular expression when match failed
9. fix bug that set default config doesn't work
1. Attenuator: control programmable attenuator
2. PowerControl: control APC PDU to power on/off devices
3. LineChart: use matplotlib to draw line chart
Refactor IDF "project" functionality under a wrapping of the default
"project" command, so we can tweak it a bit...
Will need more testing in other environments.
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
Unit test runner expects to see the test name echoed after test
selection is made. If the unit test immediately goes into sleep mode,
UART output will not be complete, and test runner will not see the
test name. This flushes the stream buffer and waits for UART FIFO to
be empty before starting the test.
Additionally some parts of code used unity_printf and some used
printf. Since unity_printf was only useful during very early tests
when newlib was not available, replace all its usages with printf.
By default, any job will run unless a filter is given, in that case
the filter will determine if the job should run or not. Some jobs do
not need to be run by default, and should only be triggered using the
bot. For such jobs, BOT_NEEDS_TRIGGER_BY_NAME can added to
environment variables.
With V=0, build process would print “including .../Makefile.projbuild" lines, causing problems for print_flash_cmd target.
The issue was due to the way macro expansion works in make. To delay evaluation of info function until the execution of expanded block, two dollar signs are required.
Test for print_flash_cmd target added.
Since !2190, source files located outside of the component root
produce object files inside build directory. This change fixes
handling of COMPONENT_OBJEXCLUDE variable for such files. Tests are
added.
We had some problems:
simultaneous compiling of kconfig in the same tree;
attempt to use menuconfig in some examples and ut in batch mode (w/o interactive console)
Also increase debug abilities in CI:
force non-interactive building;
add variable DEBUG_SHELL to toggle verbosity of scripts
tools/gen_esp_err_to_name.py generates
components/esp32/esp_err_to_name.c for lookup of the error codes from
all of the IDF project and easily identify all codes in one place
current unit-test-app don't support test components need to communicate
with each other (like GPIO, SPI ...). Now we add multiple devices mode
to unit test app, support writing and running test with multiple DUTs.
please refer to `docs/api-guides/unit-tests.rst` for detail.
This commit backports vTaskDelete() behavior from FreeRTOS v9.0.0 which
allows for the immediate freeing of task memory if the task being deleted
is not currently running and not pinned to the other core. This commit also
fixes a bug in prvCheckTasksWaitingTermination which prevented the
Idle Task from cleaning up all tasks awaiting deletion. Each iteration of the Idle
Task should traverse the xTasksWaitingTermination list and clean up all tasks
not pinned to the other core. The previous implementation would cause
prvCheckTasksWaitingTermination to return when encountering a task
pinned to the other core whilst traversing the xTasksWaitingTermination list.
The test case for vTaskDelete() has been updated to test for the bugfix and
backported deletion behavior.
This commit backports the following features from FreeRTOS v9.0.0
- uxSemaphoreGetCount()
- vTimerSetTimerId(), xTimerGetPeriod(), xTimerGetExpiryTime()
- xTimerCreateStatic()
- xEventGroupCreateStatic()
- uxSemaphoreGetCount()
Functions backported previously
- xTaskCreateStatic()
- xQueueCreateStatic()
- xSemaphoreCreateBinaryStatic(), xSemaphoreCreateCountingStatic()
- xSemaphoreCreateMutexStatic(), xSemaphoreCreateRecursiveMutexStatic()
- pcQueueGetName()
- vTaskSetThreadLocalStoragePointer()
- pvTaskGetThreadLocalStoragePointer()
Unit tests were also written for the functions above (except for pcQueueGetName
which is tested in a separate Queue Registry MR). The original tlsp and del cb test case
was deleted and integrated into the test cases of this MR.
This commit makes the configQUEUE_REGISTRY_SIZE and
configGENERATE_RUN_TIME_STATS configurable in menuconfig.
- configQUEUE_REGISTRY_SIZE can now be set in menuconfig.
- The functions vQueueAddToRegistry() and vQueueUnregisterQueue() were made
SMP compatbile
- pcQueueGetName() was backported from FreeRTOS v9.0.0
- Added test case for Queue Registry functions
- configGENERATE_RUN_TIME_STATS can now be enabled in menuconfig. CCOUNT or
esp_timer can be selected as the FreeRTOS run time clock in menuconfig as
well, although CCOUNT will overflow quickly.
- Run time stats collection (in vTaskSwitchContext) and generation (in
uxTaskGetSystemState) have been made SMP compatible. Therefore
vTaskGetRunTimeStats() now displays the run time usage of each task as a
percentage of total runtime of both CPUs
Squash
This commit makes configUSE_TRACE_FACILITY and
configUSE_STATS_FORMATTING_FUNCTIONS configurable in kconfig. Test cases fro the
functions enabled by the two configurations above have also been added.
Test cases for the following functions have been added...
- uxTaskGetSystemState()
- uxTaskGetTaskNumber()
- vTaskSetTaskNumber()
- xEventGroupClearBitsFromISR()
- xEventGroupSetBitsFromISR()
- uxEventGroupGetNumber()
- uxQueueGetQueueNumber()
- vQueueSetQueueNumber()
- ucQueueGetQueueType()
Test cases for the following functions were not required...
- prvListTaskWithinSingleList()
- prvWriteNameToBuffer()
- vTaskList()
* Only direct invocation of cypto_hash_sha256 & crypto_hash_sha512 produced correct output.
* Some remaining header file mixups, so changed approach to wrapping the state structures.
* Fixes https://github.com/espressif/esp-idf/issues/1044 (crypto_sign problems)
* Add option to disable mbedTLS implementation for SHA256 & SHA512 in libsodium
Fix warnings where undefined vars are used.
Make Kconfig emit "FOO=" for unset bool options
To ensure make variables are always defined, even if empty.
When writing auto.conf, include symbols disabled by dependency to make sure all make variables are always defined.
Fixesespressif/esp-idf#137
Cherry-picked from https://github.com/espressif/esp-idf/pull/138
Too hard to stage the dependencies so that all clean steps complete before any build steps begin. Also, using and then
deleting and then regenerating SDKCONFIG_MAKEFILE in one pass is really hard to manage successfully.
Other terminal emulators (screen, minicom) default to sending CR when
ENTER is pressed on the keyboard. Make behavior of idf_monitor
consistent with them.
Between 3.1.0 and 3.3.0, pyserial had thread cancellation implemented
using a select, which blocked on the stdin and an auxiliary pipe. When
thread had to be cancelled, a byte would be sent into the pipe,
unblocking stdin.
Unfortunately, this method suffers from a problem with using select on
a StreamReader (which represents the decoder wrapped around stdin).
In some cases, when the TTY sends an escape sequence in response to
an escape sequence received from serial, this escape sequence will not
be read from stdin until some key is pressed.
In https://github.com/pyserial/pyserial/commit/cab3dab, this method
was replaced with an TIOCSTI ioctl. This change makes sure we use the
new cancellation method even if the script is running with older
pyserial.
1. Hello World application shows no footprint difference before and
after this change
2. examples/ethernet/ethernet application compiles properly (can't
test with my board)
Makes app image booting more reliable (256-bit rather than 8-bit verification.)
Some measurements, time to boot a 655KB app.bin file and run to app_main() execution.
(All for rev 1 silicon, ie no 340ms spurious WDT delay.)
80MHz QIO mode:
before = 300ms
after = 140ms
40MHz DIO mode:
before = 712ms
after = 577ms
40MHz DIO mode, secure boot enabled
before = 1380ms
after = 934ms
(Secure boot involves two ECC signature verifications (partition table, app) that take approx 300ms each with 80MHz CPU.)
For config-only components, component.mk should now contain "COMPONENT_CONFIG_ONLY := 1"
Also refactored some of the generation of linker paths, library list. This required cleaning up the way the bootloader
project works, it's now mostly independent from the parent.
Now the tests do not pass
Before we had incorrect code of the error code checking
in build_examples.sh for that case and did nothing in test_build_system.sh.
The CI uses the mirror-submodule-update.sh for non-permanent reassignment
of 3rdparty sub-modules to local mirrors (only for 'non-master' branches).
List of the mirrors in mirror-list.txt
You can use the script locally to speed up data retrieval on network problems
check_submodule_sync job uses 'GIT_STRATEGY: clone' to check the availability of public sources
Implements support for system level traces compatible with SEGGER
SystemView tool on top of ESP32 application tracing module.
That kind of traces can help to analyse program's behaviour.
SystemView can show timeline of tasks/ISRs execution, context switches,
statistics related to the CPUs' load distribution etc.
Also this commit adds useful feature to ESP32 application tracing module:
- Trace data buffering is implemented to handle temporary peaks of events load
Add new "make size_components" and "make size_files" targets for analysing makeup of the ELF file.
Plain "make size" output now looks like this:
Total sizes:
DRAM .data size: 11156 bytes
DRAM .bss size: 22304 bytes
Used static DRAM: 33460 bytes ( 294220 available, 10.2% used)
Used static IRAM: 80480 bytes ( 50592 available, 61.4% used)
Flash code: 421463 bytes
Flash rodata: 103824 bytes
Total image size:~ 616923 bytes (.bin may be padded larger)
Run some unit test cases based (poorly) around sodium's test infrastructure.
Increase in unity test task stack is due to signature tests, load a lot of data onto the stack.
CI: auto generate configs for test jobs
Currently CI test jobs are static configured by several pre-generated config files.
This approach have several disadvantages:
1. not flexible to select test cases to run, which is important feature of @bot
2. difficult to update test as we need to pre-generate quite a lot files
3. need to maintain extra config files in IDF
4. not flexible to support new test apps or chips, can't use some new features of test bench
Therefore we'll add assign_test stage between build and test, to generate configs for test jobs.
See merge request !738
CI: save download config in CI build jobs:
IDF built app download config may change with some modification.
save the download configs for built APP in build stage.
then we can use the correct download config in test stage.
See merge request !676
FATFS fixes and tests
This MR includes a set of fixes related to FATFS, SDMMC, and wear levelling:
- `esp_vfs_fat_spiflash_mount` uses `FM_SFD` flag when creating the partition. The volume layout (given in `VolToPart` variable) was not compatible with SFD mode, so mkfs was failing. This fixes the volume layout to use "autodetect" for both volumes. Merges https://github.com/espressif/esp-idf/pull/559.
- fix `prepend_drive_to_path`function, which didn't prepend drive to path (while consuming 2k of stack space)
- fix stack overflow in vfs_fat_link function which allocated two 4kbyte `FIL` structures on the stack
- fix support for having two FATFS instances (in flash and SD) mounted at the same time
- unit tests written for FATFS on SDMMC are made common between SDMMC and WL implementations; FATFS unit tests on WL will run during CI
- fix inconsistent definition of PATH_MAX and ARG_MAX (TW12207, TW12104, https://github.com/espressif/esp-idf/issues/289)
See merge request !732
IDF built app download config may change with some modification.
save the download configs for built APP in build stage.
then we can use the correct download config in test stage.
wear levelling
This MR adds wear levelling component. It presents an interface similar to the interface of spi_flash and esp_partition. Inside, it stores data inside a partition to in a way that reduces worst case number of erase cycles for any given sector.
Also included are APIs similar to the ones provided for SDMMC to mount FAT filesystem on top of the wear levelling partition. A simple example shows how this API can be used.
Ref TW10338.
See merge request !567
- Implements application tracing module which allows to send arbitrary
data to host over JTAG. This feature is useful for analyzing
program modules behavior, dumping run-time application data etc.
- Implements printf-like logging functions on top of apptrace module.
This feature is a kind of semihosted printf functionality with lower
overhead and impact on system behaviour as compared to standard printf.
Windows setup improvements
* Tweak Makefile to significantly reduce startup time on Windows (also reduced on other platforms, but less obvious.)
* Revise windows setup installation script to fix various issues, use latest toolchain.
* Tweak Windows config docs
See merge request !597
SPI flash operations lock for shorter periods
Based on bug report here:
https://esp32.com/viewtopic.php?f=13&t=1489&p=6816#p6773
Long SPI flash erase (and possibly write/read) cycles block all tasks on both CPUs for an extended period, and can lead to WiFi dissassociation (and general priority inversion.)
This MR inserts preemption breaks into large operations (all erases, writes every 8KB, reads every 16KB).
Overhead of a single spi_flash_guart_start()/spi_flash_guard_end() cycle measured at approx 67us (assuming no preemption.)
See merge request !600
* "make monitor" not passed the configured baud rate
Closes#436https://github.com/espressif/esp-idf/issues/436
* Pass toolchain prefix from sdkconfig into monitor tool
* Allow setting EOL in idf_monitor.py, use CRLF by default
* Detect if /dev/tty.X is used on macOS, warn and replace with /dev/cu.X
* If a build fails or gdb exits, ignore Ctrl-T (allowing Ctrl-T Ctrl-A/F to be same key sequence everywhere)
* Add a note about winpty on Windows
Ref 02fdf8271d (commitcomment-21369196)
Expand 'make monitor' support
New 'make monitor' idf_monitor tool for better monitor output. Running 'make monitor' will now:
* Automatically look up code addresses via addr2line and print function, source file, line number in terminal.
* Can reset the ESP32 by typing Ctrl-T Ctrl-R.
* Can run "make flash" by typing Ctrl-T Ctrl-F.
* Can run "make app-flash" by typing Ctrl-T Ctrl-A.
* If gdb stub starts, monitor will automatically run gdb and connect. When gdb exits, ESP32 resets and monitor resumes.
* Exit is still Ctrl-[
Have some more features I'd like to add (log output to file, crash dump support) but I think this is at the point of being useful.
See merge request !565
unit tests: Make UnitTestParser ignore non-directories inside 'components' dirs
Tiny fix to UnitTestParser script behaviour, if you happen to have any stray files in your "components" directories.
See merge request !561
