Secure boot related fixes
Fix some issues (mostly build system) from the secure boot implementation
Also refactor the way submodule checks are applied to make them more reliable.
See merge request !207
nvs_flash: adjust the nvs_flash start sector number
Modify the nvs flash start sector from 6 to 9 because the partition table bin burn address is modified from 0x4000 to 0x8000
See merge request !215
lwip: add option to memcopy packet from L2 to L3
Menuconfig add an option to copy the packet from layer2 (WIFI driver) to layer3 (LWIP), default not copy
See merge request !211
Required at project level because some components use header files in
other components' submodules, and one component with a
submodule (esptool.py) doesn't have or need a component.mk.
Default esp-idf builds now show -Wextra warnings (except for a few:
signed/unsigned comparison, unused parameters, old-style C declarations.)
CI building of examples runs with that level raised to -Werror, to catch
those changes going into the main repo.
1. Actually, btc & api are the same module thing. So the type defined in api header file can be included by btc c/h files.
Besides, why btc & api should be separated, because that user may be aware of api and api refer defines/types, rather than other non-refer things.
All defines/types that user won't use should be put in btc directory (and define with prefix "btc_"), and which will be used by user should be defined with prefix "esp_" .
2. rename attribute value(all is value, not data)
1. which named uuid is esp_bt_uuid_t type
2. which named id is xxx_id_t type, such as service_id_t char_id_t, include inst & uuid
3. share include directory
Per-CPU interrupt handlers and args
Up to now, the interrupt handlers and args were shared between CPUs, that is, if you set an interrupt handler on CPU0, CPU1 would invoke the same interrupt handler on that interrupt. This code gives every CPU its own space for interrupt handlers.
See merge request !192
nvs: fix Page::findItem and Storage::findItem regression
When read caching was added, Page::findItem started modifying itemIndex reference argument even if item wasn't found.
Incidentally, Storage::findItem reused itemIndex when starting search at next page.
So,
- if the first page had a cached index (findItem was called for that page), and it pointed to a non-zero index,
- first page has a few empty items at the end (but is marked full),
- next search looked up the item on the second page,
- index of the item on the second page was less than the cached index on the first page,
then the search would fail because cached starting index was reused.
This change fixes both sides of the problem:
- Page::findItem shouldn't modify itemIndex argument if item is not found
- Storage::findItem should not reuse itemIndex between pages
Two tests have been added.
ref. TW8406
See merge request !171
New makefile component_wrapper.mk allows some variables to be set
before component.mk is evaluated. This properly fixes problems with
sdkconfig being hard to access in all phases of the build.
Including component_common.mk is no longer necessary and will print a
deprecation warning for components which use it.
Reduces number of make invocations, allows variables exported in project
to be seen in all component make processes, not just the main ones.
Also makes a no-op build about 3x faster than it was.
Fixes for github issues
This MR contains a bunch of small fixes for issues raised on Github and esp32.com forum:
- vfs doesn't check error code returned by open syscall
- spi_flash doesn't work when built in release mode
- duplicate definition of O_NONBLOCK when combining LwIP socket.h with sys/fcntl.h
- wrong order of creation of standard streams
- `_times_r` returning incorrect values, causing `clock`to return double of the actual time
- driver/gpio.h: comment fix
- wifi event handlers: fix incorrect MAC address printed by logging statements
- move some functions out of IRAM when compiling for bootloader
Please check commit descriptions for links to issues/forum posts and more details.
See merge request !183
newlib: fix compilation error when no timers are enabled in menuconfig
I have checked that three options (RTC/RTC+FRC1/FRC1) work, but didn't check "None" option.
This adds some missing ifdef blocks to remove "unused function/variable" warnings and fixes the use of `__errno_r` in a non-reentrant function.
See merge request !174
LwIP will define O_NONBLOCK in sockets.h if it isn't defined yet.
If sys/fcntl.h is included after socket.h, there will be duplicate definition.
Work around by including sys/fcntl.h into lwipopts.h.
https://github.com/espressif/esp-idf/issues/75
esp32/make: add detailed return error code for wifi APIs
1. Add detailed return error code for wifi APIs
2. Add description about error code in esp_wifi.h
3. Modify esp_wifi_reg_rxcb to esp_wifi_internal_reg_rxcb
4. Modify esp_wifi_set_sta_ip to esp_wifi_internal_set_sta_ip
5. Mark system_init as deprecated API
See merge request !187
1. Add detailed return error code for wifi APIs
2. Add description about error code in esp_wifi.h
3. Modify esp_wifi_reg_rxcb to esp_wifi_internal_reg_rxcb
4. Modify esp_wifi_set_sta_ip to esp_wifi_internal_set_sta_ip
5. Mark system_init as deprecated API
wpa_supplicant: replace pre-built crypto library with source code
This change replaces libcrypto.a pre-built library with source code from wpa_supplicant.
For now i've decided to keep the modified layout of source and header files which is used in the esp32_rtos_sdk_core project. This will make it easier to move other parts of wpa_supplicant from esp32_rtos_sdk_core into esp-idf. Once we move everything, we can rearrange the files and upgrade wpa_supplicant to latest version.
See merge request !177
1. remove "\n" when calling ESP_LOGX APIs.
2. modify uart_event_t for uart rx data.
3. use MICRO for uart inverse value
4. add uart_tx_data_t for internal tx function.
1. Set XXX_TAG static, remove extern XXX_TAG in uart.h/ledc.h/gpio.h
2. I removed uart_set/get_print_port() functions, these functions are not well tested, I removed them for now.
3. Modify some function names for uart_read/write_bytes
4. Modify uart_write_bytes and uart_write_bytes_with_break.
1. fix bug in ringbuffer.c:
When return an dummy item, free_ptr might exceed rd_ptr, so the write_ptr would overwrite rd_ptr in this case.
2. Delete UART tx task in buffer mode. UART ISR will copy the data from tx buffer to fifo.
When read caching was added, Page::findItem started modifying itemIndex reference argument even if item wasn't found.
Incidentally, Storage::findItem reused itemIndex when starting search at next page.
So,
- if the first page had a cached index (findItem was called for that page), and it pointed to a non-zero index,
- first page has a few empty items at the end (but is marked full),
- next search looked up the item on the second page,
- index of the item on the second page was less than the cached index on the first page,
then the search would fail because cached starting index was reused.
This change fixes both sides of the problem:
- Page::findItem shouldn't modify itemIndex argument if item is not found
- Storage::findItem should not reuse itemIndex between pages
Two tests have been added.
implement time syscalls
This change adds optional support for libc time functions.
New menuconfig option allows selecting FRC1, RTC, both, or none to be used for timekeeping.
- If only FRC1 timer is used, gettimeofday will provide time at
microsecond resolution. Time will not be preserved when going
into deep sleep mode.
- If both FRC1 and RTC timers are used, timekeeping will
continue in deep sleep. Time will be reported at 1 microsecond
resolution.
- If only RTC timer is used, timekeeping will continue in
deep sleep, but time will be measured at 6.(6) microsecond
resolution. Also the gettimeofday function itself may take
longer to run.
- If no timers are used, gettimeofday function will return -1 and
set errno to ENOSYS.
`times` function returns time derived from FreeRTOS ticks. It reports all as 'system' time, 'user' time is reported as zero.
`settimeofday` function may be used to set current time.
LwIP SNTP module is hooked up into the system through `settimeofday`/`gettimeofday`.
Example demonstrating this functionality is also added.
ref. TW6415
See merge request !168
Deep sleep: Any source named rtc_wake_stub* is linked as RTC wake stub code
Also move esp_deepsleep.h documentation out to docs/deep-sleep-stub.rst
See merge request !142
Fix spi_flash_write regression and nvs error recovery
In the previous set of changes related to spi_flash, new alignment checks were added. One of these checks, word alignment of `src` buffer in `nvs_flas_write`, was unnecessary. ROM `SPIWrite` function can handle unaligned source buffers.
This change caused an error to be returned to nvs for some legitimate write operations. Due to a bug in nvs, further write operations were possible, while the internal state of `Page` instance was broken. In WiFi functional tests, this inflicted havoc on the nvs partition, creating multiple duplicate items in the affected page. Because multiple duplicate items per page were never supposed to be handled during page load, duplicates were not removed. Thankfully this caused an assert at a later point in the loading process, otherwise this bug would be very difficult to detect.
This change set fixes the original spi_flash regression, handling of `INVALID` state of `nvs::Page`, and handling of duplicate items.
See merge request !161
Feature/tw8155 optimize tx flow control
1. Remove TX flow control in LWIP
2. Make the return value of esp_wifi_internal_tx consistent with LWIP error code
See merge request !164
146f5962 - Make the return value of esp_wifi_internal_tx consistent with LWIP error code so that
the up-layer can detect the out-of-memory error and take action accordingly, such do flow control.
1. Add a ring buffer for UART TX.
If the buffer size is set to zero, driver will not use a buffer. But we need a task to send data from buffer to fifo. I tried directly copy data in ISR, but the code looked too long for ISR.
2. Modify the format in uart.h
Due to previous flash write bug it was possible to create multiple duplicate entries in a single page.
Recovery logic detected that case and bailed out with an assert.
This change adds graceful recovery from this condition.
Tests included.
lwip: fix tcp rx abnormal issue(tw8242)
In tcp_alloc(), initialize per_soc_tcp_wnd before initializing recv_wnd because recv_wnd depends on per_soc_tcp_wnd.
See merge request !160
Currently a restart is required to recover a page from invalid state.
The long-term solution is to detect such a condition and recover automatically (without a restart). This will be implemented in a separate change set.
Previously the test bench would check failure recovery by introducing error after each write operation.
This makes checks a bit more extensive (and much longer) by failing after every word written. Surprisingly, this change didn't expose any bugs.
ROM SPIWrite routine can work with unaligned sources, so this check is unnecessary.
Furthermore, it breaks nvs_set_str and nvs_get_blob when data pointer is unaligned.
Also fix stray backslash in COUNTER_STOP macro
nvs: fix memory leaks in HashList and nvs_close
Fixes TW8162.
Associated test case is run under Instruments on macOS, until I set up valgrind to test this automatically on Linux.
See merge request !150
Add cross-core int to accelerate task being awoken from another CPU.
This adds a per-CPU interrupt that can be used to poke the CPU to go do something. In this case all that is implemented is a request to yield the current task, used in case a CPU unblocks a task that runs on another CPU. This gets rid of the limitation that inter-CPU communication using queues, muxes etc can take up to a FreeRTOS tick to happen.
Specs!
Sending an in in a queue of length 1 (essentially a semaphore) as quickly as possible (just a small delay in the sender, to make sure the receiver task gets swapped out) for 10 seconds. Number indicates the amount of ints transferred
Old code:
CPU0->CPU0: 42986
CPU0->CPU1,: 2999
New code:
CPU0->CPU0: 42868
CPU0->CPU1: 62073
See merge request !155
Fix the things that broke when adding the new WDTs
Seemingly, I broke a bunch of things when adding the interrupt WDTs and moved the panic handler to the esp32 directory. This fixes that, as well as the issue where flashing would trigger the int wdt. It also bodges in a fix for a merge artifact breaking the halt-on-first-thread-when-openocd-is-connected; that fix should be refined later.
See merge request !157
lwip: support max 16 sockets
Since the customers need more sockets in their application, support max 16 sockets,
in other words, the total socket number of UDP/TCP/RAW sockets should not exceed 16.
See merge request !156
1. change the description of Makefile.projbuild
2. remove the license header in the API document
3. add private inlcuding header code in the components file
Since the customers need more sockets in their application, support max 16 sockets,
in other words, the total socket number of UDP/TCP/RAW sockets should not exceed 16.
This implements esp_partition_read, esp_partition_write, esp_partition_erase_range, esp_partition_mmap.
Also removed getters which didn't add much sugar after all.
spi_flash_read and spi_flash_write currently have a limitation that source and destination must be word-aligned.
This can be fixed by adding code paths for various unaligned scenarios, but function signatures also need to be adjusted.
As a first step (since we are pre-1.0 and can still change function signatures) alignment checks are added, and pointer types are relaxed to uint8_t.
Later we will add handling of unaligned operations.
This change also introduces spi_flash_erase_range and spi_flash_get_chip_size functions.
We probably need something like spi_flash_chip_size_detect which will detect actual chip size.
This is to allow single application binary to be used on a variety of boards and modules.
- spaces->tabs in tasks.c
- update vfs_uart.c to use per-UART locks
- add license to vfs_uart.c
- allocate separate streams for stdout, stdin, stderr, so that they can be independently reassigned
- fix build system test failure
- use posix off_t instead of newlib internal _off_t
- spaces->tabs in tasks.c
- update vfs_uart.c to use per-UART locks
- add license to vfs_uart.c
- allocate separate streams for stdout, stdin, stderr, so that they can be independently reassigned
- fix build system test failure
Feature/wdts
This adds two watchdogs to esp-idf:
- An interrupt watchdog. Kicks in if the FreeRTOS timer interupt on either the PRO_CPU or (when configured) the APP CPU isn't called for a configurable time. Panics, displaying which CPU caused the problem and the registers that may lead to the offending code.
- A task watchdog. A task has to feed it every once in a while. If not, it will print the name of the offending tasks, as well as the tasks currently running on both CPUs, and optionally panic.
Also adds a panic reason to the panic call, as well as fixes the panic code a bit.
See merge request !148
lwip: refactoring for lwip
1. All espressif specific code are prefix with ESP_
2. Define all ESP_ options in lwipopts.h
3. Remove useless code added in 8266
See merge request !154
The customers need to get information about AP that associated with ESP32 station,
these information includes RSSI, channel number etc, so add this new API
Feature/refractor for some wifi ap is
1. Modify esp_wifi_get_station_list to esp_wifi_ap_get_sta_list
2. Modify tcpip_adapter_get_station_list to tcpip_adapter_get_sta_list
3. Remove esp_wifi_free_station_list
4. Remove tcpip_adapter_free_station_list
5. Modify related data struct accordingly
6. Move soft watchdog from wifi lib to IDF
See merge request !152
1. Modify sta to station in comments
2. Modify esp_wifi_get_ap_num to esp_wifi_scan_get_ap_num
3. Modify esp_wifi_get_ap_list to esp_wifi_scan_get_ap_records
1. Modify comments for esp_wifi_internal_tx
2. Fix delay time error in esp32_tx_flow_ctrl which is found in code review,
modify _wait_delay init value from 0 to 1
6ce01d76: rename some wifi internal APIs
aa4d2aa9: last rx buffer is reserved for mgmt frame
bb0ff4a8: tw7775 fix assert when rx auth frame before create bss
component/tcpip_adapter: not update dhcps status when it is stopped after mode switch
When switch the mode from WIFI_MODE_STA/WIFI_MODE_NULL to WIFI_MODE_AP/WIFI_MODE_APSTA,
if the dhcp server is STOPPED, then dhcp server will not start automatically.
See merge request !147
When switch the mode from WIFI_MODE_STA/WIFI_MODE_NULL to WIFI_MODE_AP/WIFI_MODE_APSTA,
if the dhcp server is STOPPED, then dhcp server will not start automatically.
1. Use esp_log API for LEDC and GPIO code.
2. Modify some API return value.
3. Add ledc_set_pin() for LEDC
4. Modify typo in uart.h
Questions: In uart driver ISR handler, I used xxxFromISR , like xSemaphoreGiveFromISR, do those FromISR functions need to be put in IRAM?
Feature/trax
Add Trax support to esp-idf. OpenOCD already has trax support, this allows an esp-idf program to also trigger this when needed. Also included: some more logic to correctly reserve memory blocks for Trax.
See merge request !73
Add menuconfig setting for optimization level
This change adds two options (Debug/Release) for optimization level.
- Debug (default) enables -O0
- Release enables -Os and adds `-DNDEBUG`, which removes all assert() statements
These options may be overriden at project level by adding necessary flags to CFLAGS/CXXFLAGS.
Debugging symbols are kept in both cases, although we may add an option to strip output file if necessary.
Also we used to define all common compiler flags in CPPFLAGS, and then prepended them to CFLAGS/CXXFLAGS.
It made it impossible to add preprocessor macros to CPPFLAGS at component level (one had to use CFLAGS/CXXFLAGS instead).
Some third party libraries are not compatible with this approach. Changed to the more common way of using these variables.
CI will build both debug (default) and release.
See merge request !138
1. (^)TCPIP_DHCP_0101, (^)TCPIP_DHCP_0301: IDF set IP 0.0.0.0 may return error, don't check the result of setting IP to 0.0.0.0;
2. rename (^)TCPIP_DHCP_0102 to (^)TCPIP_DHCP_0212 as it's a DHCP server test case;
3. (^)TCPIP_TCP_0204,(^)TCPIP_TCP_0210,(^)TCPIP_UDP_0201,(^)TCPIP_UDP_0202: recv thread can't be deleted, change case to not create recv thread when create socket ;
4. (^)TCPIP_TCP_0206,(^)TCPIP_TCP_0212: query TCP server status format changed.
5. WIFI_SCAN_0301: check command error and test environment not correct
6. WIFI_SCAN_0302, WIFI_SCAN_0303, WIFI_SCAN_0304: test environment not correct
1. unsupported cases by "phy" command
2. DHCP server behavior different from ESP8266
3. TCP failed cases causes by unstable TCP behavior
4. UDP failed cases by poor UDP performance
This change adds two options (Debug/Release) for optimization level.
Debug enables -O0, release enables -Os and adds -DNDEBUG (which removes all assert() statements).
Debugging symbols are kept in both cases, although we may add an option to strip output file if necessary.
Also we used to define all common compiler flags in CPPFLAGS, and then appended them to CFLAGS/CXXFLAGS.
It makes it impossible to add preprocessor macros to CPPFLAGS at component level (one has to use CFLAGS/CXXFLAGS instead).
Some third party libraries are not compatible with this approach. Changed to the more common way of using these variables.
2.modified the thread.h file==> added the profile task event in it
3.modified bt_prf_task.c==>added the ready function in it
4.added the bt_prf_sys.h file
5.remove the profile_sys.h file
Some small fixes
- Kill unused uxReturn in task.c, https://github.com/espressif/esp-idf/issues/48
- Line end conversion in gpio.c
- Move heap_alloc_caps.h so components can also use it
See merge request !135
add bt_prf_task.h for defined the data & struct used for the profile task
add prodile_sys.h defined the profile system data used for profile task
modified component.mk
1. test will be categorized by test level
2. add test level as attribute to test cases
3. will select TestCaseScript by the test cases added to CI (currently no test case uses test case script)
4. adding test level to test jobs
5. update .gitlab-ci.yml, each job need to set its test case file path
6. update .gitlab-ci.yml, test case path for test report is changed to idf_test
1. night jobs should exit without error if not triggered
2. remove get wifi connect status cases from IDF
3. use Env tag to check if test environment is special
* master: (117 commits)
build system: Add -fno-rtti when compiling C++ code
FreeRTOS KConfig: Limit tick rate to 1000Hz
bootloader: Fix accidental tabs introduced in !78
build system: Print a WARNING if any submodule is out of date
Fix stack overflow message format
'make flash' targets: Print serial port when flashing
lwip/esp32: support iperf
Add data memory for RMT peripheral
syscall write: Should return number of bytes written
Also push relevant tags over
esp32: add libsmartconfig.a to link libs
esp32: not link wps
esp32/lib: update wifi lib to a1e5f8b9
esp32: remove esp_wps.h
add smartconfig header files(merge this after updating libsmartconfig.a version v2.6.2)
esp32/lib: update wifi lib to 3853d7ae
Add Comments
Modify spinlock error in periph_ctrl.c
Define xcoreid offset, add warning in tcb struct wrt the need to also change that define when struct changes
components/tcpip_adapter: add some comments
...
# Conflicts:
# components/freertos/queue.c
# components/freertos/tasks.c
>1000Hz breaks portTICK_PERIOD_MS (see gitlab 4)
A working >1000Hz tick rate is possible with some changes, but beyond a
certain point it's dimishing returns to preempt tasks this often.
lwip/esp32: support iperf
1. Add _exit() definition in syscalls.c
2. Fix a bug in sockets.c on which iperf depends
I'm now investigating performance of esp32, need iperf to be merged in to master branch for convience
See merge request !136
1. add libphy.a librtc.a, but store in bt submodule, if someone use them, please copy to esp32/lib/ instead the origin one
2. add 07_blufi demo
3. change esp32/Kconfig to allow bt/wifi coexist
esp32: Bootloader wake deep sleep stub
App can contain a stub program resident in RTC fast memory. Bootloader
will load the stub on initial boot. If the device wakes from deep sleep,
the stub is run immediately (before any other data is loaded, etc.)
To implement a custom wake stub, implement a function in your program:
```
void RTC_IRAM_ATTR esp_wake_deep_sleep(void)
{
esp_default_wake_deep_sleep();
// other wake logic
}
```
... and it will replace the default implementation.
See merge request !78
We reserve 4KB Slow RTC memory to save RF calibation result and BT NVS data.
If not all these Slow RTC momory Blocks are used, we will open the other parts.
Workaround: Automatically pin no-cpu-affinity task to a core when FPU is used
FPU status at the moment does not migrate cleanly between cores, so tasks without affinity that happen to migrate across FPUs will run into problems. As a workaround, this modification will automatically pin the task to the current CPU when FPU activity is detected. If anything, it's better than getting all kinds of weird and wonderful FPU corruption issues...
See merge request !124
Assert on untested FreeRTOS features
When I SMP-enabled, I went through all the functions and made the necessary modifiucations. I however only managed to test the most used functions thoroughly. While we should test all FreeRTOS functions eventually, this MR will mark the ones we have not tested yet with a macro call that will, if enabled in MenuConfig (default enabled) assert() when the untested function is called.
Also removes some superfluous mux initializion code in EventGroups code.
See merge request !120
1. configure LEDC timer saparately
2. add peripher_crtl.c/.h
To enable the peripheral modules, we have to set/clear the control register in dport_reg.h.
These bits are disabled by default and they are all in a same register, so we need to add a lock on that.
3. add include esp_err.h in gpio.h
component/bt: update libbtdm_app.a
1. update the lib to version 010102 because the branch feature/btdm_controller missed the lib
See merge request !122
App can contain a stub program resident in RTC fast memory. Bootloader
will load the stub on initial boot. If the device wakes from deep sleep,
the stub is run immediately (before any other data is loaded, etc.)
To implement a custom wake stub, implement a function in your program:
```
void RTC_IRAM_ATTR esp_wake_deep_sleep(void)
{
esp_default_wake_deep_sleep();
// other wake logic
}
```
... and it will replace the default implementation.
* master:
app_main: Return type to void
wifi: move type definitions into separate header file
wifi: use default esp_event_send handler in WIFI_INIT_CONFIG_DEFAULT
nvs: Remove flash layout arguments from nvs_init()
Add contributor agreement, update CONTRIBUTING file
Update http_request & https_request examples for new startup flow
Add very simple "hello world" & "blink" examples
Build examples out-of-tree as part of CI process
Examples: Use event groups for waiting until WiFi is associated & ESP has IP
BT example: Enable BT stack in config by default
BT: Relink component on new BT library, auto-initialise submodule if missing
Documentation: Add contributor guide, expand README & add an examples README
Update gitignore for examples
examples: Add https_request example
mbedtls: Add some initial menuconfig options
examples: HTTP request example
component/esp32: udpate wifi lib
component/esp32: modify bool argument name from enable to en
component/esp32: adjust some APIs
esptool: Bump upstream revision
1. move the app demo code to example
2. move the header files to correct path
3. move profile to profiles directory and seperate esp&std
4. some other codes
5. needs continue to cleanup the code
esptool: Bump upstream revision
Fixes github #14 (unexpected errors writing to DIO flash)
Also speed boost when writing compressed data (can now go via stub)
See merge request !103
* master: (32 commits)
components/nghttp: rename Makefile to component.mk
components/nghttp: add HTTP2.0 protocol feature
compoenents/esp32: don't alias start_cpu1 to start_cpu0_default
components/freertos: fix a bug with an uninitialised return value
Change application entry point name back to app_main
gitlab-ci: build SSC with matching branch name, if available
components/spi_flash: remove stray level of indentation
components/nvs: fix broken sentences in comment blocks
components/esp32: fix renaming of esp_event_set_cb, minor clean up
components/esp32: remove "_user" part from entry points, weaken start_cpu0/1
Fix a spelling mistake in panic message, add carriage return to stack canary message
components/lwip,esp32: fixes for C++
components/bt: fix compilation, remove ./ from makefile
components/esp32: clarify type of queue in wifi_init_config_t, add default init macro
components/esp32: add ESP_ERROR_CHECK
Event handling refactoring
clean up warnings
components/esp32,bt: fix typos in comments
Startup flow refactoring
Roll back submodule version
...
While this may reduce esp_wifi.h file readability for people who don't have a "go to definition" function in their editors, this is needed to decouple esp_wifi and esp_event headers, and possibly other headers which may use wifi types in the future.
Startup flow refactoring
This set of commits changes the startup code in a way that lets the application choose if/when to initialize WiFi/BT.
Application entry point is now a more familiar `main()` function. This function is executed in its own task. Application may choose to do some initialization from main function, create some tasks and then return from `main`. Simple applications may choose to do all their work from `main`.
Additionally this MR splits event handling code into two parts.
- One part is a set of standard handlers for WiFi and DHCP events. Most applications will use this set of handlers, and it is made available via new `esp_event_process_default` function.
- Another part is the default implementation of event handling loop. Some applications may choose to use default event loop through `esp_event_loop_` set of APIs, which start an event handling task and call user-provided event callback from this task. Other applications may create an event queue and implement event loop themselves. In this case application has to provide `esp_event_send` function. In this case the implementation provided by `esp_event_loop_` module is unused.
esp-idf-template has been updated to match this set of changes: https://github.com/espressif/esp-idf-template/tree/feature/init_refactoring
BT example has also been updated.
We need to provide examples of both event handling approaches. This will be done in a separate follow-up MR.
See merge request !112
Fix a spelling mistake in panic message, add carriage return to stack canary message
Probably the most trivial mr ever, but the spelling error was reported as a bug and the missing \r bugged me ever since I saw it.
See merge request !113
* master:
components/spi_flash: remove stray level of indentation
components/nvs: fix broken sentences in comment blocks
Roll back submodule version
Spinlocks already come initialized. Remove the code that would essentially re-initialize them at runtime
Remove all references to prvLockQueue / prvUnlockQueue
components/esp32: clean up unused function warnings in single core mode
clean up warnings
components/nvs: fix build, use log library instead of printf
components/spi_flash: add flash operation counters option to Kconfig
components/nvs: add erase function
components/nvs: fix formatting
components/nvs: batch writes when possible
components/spi_flash: add performance counters
components/nvs: maintain item hash list at page level
components/nvs: avoid reading just-erased page
# Conflicts:
# components/esp32/cpu_start.c
# components/esp32/event_default_handlers.c
nvs: speed improvement and erase APIs
NVS initialization speed issue raised in https://ezredmine.espressif.com/issues/6881 has been addressed for single core mode through optimization of `spi_flash_*` routines. This MR addresses same issue for dual core mode.
- An index table is added at Page level, it allows doing fast search of item index based on key name and namespace. This heavily reduces the number of flash reads.
- Write operations are batched when writing/erasing long variable-length values (str and blob). This gives a 33% reduction in the number of writes for a 64-byte value, 50% reduction for a 96-byte value.
- Add optional (configurable via menuconfig) perfomance counters for spi_flash APIs
With these changes, total init time for NVS goes down from 2 seconds to 140 ms.
Additional RAM usage is 128-640 bytes per page, depending on content. Typical RAM usage for current use case (storing wifi configuration parameters) is 256 bytes.
This change also exposes APIs to erase single key or the whole namespaces, needed for https://ezredmine.espressif.com/issues/6769.
See merge request !105
Clean up warnings and remove unused parts of FreeRTOS
For third party components (lwip and expat), compilation flags are adjusted to silence existing warnings (i have manually checked that all warnings are benign).
In components/esp32, replaced use of WIFI_DEBUG with ESP_LOG functions.
Additionally prvLockQueue/prvUnlockQueue have been removed, dragging some other bits of FreeRTOS along.
Original issue: https://github.com/espressif/esp-idf/issues/18
See merge request !111
- put contents of a few headers into c++ guard blocks
- fix off-by-one error in do_global_ctors
- remove system_init from startup code (should be called from main)
This change separates definitions in esp_event.h and functions in event.c into several parts:
- event structure definitions (esp_event.h)
- default implementations of event handlers (event_default_handlers.c)
- default implementation of event loop (event_loop.c, esp_event_loop.h)
Purpose of this change is to allow applications choose their own poison:
- full control of event loop at the expense of more bootstrap code
- pre-defined event task firing event callbacks, but less code in app_main.c
For third party components (lwip and expat), compilation flags are adjusted to silence existing warnings (i have manually checked that all warnings are benign).
In components/esp32, replaced use of WIFI_DEBUG with ESP_LOG functions.
Only remaining warning is in FreeRTOS queue.c, and it may be a useful one.
This change removes implicit WiFi/BT initialization from startup code.
"main" task is started once essential part of startup code is complete.
This task calls application-provided "int main(void)" function, which can call WiFi/BT init functions if necessary.
1. add SSL session new and free function
2. add SSL session peer cert get and free operation
3. above all, change low-level cert object to be object point not object
For third party components (lwip and expat), compilation flags are adjusted to silence existing warnings (i have manually checked that all warnings are benign).
In components/esp32, replaced use of WIFI_DEBUG with ESP_LOG functions.
Only remaining warning is in FreeRTOS queue.c, and it may be a useful one.
Introduces new internal function, Page::alterEntryRangeState, which gathers changes to multiple elements of entry state table into a single write, provided that these changes fall into a single word. This allows changing state of up to 16 entries in a single write.
Also adds new function, writeEntryData, which writes the whole payload of SZ and BLOB type entries in one go, instead of splitting it into multiple 32-byte writes.
This reduces number of writes required for SZ and BLOB entries.
1. add empty function to load private key into SSL context
2. add empty function to load certification into SSL context
3. add function to load RSA private key
1. add openssl 'new' and 'free' function
2. add clear unused variate to void warning to appear when compile
3. add internal function 'X509_new' to take the place of 'sk_X509_NAME_new_null' function whitch is openssl stack function
This feature allows to use static buffers (or from a pool of memory which is not
controlled by FreeRTOS).
In order to reduce the impact of the changes, the static feature has only been added
to the queus (and in consequence to the semaphores and the mutexes) and the tasks.
The Timer task is always dynamically allocated and also the idle task(s), which in the
case of the ESP-IDF is ok, since we always need to have dynamic allocation enabled.
lwip: add debug code to show udp/tcp pcbs
Add code to show all tcp/udp pcbs, these kind of debug info is helpful for lwip issue debugging.
See merge request !98
* master: (57 commits)
components/lwip: fix grammar
components/lwip: make SO_REUSE configurable via menuconfig
bootloader: remove trailing newlines from log messages
components/freertos: override per-task __cleanup handler to close stdin, stdout, stderr
components/esp32: move peripheral symbols to separate ld script
components/log: regroup macros for better readability
gitlab-ci: allow running tests for branches, triggered via API
components/log: fix timestamp calculation
components/log: set default runtime log level to ESP_LOG_VERBOSE
components/log: fix error when using ESP_LOGx from C++ code
components/log: fix bugs, add options to override log level for files, components, and bootloader
fix ledc and spi typo
remove prefix and postfix
Enable SO_REUSEADDR in LWIP
freertos: fix memory debug macro issue Define configENABLE_MEMORY_DEBUG according to CONFIG_ENABLE_MEMORY_DEBUG
peripheral structure headers: move volatile keyword from members to typedef
Adding -fstrict-volatile-bitfields to the CFLAGS/CXXFLAGS. Without this, gcc tries to access bitfields using the smallest possible methods (eg l8i to grab an 8-bit field from a 32-bit). Our hardware does not like that. This flag tells gcc that if a bitfield is volatile, it should always use the type the field is defined at (uint32_t in our case) to size its access to the field. This fixes accessing the hardware through the xxx_struct.h headers.
add peripheral module struct headers
build system docs: Add note about no spaces in component names
Docs: Add note about unusual submodule messages when cloning on Windows
...
# Conflicts:
# components/esp32/cpu_start.c
# components/esp32/include/soc/cpu.h
Add logging library
Logging library, intended to be used across other components.
Uses design similar to Android NDK logging APIs and allows for both compile time and run time filters, similar to logcat.
Also includes cleanup of cpu_startup.c — i was changing some logging output in this file so decided to re-format it and reduce code duplication.
Ref TW6703
See merge request !68
* branch 'master':
components/freertos: override per-task __cleanup handler to close stdin, stdout, stderr
gitlab-ci: allow running tests for branches, triggered via API
Enable SO_REUSEADDR in LWIP
Daniel initially asked me why this wasn't enabled, and I don't think I got any reasons against enabling this. If any, it makes porting existing software easier. Tuan needs it for UDP multicast as well.
Code changes are by Tuan: basically the enable for SO_REUSEADDR in LWIP as well as a bugfix in a bit of mbedTLS that gets enabled.
See merge request !90
Move CPU region protection setup into soc/cpu.h
change tabs to spaces
remove unused extern declarations
use RTC_WDTCONFIG0 instead of numeric address (still need to fix BB reg)
This also removes logging implementation from bootloader and replaces it
with the one provided by the log component. Some occurrences of printf
and ets_printf have been changed to ESP_LOGx APIs.
Note that with WiFi stack enabled, system_init will reset frequency to 240MHz.
To make this setting useful, esp32-wifi-libs submodule needs to be updated.
esp32/lib: update to 1303c92c
1. support to set vendor ie;
2. fix "mode:" log twice issue; TW7019
3. fix station can not connect to some AP in WEP mode; TW7021
See merge request !85
Function declaration should only be included if MBEDTLS_SHA512_ALT is
not set. This matches sha1.h and sha256.h
This change should be contributed back upstream to mbedTLS project.
* branch master: (51 commits)
fix typos
callback: add a void* pointer with each callback
header: callbacks use "void *" as arguments
header: format and fix typos
add wifi api comments
add wifi api comments
components/spi_flash: call SPIUnlock only once
components/spi_flash: improve comments and readability
esp32 cpu_start: Include rom/uart.h, remove inline ROM function declarations
bootloader: Add bootloader Kconfig
Fix bug where nesting vTaskEnterCritical calls would not re-enable interrupts after vTaskExitCritical sections.
installed gitlab key in global before_script, don't need to overwrite in template before_script
ci: Use github repo for wifi libraries on master branch, gitlab repo otherwise
components/esp32,spi_flash: update some comments
components/esp32: remove dependency of WIFI_ENABLED on FREERTOS_UNICORE
esp32/lib: Bump WiFi revision
update ci: build ssc with idf submodule wifi lib
components/esp32: add inter-processor call API and implement spi_flash through it
add MACRO to get flash pad number from efuse value
add comments to register read/write operations
...
lwip: add socket tx flow control
This fix is used to solve wifi q full issue:
1. When wifi tx q is full, set tx_enable to false
2. When wifi tx q decrease 1/2 max length, set tx_enable to true again
3. esp_wifi_tx_enable return whether tx_enable is true
4. If esp_wifi_tx_enable return false, socket will be blocked
See merge request !62
This fixes the performance impact for spi_flash_write and spi_flash_erase.
With this change, NVS init in single core mode takes about 50ms (compared to >2seconds before that).
Dual core compatible spi flash APIs
Implementation of spi flash APIs which work in dual core mode.
This also includes other changes:
- fixes to FreeRTOS tasks.c made by Jeroen and me — these were originally made to fix issues with high priority tasks seen in WiFi stack, but were required to get stuff in ipc.c working as well
- initial base idea with two high-priority tasks was separated into a separate API — esp_ipc.h. That API allows one to run a function on the other CPU from a high-priority task. Because we don't have preemption across CPUs, this is currently not very fast: it requires on average half a tick until the second CPU switches to the high priority task. Currently that API can be either blocking, or non-blocking with queue length of 1. I haven't yet found a use case for arbitrary sized queue of IPC requests. Anyone who needs that can trivially add that later by replacing a binary semaphore with a generic queue.
With these changes i can run application with WiFi stack and NVS on both cores.
See merge request !25
1. timer reg file for both time group 0 and time group 1, not only timer group 0
2. fix bug that io mux header file mismatch with chip
3. fix bug that some BASE address not correct
4. add some static function to eagle.fpga32.rom.addr.v7.ld
5. add interrupts usage table
6. add some comments for rom code functions
* master: (130 commits)
lwip: Define LWIP_ESP8266 in port lwipopts.h not gcc command line
CI: Build the esp-idf-template with the matching branch name, if it exists
README: Add Resources section with some links
Rename README.buildenv to docs/build_system.rst and ReST-ify it
Eclipse docs: Easier to just replace entire PATH, msys32 has everything we need to build/flash
test_build_system: Print ESP_IDF_TEMPLATE_GIT for easier debugging
Name component makefiles component.mk instead of Makefile
Eclipse doc: Add troubleshooting note about Makefile directories
eclipse_make.sh: Fix printing of make directory
Move bin/eclipse_windows_make.sh to tools/windows_eclipse_make.sh
Eclipse docs: Prepend IDF paths to beginning of PATH
Set default SPI flash access mode to DIO
FreeRTOS: temporary solution for memory canaries and memory debug
tcpip_adapter: fix dhcp client work flow
event: not post got ip event if static is invalid
tcpip_adapter: typedef clean up
event: post got ip event when use static ip
tcpip_adapter: use dhcp callback to post got ip event
dhcp: add dhcp callback
lwip: remove netif_reg_addr_change_cb
...
Used by mbedTLS to set MBEDTLS_CONFIG_FILE in all components.
This change sets CFLAGS/etc at the project level and then exports those
variables for components, rather than setting them independently each time
a component Makefile is invoked.
Eclipse fixes, including renaming component Makefiles to component.mk
Some changes after following up some Eclipse problems with @nilay .
Biggest thing is renaming all component Makefiles to component.mk. This requires changes to all esp-idf projects, ie:
* https://github.com/espressif/esp-idf-template/compare/bugfix/docs_eclipse?expand=1
* http://gitlab.espressif.cn/idf/esp-idf-tests/tree/update/component_makefiles
... this fixes Eclipse weirdness and also means the only "Makefile"s in the project are the ones you should run make against.
If this MR is merged then I'll also merge these, email all developers about the breaking change and post on esp32.com.
See merge request !34
Set default SPI flash access mode to DIO
We have a bunch of boards out there (the NodeMCU-lookalike, breadboardable boards) in the hands of early reviewers. Because the flash chips on these boards have a different command to go into QIO mode, ESP-IDF will break on these boards out of the box. Eventually, we need some nifty way to autodetect this or something, but for now, the quick fix is to make DIO the default access mode.
Once this is merged, I will also modify the default in the template apps sdkconfig.
See merge request !65
Should protect against concurrent use of hardware crypto primitives,
with good performance.
Not necessary to call esp_aes_acquire_hardware(),
esp_sha_acquire_hardware(), etc when using these APIs. These are
provided for external users calling the hardware crypto hardware
directly, to coexist with this implementation.
Fixes problems with Eclipse trying to build in directories it shouldn't.
This is a breaking change for existing repositories, they need to rename
any component Makefiles to component.mk and rename their references to
$(IDF_PATH)/make/component.mk to $(IDF_PATH)/make/component_common.mk
FreeRTOS: temporary solution for memory canaries and memory debug
1. This is just a temporary solution, it will be removed when umm_malloc is ready
2. Support memory canaries mechanism
2. Add debug code to show allocated memory info
See merge request !64
tcpip_adapter: got ip event stuff
Include:
1. add dhcp callback;
2. post got ip event in dhcp callback when dhcp bind;
3. tcpip_adapter typedef clean;
4. post got ip event in station connected event handler when use valid static ip;
TW6926
See merge request !61
1. This is just a temporary solution, it will be removed when umm_malloc is ready
2. Support memory canaries mechanism
2. Add debug code to show allocated memory info
Since esp32 has much more memory than esp31B, modify the HEAP_HIGHWAT from
6K to 20K.
Notes: HEAP_HIGHWAT is used for tcp/udp tx flow control, if socket task detects
that the free heap memory is less than HEAP_HIGHWAT, it stops to trasmitting until
the free heap memory is greater than HEAP_HIGHWAT
1. Add the parameter type of api esp_wifi_set_storage.
2. WIFI_STORAGE_RAM means storing profiles in ram. WIFI_STORAGE_FLASH means storing profiles in flash.
Fix: Return after setting breakpoint when ocd is detected instead of continuing …
…into the panic routine
Made a small thinko when refactoring the panic handler code. This fixes it.
See merge request !36
The parameter aid followed -a is aid of the connected station. If aid is 0, kick out all of the connected stations.
2. Add block parameter to api esp_wifi_scan_start.
If block is true, the api blocks before scan done or returns directly.
components/expat: add XML parse feature
Develop and Compile XML Parse based on XMPP protocol, can test from unity running list, see expat file.
See merge request !32
found small mistake in it
we have:
PROVIDE ( ets_aes_crypt = 0x4005c9b8 );
PROVIDE ( ets_aes_disable = 0x4005c8f8 );
PROVIDE ( ets_aes_enable = 0x4005c8cc );
PROVIDE ( ets_aes_set_endian = 0x4005c928 );
PROVIDE ( ets_aes_setkey_dec = 0x4005c994 );
PROVIDE ( ets_aes_setkey_enc = 0x4005c97c );
Signed-off-by: Ivan Grokhotkov <ivan@espressif.com>
Implement lwip per-thread-sem with:
vTaskSetThreadLocalStoragePointerAndDelCallback/pvTaskGetThreadLocalStoragePointer
register sys_thread_tls_free to free per-thread-sem when the thread is terminated.
Also let lwip use index CONFIG_LWIP_THREAD_LOCAL_STORAGE_INDEX
2. Add struct station_list into tcpip_adapter layer
3. When ap -L cmd is received, get mac address from mac layer first and then search ip address based on mac address on dhcp layer.
update ld and heap region
Divide origin fpga ld to several lds, choose specific ld for normal/BT/Trace by menuconfig, change iram/dram len in ld to full use hardware ram region.
See merge request !28
Fix dualcore startup
This MR includes:
- a workaround for a hardware bug with cache initialization
- fix of `#if CONFIG_WIFI_ENABLED` block in cpu_start.c for the case when WiFi is disabled.
- removal of miscellaneous delays in cpu_start.c
See merge request !22
fix DRAM_ATTR attribute
Variables marked with DRAM_ATTR were not placed in any specific section and were appended to the last section, causing some errors at binary file generation stage.
This change places contents of .dram1 section into .dram0.data.
Also removes .iram1pro/.iram1app/.dram1pro/.dram1app which weren't used anywhere.
See merge request !23
Make: fix incremental builds, add build tests
Four semi-related build changes in one:
* Add basic tests for make system sanity (can be expanded as we find corner cases)
* Fix incremental building of bootloader when source files change
* Don't regenerate component libraries or re-link the ELF file if nothing changed
* Rename `$(vecho)` to `$(summary)` and add a new `$(details)` macro to echo some helpful build details when V=1.
See merge request !20
This is a workaround for a hardware bug with cache initialization.
Only two lines of code related to DPORT_APP_CACHE_MMU_IA_CLR were added around mmu_init(1); call,
and two lines at the end of comment block.
I reformatted surrounding lines to use spaces for indentation, like the rest of the code in this file does.
This change removes various delays in PRO and APP CPU startup paths.
Also some unused code and ets_printf statements are cleaned up or removed.
2nd CPU now communicates that it has started through a global variable.
The thread-local-storage feature in FreeRTOS attaches an application-usable array of pointers to a thread control block. These pointers usually point to a structure the thread allocates. When a thread gets (voluntarily or involuntarily) destroyed, this memory can leak. This merge adds a matching second array of user-settable pointers to destructor routines. As soon as the task gets cleaned up (which happens in the idle thread), the destructors get called and the memory can be freed.
See merge request !19
Add cr after a lf is printed (configurable)
Small menuconfig option to make the newlib stdout functions replicate the behaviour of ets_printf e.a. by sending out a \r before every \n.
See merge request !14
This commit fixes several issues with state handling in nvs::Page. It also adds extra consistency checks in nvs::PageManger initialization.
These changes were verified with a new long-running test ("test recovery from sudden poweroff"). This test works by repeatedly performing same pseudorandom sequence of calls to nvs_ APIs. Each time it repeats the sequence, it introduces a failure into one of flash operations (write or erase). So if one iteration of this test needs, say, 25000 flash operations, then this test will run 25000 iterations, each time introducing the failure point at different location.
rtos: change XT_CLOCK_FREQ from 13MHz to 80MHz
Freq has been change to `80MHz` in system_init, otherwise system tick will not be accurate now.
Maybe we can config freq by menuconfig.
See merge request !8
