Commit graph

155 commits

Author SHA1 Message Date
Angus Gratton fd88d6d89c heap: Add integer overflow checks 2018-06-19 23:31:37 +00:00
Angus Gratton 6b9784cc45 Merge branch 'master' into feature/cmake_update 2018-05-31 14:46:23 +10:00
Angus Gratton 3fe862dacb test_multi_heap_host: Remove race conditions from parallel "make clean test" 2018-05-22 12:09:30 +10:00
Angus Gratton 68282cf80b heap/cmake: Fix linking error when heap tracing is enabled 2018-05-01 17:27:39 +10:00
Angus Gratton 1cb5712463 cmake: Add component dependency support
Components should set the COMPONENT_REQUIRES & COMPONENT_PRIVATE_REQUIRES variables to define their
requirements.
2018-04-30 09:59:20 +10:00
Angus Gratton 88df8fd293 cmake: Use cmake_lint project, tidy up all CMake source files 2018-04-30 09:59:20 +10:00
Angus Gratton 4f1a856dbf cmake: Remove defaults for COMPONENT_SRCDIRS, COMPONENT_SRCS, COMPONENT_ADD_INCLUDEDIRS
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
  always used. Also trims the -I path from a number of components that have no actual include
  directory.
* Simplifies knowing which components will be header-only and which won't
2018-04-30 09:59:20 +10:00
Angus Gratton c671a0c3eb build system: Initial cmake support, work in progress 2018-04-30 09:59:20 +10:00
Mahavir Jain 5cdd08ef5a heap: fix build failure due to missing header
Closes https://github.com/espressif/esp-idf/issues/1881

Signed-off-by: Mahavir Jain <mahavir@espressif.com>
2018-04-26 11:08:58 +05:30
Ivan Grokhotkov 573ea385b4 multi_heap: use portENTER/EXIT_CRITICAL instead of taskENTER/EXIT_CRITICAL 2018-04-19 18:28:55 +05:30
Angus Gratton fe907bde60 Merge branch 'bugfix/heap_tracing_newlib_allocations' into 'master'
heap tracing: Fix tracing of newlib internal allocations

See merge request idf/esp-idf!1992
2018-04-18 12:07:15 +08:00
Angus Gratton 2914111c8c Merge branch 'bugfix/heap_fragmentation_on_split_pr1761' into 'master'
heap: Try to merge tail with next block when splitting

See merge request idf/esp-idf!2147
2018-04-10 12:14:57 +08:00
Anton Maklakov 046ff2e628 Make error codes looks like literals in the code of them (easy for searching). Minor fixes, typos 2018-04-09 13:54:18 +10:00
Angus Gratton 3ccb12530b heap: Change test_multi_heap_on_host multi-config to bash script not Makefile
Hopefully fix some CI building issues with parallel builds.
2018-04-03 16:34:02 +10:00
Deomid Ryabkov 4b7eb1792e Try to merge tail with next block when splitting
When splitting a memory block, check if the next block is free.
If it is, then just extend it upwards instead of creating a new block.
This fixes a bug where when shrinking existing allocations would result in irreversible free space fragmentation.

When testing on the host, test all the poisoning configurations.
2018-04-03 12:30:52 +10:00
Angus Gratton d3b8f1f927 heap tracing: Fix tracing of newlib internal allocations
Newlib internal allocations (from newlib ROM code) were not being included in the heap trace.

Ref https://github.com/espressif/esp-idf/issues/1652
2018-03-20 12:03:46 +11:00
Stephen Casner bc2879a956 heap: Add task tracking option for heap usage monitoring
Add back a feature that was available in the old heap implementation
in release/v2.1 and earlier: keep track of which task allocates each
block from the heap. The task handle is conditionally added as
another word in the heap poisoning header under this configuration
option CONFIG_HEAP_TASK_TRACKING.

To allow custom monitoring and debugging code to be added, add helper
functions in multi_heap.c and multi_heap_poisoning.c to provide access
to information in the block headers.

Add esp_heap_debug_dump_totals() to monitor heap usage

esp_heap_debug_dump_totals() dumps into a user-provided data structure
a summary of the amound of heap memory in region type that is used by
each task.  Optionally it will also dump into another data structure
the metadata about each allocated block for a given list of tasks or
for all tasks (limited by available space).

Address change requests on PR #1498

This set of changes fixes the files in e3b702c to just add the
CONFIG_HEAP_TASK_TRACKING option without adding the new function
heap_caps_get_per_task_info() in case that is the only portion of the
PR that will be accepted.  Part of the change is to remove the new .c
and .h files containing that function and to remove the line to
compile it from components/heap/component.mk since it should not have
been included in e3b702c.  One or more additional commits to add the
new function will follow.

The other changes here:
- uint32_t get_all_caps() moves to heap_private.h
- replace "void* foo" with "void *foo"
- add braces around single-line "if" blocks
- replace tab characters with spaces

Address change requests on PR #1498, part 2

This set of changes fixes the files in cdf32aa to add the new function
heap_caps_get_per_task_info() with its new name and to add the line to
compile it in components/heap/component.mk.  This does not address all
the suggested changes because there are some needing further
discussion.

This commit does not include the suggested change to move the
declaration of the new function into esp_heap_caps.h because the new
function references TaskHandle_t so esp_heap_caps.h would have to
include freertos/FreeRTOS.h and freertos/task.h, but FreeRTOS.h
includes esp_heap_caps.h through two other header files which results
in compilation errors because not all of FreeRTOS.h has been read yet.

Change heap_caps_get_per_task_info() to take struct of params

In addition to moving the large number of function parameters into a
struct as the single parameter, the following changes were made:

- Clear out the totals for any prepopulated tasks so the app code
  doesn't have to do it.

- Rather than partitioning the per-task totals into a predetermined
  set of heap capabilities, take a list of <caps,mask> pairs to
  compare the caps to the heap capabilities as masked.  This lets the
  caller configure the desired partitioning, or none.

- Allow the totals array pointer or the blocks array pointer to be
  NULL to indicate not to collect that part of the information.

- In addition to returning the total space allocated by each task,
  return the number of blocks allocated by each task.

- Don't need to return the heap capabilities as part of the details
  for each block since the heap region (and therefore its
  capabilities) can be determined from the block address.

- Renamed heap_task_info.h to esp_heap_task_info.h to fit the naming
  convention, and renamed the structs for totals and block details to
  better fit the revised function name.

- Provide full Doxygen commenting for the function and parameter
  structs.

Add copyright header to new files

Merges https://github.com/espressif/esp-idf/pull/1498
2018-02-20 10:32:06 +11:00
Stephen Casner a76d5a33c9 heap: Correct the sense of a comment 2018-02-19 09:42:40 +11:00
Angus Gratton b7fc067c8c heap: Fix bug when realloc moves data between heaps
When realloc-ing to a smaller buffer size which ends up allocated in a different heap, the heap
structure is corrupted. This can only happen:

* If heap checking is Comprehensive (meaning buffers are never shrunk in place) and the heap the buffer was originally allocated in is full.
* Calling heap_caps_realloc() to deliberately move a buffer to a different capabilities type, and shrink it at the same time.

Probable fix for https://github.com/espressif/esp-idf/issues/1582
Probably the same issue:
https://www.esp32.com/viewtopic.php?f=2&t=4583
https://www.esp32.com/viewtopic.php?f=13&t=3717
2018-02-09 16:10:52 +08:00
Kewal ae8a0bbf56 fix typo for heap cap free size 2017-12-14 10:39:26 +05:30
Mahavir Jain 18553c451a heap_trace: fix bug in realloc for copying trace record
Closes https://github.com/espressif/esp-idf/issues/1354

Signed-off-by: Mahavir Jain <mahavir@espressif.com>
2017-12-07 18:20:23 +05:30
Ivan Grokhotkov 5fe91ae7cd Merge branch 'bugfix/cpp_guards' into 'master'
Add C++ guards to freertos/ringbuf.h, heap headers

See merge request !1552
2017-11-22 22:26:26 +08:00
Angus Gratton a556e1c782 heap: Add C++ header guards to heap headers
Closes https://github.com/espressif/esp-idf/issues/1195
2017-11-22 10:08:50 +11:00
Angus Gratton 7a924bd85a spi_flash: Expose an accessor the current SPI flash guard functions
Change places which uses g_flash_guard_default_ops to use this. Probably exact same data, but a bit
cleaner.
2017-11-20 15:54:31 +11:00
Tian Hao e7a9ddcf72 component/heap : fix heap_region_add check bug 2017-11-16 11:10:13 +08:00
Darian Leung fff482acd9 freertos/add C++ include guards for esp_heap_caps.h 2017-11-10 12:01:30 +08:00
Anton Maklakov bb4838314c doc: Add internal links for chapters and fix typos
Used 'git grep -i -E "see *\`[^\`]+\`[^_]"' to find the invalid links
2017-11-09 11:45:13 +08:00
Ivan Grokhotkov dbebece1d2 Merge branch 'feature/unit-test-configs' into 'master'
unit-test-app: add support for testing multiple configurations

See merge request !1249
2017-10-20 14:03:53 +08:00
Angus Gratton 47aaf402b8 heap: Add heap_caps_dump() / heap_caps_dump_all() functions
Dump the structure of the heap for debugging purposes.
2017-10-20 11:38:24 +08:00
Ivan Grokhotkov 90bbcbcdc0 unit tests: fix warnings, build with -Werror
- libsodium: silence warnings
- unit tests: fix warnings
- spiram: fix warnings
- ringbuf test: enable by default, reduce delays
2017-10-19 21:35:23 +08:00
Ivan Grokhotkov 6cc8099610 Merge branch 'bugfix/malloc_failure' into 'master'
heap: Fix race condition causing malloc() to fail under some conditions

See merge request !1424
2017-10-19 21:30:26 +08:00
Angus Gratton b0c5665f15 heap: Fix race condition causing malloc() to fail under some conditions
During a call to multi_heap_malloc(), if both these conditions were true:
- That heap only has one block large enough for the allocation
  (this is always the case if the heap is unfragmented).
- Another allocation is simultaneously occurring in the same heap.

... multi_heap_malloc() could incorrectly return NULL.

This caused IDF heap_caps_malloc() and malloc() to also fail, particularly
often if only one or two heaps had space for the allocation (otherwise
heap_caps_malloc() fails over to the next heap).
2017-10-19 16:05:00 +08:00
Angus Gratton 04188d8ec7 heap: Fix spurious heap_caps_check_integrity() errors in Comprehensive mode
Heap was not being locked before poisoning, so heap_caps_check_integrity()
would sometimes race with checking the poison bytes and print unnecessary
errors.

Details: https://esp32.com/viewtopic.php?f=2&t=3348&p=15732#p15732
2017-10-18 15:57:52 +08:00
XiaXiaotian 5df39cd4b6 Allocate some memories in SPIRAM first.
Try to allocate some WiFi and LWIP memories in SPIRAM first. If
    failed, try to allocate in internal RAM then.
2017-10-13 10:11:24 +08:00
Angus Gratton dda136eab7 Merge branch 'bugfix/heap_debugging_cleanup' into 'master'
Heap debugging docs cleanup & convenience functions

See merge request !1347
2017-10-10 14:59:54 +08:00
Angus Gratton f0d7cfdafe heap: Add new heap_caps_check_integrity_all() & heap_caps_check_integrity_addr() debugging functions
Easier to either check all heaps, or focus on checking a particular region.
2017-10-10 16:19:30 +11:00
Angus Gratton bb8ba76604 Merge branch 'bugfix/multi_heap_get_info_typo' into 'master'
Fix multi_heap_get_info alias (from github)

See merge request !1357
2017-10-10 13:14:09 +08:00
Angus Gratton ec85f9fc3f heap: Fix documentation references to multi_get_heap_info() 2017-10-09 14:42:31 +11:00
Jeroen Domburg 740f8a79f0 Add logic to make external RAM usable with malloc() 2017-09-28 17:17:50 +08:00
Deomid Ryabkov 882f164e7c Fix multi_heap_get_info alias 2017-09-26 15:00:47 +01:00
Angus Gratton 1773770f44 Merge branch 'feature/multi_heap_assert' into 'master'
multi_heap: Print the problem address when aborting due to heap corruption

See merge request !1277
2017-09-25 09:51:14 +08:00
Tian Hao b54719d00f component/bt : fix bluetooth controller enable limit && release memory when de-initialize bluetooth controller
1. fix bluetooth controller enable limit
2. release memory when de-initialize bluetooth controller
3. fix heap_caps_add_region limit
2017-09-19 21:14:28 +08:00
Angus Gratton 959462ffb6 multi_heap_poisoning: Use MULTI_HEAP_STDERR_PRINTF (ets_printf) to print heap errors
Needed because normal printf() can trigger a malloc() (for standard stream locks) which
then re-triggers this check.
2017-09-18 16:54:28 +10:00
Angus Gratton 76d8190444 multi_heap: Print the problem address when aborting due to heap corruption
New multi_heap code has proven effective at aborting when buffer overruns occur,
but it's currently hard to debug the stack traces from these failures.
2017-09-18 16:54:23 +10:00
Jeroen Domburg 875ae6a134 Add option to allocate external RAM using heap_alloc_caps 2017-09-14 10:47:44 +08:00
Angus Gratton 72995bfcec doc: Add docs for heap trace & poisoning 2017-09-07 16:32:05 +10:00
Angus Gratton 1c7b8aa3a5 Heap tracing support 2017-09-07 16:32:05 +10:00
Angus Gratton 5c417963eb multi_heap: Add heap poisoning features 2017-09-07 16:32:05 +10:00
Angus Gratton 5222428dde unit tests: Check heap integrity after each test, check for obvious leaks 2017-09-07 16:32:05 +10:00
Angus Gratton 1cb0f30933 heap_caps: Add heap_caps_check_integrity() function 2017-09-07 16:32:05 +10:00
Angus Gratton 5361c08989 heap: Support adding new heap regions at runtime
To facilitate this, the list of registered heap regions is now a linked list
(allowing entries to be appended at runtime.)
2017-09-05 14:07:02 +10:00
Angus Gratton 0feb40833a heap_caps: Allow for possibility a region is too small to register a heap
May happen due to sdkconfig, static allocation of RAM.

Closes https://github.com/espressif/esp-idf/issues/802
2017-07-19 17:24:24 +10:00
Angus Gratton ad60c30de0 heap: Rename memory "tags" to "types" to avoid confusion w/ old tag allocator API 2017-07-10 17:46:03 +08:00
Angus Gratton 71c70cb15c heap: Refactor heap regions/capabilities out of FreeRTOS
Remove tagged heap API, rename caps_xxx to heap_caps_xxx

Also includes additional heap_caps_xxx inspection functions.
2017-07-10 17:46:03 +08:00
Angus Gratton 5ee49fd311 heap: Add new multi_heap heap implementation to replace FreeRTOS-based tagged heaps 2017-07-10 17:46:03 +08:00