docs: Add some more explanation about heap, memory types
This commit is contained in:
Angus Gratton
Angus Gratton
parent
fa59b1b1c9
commit
a3a1cc080e
|
|
@ -88,6 +88,8 @@ Multiple ``idf.py`` commands can be combined into one. For example, ``idf.py -p
|
|||
|
||||
.. note:: The environment variables ``ESPPORT`` and ``ESPBAUD`` can be used to set default values for the ``-p`` and ``-b`` options, respectively. Providing these options on the command line overrides the default.
|
||||
|
||||
.. _idf.py-size:
|
||||
|
||||
Advanced Commands
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
|
|
|
|||
|
|
@ -344,6 +344,16 @@ Setting ``BATCH_BUILD`` implies the following:
|
|||
- If the project configuration is missing new configuration items (from new components or esp-idf updates) then the project use the default values, instead of prompting the user for each item.
|
||||
- If the build system needs to invoke ``menuconfig``, an error is printed and the build fails.
|
||||
|
||||
.. _make-size:
|
||||
|
||||
Advanced Make Targets
|
||||
---------------------
|
||||
|
||||
- ``make app``, ``make bootloader``, ``make partition table`` can be used to build only the app, bootloader, or partition table from the project as applicable.
|
||||
- ``make erase_flash`` and ``make erase_ota`` will use esptool.py to erase the entire flash chip and the OTA selection setting from the flash chip, respectively.
|
||||
- ``make size`` prints some size information about the app. ``make size-components`` and ``make size-files`` are similar targets which print more detailed per-component or per-source-file information, respectively.
|
||||
|
||||
|
||||
Debugging The Make Process
|
||||
--------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -36,29 +36,84 @@ Connections for the ESP32D2W* chips are TBD.
|
|||
Espressif sells an ESP-WROVER module which contains an ESP32, 1.8V flash and the ESP-PSRAM32 integrated in a module, ready for inclusion
|
||||
on an end product PCB.
|
||||
|
||||
Software
|
||||
========
|
||||
.. _external_ram_config:
|
||||
|
||||
ESP-IDF fully supports integrating external memory use into your applications. ESP-IDF can be configured to handle external RAM in several ways:
|
||||
* Only initialize RAM. This allows the application to manually place data here by dereferencing pointers pointed at the external RAM memory
|
||||
region (0x3F800000 and up).
|
||||
* Initialize RAM and add it to the capability allocator. This allows a program to specifically allocate a chunk of external RAM using
|
||||
``heap_caps_malloc(size, MALLOC_CAP_SPIRAM)``. This memory can be used and subsequently freed using a normal ``free()`` call.
|
||||
* Initialize RAM, add it to the capability allocator and add memory to the pool of RAM that can be returned by ``malloc()``. This allows
|
||||
any application to use the external RAM without having to rewrite the code to use ``heap_caps_malloc``.
|
||||
* Initialize RAM, use a region start from 0x3F800000 for storing zero initialized data(BSS segment) of lwip,net802.11,pp,bluedroid library
|
||||
by enabling :ref: `CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY` in menuconfig, this way can save some internal memory，because the BSS segment
|
||||
originally stored in internal memory，and the rest of external RAM can be add the capability allocator and add memory to the pool of RAM as above way
|
||||
Configuring External RAM
|
||||
========================
|
||||
|
||||
All these options can be selected from the menuconfig menu.
|
||||
ESP-IDF fully supports using external memory in applications. ESP-IDF can be configured to handle external RAM in several ways after it is initialized at startup:
|
||||
|
||||
* :ref:`external_ram_config_memory_map`
|
||||
* :ref:`external_ram_config_capability_allocator`
|
||||
* :ref:`external_ram_config_malloc` (default)
|
||||
* :ref:`external_ram_config_bss`
|
||||
|
||||
.. _external_ram_config_memory_map:
|
||||
|
||||
Integrate RAM into ESP32 memory map
|
||||
-----------------------------------
|
||||
|
||||
Select this option by choosing "Integrate RAM into ESP32 memory map" from :ref:`CONFIG_SPIRAM_USE`.
|
||||
|
||||
This is the most basic option for external SPIRAM integration. Most users will want one of the other, more advanced, options.
|
||||
|
||||
During ESP-IDF startup, external RAM is mapped into the data address space starting at at address 0x3F800000 (byte-accessible). The length of this region is the same as the SPIRAM size (up to the limit of 4MiB).
|
||||
|
||||
The application can manually place data in external memory by creating pointers to this region. The application is responsible for all management of the external SPIRAM: coordinating buffer usage, preventing corruption, etc.
|
||||
|
||||
.. _external_ram_config_capability_allocator:
|
||||
|
||||
Add external RAM to the capability allocator
|
||||
--------------------------------------------
|
||||
|
||||
Select this option by choosing "Make RAM allocatable using heap_caps_malloc(..., MALLOC_CAP_SPIRAM)" from :ref:`CONFIG_SPIRAM_USE`.
|
||||
|
||||
When enabled, memory is mapped to address 0x3F800000 but also added to the :doc:`capabilities-based heap memory allocator </api-reference/system/mem_alloc>` using ``MALLOC_CAP_SPIRAM``.
|
||||
|
||||
To allocate memory from external RAM, a program should call ``heap_caps_malloc(size, MALLOC_CAP_SPIRAM)``. After use, this memory can be freed by calling the normal ``free()`` function.
|
||||
|
||||
.. _external_ram_config_malloc:
|
||||
|
||||
Provide external RAM via malloc()
|
||||
---------------------------------
|
||||
|
||||
Select this option by choosing "Make RAM allocatable using malloc() as well" from :ref:`CONFIG_SPIRAM_USE`. This is the default selection.
|
||||
|
||||
Using this option, memory is added to the capability allocator as described for the previous option. However it is also added to the pool of RAM that can be returned by standard ``malloc()``.
|
||||
|
||||
This allows any application to use the external RAM without having to rewrite the code to use ``heap_caps_malloc(..., MALLOC_CAP_SPIRAM)``.
|
||||
|
||||
An additional configuration item, :ref:`CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL`, can be used to set the size threshold when a single allocation should prefer external memory:
|
||||
|
||||
- When allocating a size less than the threshold, the allocator will try internal memory first.
|
||||
- When allocating a size equal to or larger than the threshold, the allocator will try external memory first.
|
||||
|
||||
If a suitable block of preferred internal/external memory is not available, allocation will try the other type of memory.
|
||||
|
||||
Because some buffers can only be allocated in internal memory, a second configuration item :ref:`CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL` defines a pool of internal memory which is reserved for *only* explicitly internal allocations (such as memory for DMA use). Regular ``malloc()`` will not allocate from this pool. The :ref:`MALLOC_CAP_DMA <dma-capable-memory>` and ``MALLOC_CAP_INTERNAL`` flags can be used to allocate memory from this pool.
|
||||
|
||||
.. _external_ram_config_bss:
|
||||
|
||||
Allow .bss segment placed in external memory
|
||||
--------------------------------------------
|
||||
|
||||
Enable this option by setting :ref:`CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY`. This configuration setting is independent of the other three.
|
||||
|
||||
If enabled, a region of the address space starting from 0x3F800000 will be to used to store zero initialized data (BSS segment) from the lwip, net80211, libpp and bluedroid ESP-IDF libraries.
|
||||
|
||||
Additional data can be moved from the internal BSS segment to external RAM by applying the ``EXT_RAM_ATTR`` macro to any static declaration (which is not initialized to a non-zero value).
|
||||
|
||||
This option reduces the internal static memory used by the BSS segment.
|
||||
|
||||
Remaining external RAM can also be added to the capability heap allocator, by the method shown above.
|
||||
|
||||
Restrictions
|
||||
------------
|
||||
============
|
||||
|
||||
The use of external RAM has a few restrictions:
|
||||
* When disabling flash cache (for example, because the flash is being written to), the external RAM also becomes inaccessible; any reads from or
|
||||
writes to it will lead to an illegal cache access exception. This is also the reason that ESP-IDF will never allocate a tasks stack in external
|
||||
RAM.
|
||||
External RAM use has the following restrictions:
|
||||
|
||||
* When flash cache is disabled (for example, because the flash is being written to), the external RAM also becomes inaccessible; any reads from or
|
||||
writes to it will lead to an illegal cache access exception. This is also the reason that ESP-IDF does not by default allocate any task stacks in external RAM (see below).
|
||||
* External RAM cannot be used as a place to store DMA transaction descriptors or as a buffer for a DMA transfer to read from or write into. Any
|
||||
buffers that will be used in combination with DMA must be allocated using ``heap_caps_malloc(size, MALLOC_CAP_DMA)`` (and can be freed using a
|
||||
standard ``free()`` call.)
|
||||
|
|
@ -66,22 +121,12 @@ The use of external RAM has a few restrictions:
|
|||
modified almost as quickly as in internal ram. However, when accessing large chunks of data (>32K), the cache can be insufficient and speeds
|
||||
will fall back to the access speed of the external RAM. Moreover, accessing large chunks of data can 'push out' cached flash, possibly making
|
||||
execution of code afterwards slower.
|
||||
* External RAM cannot be used as task stack memory; because of this, xTaskCreate and similar functions will always allocate internal memory
|
||||
for stack and task TCBs and xTaskCreateStatic-type functions will check if the buffers passed are internal. However, for tasks not calling
|
||||
* External RAM cannot be used as task stack memory. Because of this, :cpp:func:`xTaskCreate` and similar functions will always allocate internal memory
|
||||
for stack and task TCBs and functions like :cpp:func:`xTaskCreateStatic` will check if the buffers passed are internal. However, for tasks not calling
|
||||
on code in ROM in any way, directly or indirectly, the menuconfig option :ref:`CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY` will eliminate
|
||||
the check in xTaskCreateStatic, allowing task stack in external RAM. Using this is not advised, however.
|
||||
* External RAM initialized failed can not be ignored if enabled :ref:`CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY`; because of this, some BSS segment
|
||||
can not be placed into external memory if PSRAM can't work normally and can not be moved to internal memory at runtime because the address of
|
||||
them is defined by linkfile, the :ref:`CONFIG_SPIRAM_IGNORE_NOTFOUND` can't handle this situation,if you want to enable :ref:
|
||||
`CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY` the :ref:`CONFIG_SPIRAM_IGNORE_NOTFOUND` will be disabled, and if initialize SPIRAM failed,the system
|
||||
will invoke abort.
|
||||
* External RAM of 4MBit (test up to know) has to be used with one of HSPI/VSPI occupied under 80MHz. Select which SPI host to be used by :ref:`CONFIG_SPIRAM_OCCUPY_SPI_HOST`.
|
||||
|
||||
Because there are a fair few situations that have a specific need for internal memory, but it is also possible to use malloc() to exhaust
|
||||
internal memory, there is a pool reserved specifically for requests that cannot be resolved from external memory; allocating task
|
||||
stack, DMA buffers and memory that stays accessible when cache is disabled is drawn from this pool. The size of this pool is configurable
|
||||
in menuconfig.
|
||||
|
||||
the check in xTaskCreateStatic, allowing a task's stack to be in external RAM. Using this is not advised, however.
|
||||
* By default, failure to initialize external RAM will cause ESP-IDF startup to abort. This can be disabled by enabling config item :ref:`CONFIG_SPIRAM_IGNORE_NOTFOUND`. If :ref:`CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY` is enabled, the option to ignore failure is not available as the linker will have assigned symbols to external memory addresses at link time.
|
||||
* When used at 80MHz clock speed, external RAM must also occupy either the HSPI or VSPI bus. Select which SPI host will be used by :ref:`CONFIG_SPIRAM_OCCUPY_SPI_HOST`.
|
||||
|
||||
Chip revisions
|
||||
==============
|
||||
|
|
|
|||
|
|
@ -1,47 +1,112 @@
|
|||
Heap Memory Allocation
|
||||
======================
|
||||
|
||||
Overview
|
||||
--------
|
||||
Stack and Heap
|
||||
--------------
|
||||
|
||||
The ESP32 has multiple types of RAM. Internally, there's IRAM, DRAM as well as RAM that can be used as both. It's also
|
||||
possible to connect external SPI RAM to the ESP32 - external RAM can be integrated into the ESP32's memory map using
|
||||
the flash cache.
|
||||
ESP-IDF applications use the common computer architecture patterns of *stack* (dynamic memory allocated by program control flow) and *heap* (dynamic memory allocated by function calls), as well as statically allocated memory (allocated at compile time).
|
||||
|
||||
For most purposes, the standard libc ``malloc()`` and ``free()`` functions can be used for heap allocation without any
|
||||
issues.
|
||||
Because ESP-IDF is a multi-threaded RTOS environment, each RTOS task has its own stack. By default, each of these stacks is allocated from the heap when the task is created. (See :cpp:func:`xTaskCreateStatic` for the alternative where stacks are statically allocated.)
|
||||
|
||||
However, in order to fully make use of all of the memory types and their characteristics, esp-idf also has a
|
||||
capabilities-based heap memory allocator. If you want to have memory with certain properties (for example, DMA-capable
|
||||
memory, or executable memory), you can create an OR-mask of the required capabilities and pass that to
|
||||
:cpp:func:`heap_caps_malloc`. For instance, the standard ``malloc()`` implementation internally allocates memory via
|
||||
``heap_caps_malloc(size, MALLOC_CAP_8BIT)`` in order to get data memory that is byte-addressable.
|
||||
Because ESP32 uses multiple types of RAM, it also contains multiple heaps with different capabilities. A capabilities-based memory allocator allows apps to make heap allocations for different purposes.
|
||||
|
||||
Because malloc uses this allocation system as well, memory allocated using :cpp:func:`heap_caps_malloc` can be freed by calling
|
||||
For most purposes, the standard libc ``malloc()`` and ``free()`` functions can be used for heap allocation without any special consideration.
|
||||
|
||||
However, in order to fully make use of all of the memory types and their characteristics, ESP-IDF also has a
|
||||
capabilities-based heap memory allocator. If you want to have memory with certain properties (for example, :ref:`dma-capable-memory` or executable-memory), you can create an OR-mask of the required capabilities and pass that to :cpp:func:`heap_caps_malloc`.
|
||||
|
||||
Memory Capabilities
|
||||
-------------------
|
||||
|
||||
The ESP32 contains multiple types of RAM:
|
||||
|
||||
- DRAM (Data RAM) is memory used to hold data. This is the most common kind of memory accessed as heap.
|
||||
- IRAM (Instruction RAM) usually holds executable data only. If accessed as generic memory, all accesses must be :ref:`32-bit aligned<32-Bit Accessible Memory>`.
|
||||
- D/IRAM is RAM which can be used as either Instruction or Data RAM.
|
||||
|
||||
For more details on these internal memory types, see :ref:`memory-layout`.
|
||||
|
||||
It's also possible to connect external SPI RAM to the ESP32 - :doc:`external RAM </api-guides/external-ram>` can be integrated into the ESP32's memory map using the flash cache, and accessed similarly to DRAM.
|
||||
|
||||
DRAM uses capability ``MALLOC_CAP_8BIT`` (accessible in single byte reads and writes). When calling ``malloc()``, the ESP-IDF ``malloc()`` implementation internally calls ``heap_caps_malloc(size, MALLOC_CAP_8BIT)`` in order to allocate DRAM that is byte-addressable. To test the free DRAM heap size at runtime, call cpp:func:`heap_caps_get_free_size(MALLOC_CAP_8BIT)`.
|
||||
|
||||
Because malloc uses the capabilities-based allocation system, memory allocated using :cpp:func:`heap_caps_malloc` can be freed by calling
|
||||
the standard ``free()`` function.
|
||||
|
||||
The "soc" component contains a list of memory regions for the chip, along with the type of each memory (aka its tag) and the associated capabilities for that memory type. On startup, a separate heap is initialised for each contiguous memory region. The capabilities-based allocator chooses the best heap for each allocation, based on the requested capabilities.
|
||||
Available Heap
|
||||
--------------
|
||||
|
||||
Special Uses
|
||||
------------
|
||||
DRAM
|
||||
^^^^
|
||||
|
||||
At startup, the DRAM heap contains all data memory which is not statically allocated by the app. Reducing statically allocated buffers will increase the amount of available free heap.
|
||||
|
||||
To find the amount of statically allocated memory, use the :ref:`make size <make-size>` or :ref:`idf.py size <idf.py-size>` (for CMake) command.
|
||||
|
||||
.. note:: Due to a technical limitation, the maximum statically allocated DRAM usage is 160KB. The remaining 160KB (for a total of 320KB of DRAM) can only be allocated at runtime as heap.
|
||||
|
||||
.. note:: At runtime, the available heap DRAM may be less than calculated at compile time, because at startup some memory is allocated from the heap before the FreeRTOS scheduler is started (including memory for the stacks of initial FreeRTOS tasks).
|
||||
|
||||
IRAM
|
||||
^^^^
|
||||
|
||||
At startup, the IRAM heap contains all instruction memory which is not used by the app executable code.
|
||||
|
||||
The :ref:`make size <make-size>` and :ref:`idf.py size <idf.py-size>` commands can be used to find the amount of IRAM used by the app.
|
||||
|
||||
D/IRAM
|
||||
^^^^^^
|
||||
|
||||
Some memory in the ESP32 is available as either DRAM or IRAM. If memory is allocated from a D/IRAM region, the free heap size for both types of memory will decrease.
|
||||
|
||||
Heap Sizes
|
||||
^^^^^^^^^^
|
||||
|
||||
At startup, all ESP-IDF apps log a summary of all heap addresses (and sizes) at level Info:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
I (252) heap_init: Initializing. RAM available for dynamic allocation:
|
||||
I (259) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
|
||||
I (265) heap_init: At 3FFB2EC8 len 0002D138 (180 KiB): DRAM
|
||||
I (272) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
|
||||
I (278) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
|
||||
I (284) heap_init: At 4008944C len 00016BB4 (90 KiB): IRAM
|
||||
|
||||
Finding available heap
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
See :ref:`heap-information`.
|
||||
|
||||
Special Capabilities
|
||||
--------------------
|
||||
|
||||
.. _dma-capable-memory:
|
||||
|
||||
DMA-Capable Memory
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Use the MALLOC_CAP_DMA flag to allocate memory which is suitable for use with hardware DMA engines (for example SPI and I2S). This capability flag excludes any external PSRAM.
|
||||
Use the ``MALLOC_CAP_DMA`` flag to allocate memory which is suitable for use with hardware DMA engines (for example SPI and I2S). This capability flag excludes any external PSRAM.
|
||||
|
||||
.. _32-bit accessible memory:
|
||||
|
||||
32-Bit Accessible Memory
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If a certain memory structure is only addressed in 32-bit units, for example an array of ints or pointers, it can be
|
||||
useful to allocate it with the MALLOC_CAP_32BIT flag. This also allows the allocator to give out IRAM memory; something
|
||||
useful to allocate it with the ``MALLOC_CAP_32BIT`` flag. This also allows the allocator to give out IRAM memory; something
|
||||
which it can't do for a normal malloc() call. This can help to use all the available memory in the ESP32.
|
||||
|
||||
Memory allocated with MALLOC_CAP_32BIT can *only* be accessed via 32-bit reads and writes, any other type of access will
|
||||
Memory allocated with ``MALLOC_CAP_32BIT`` can *only* be accessed via 32-bit reads and writes, any other type of access will
|
||||
generate a fatal LoadStoreError exception.
|
||||
|
||||
External SPI RAM under 4MiB in size can be allocated using standard ``malloc`` calls, if that is enabled in menuconfig. To
|
||||
use the region above the 4MiB limit, you can use the :doc:`himem API</api-reference/system/himem>`.
|
||||
External SPI Memory
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
When :doc:`external RAM </api-guides/external-ram>` is enabled, external SPI RAM under 4MiB in size can be allocated using standard ``malloc`` calls, or via ``heap_caps_malloc(MALLOC_CAP_SPIRAM)``, depending on configuration. See :ref:`external_ram_config` for more details.
|
||||
|
||||
To use the region above the 4MiB limit, you can use the :doc:`himem API</api-reference/system/himem>`.
|
||||
|
||||
|
||||
API Reference - Heap Allocation
|
||||
-------------------------------
|
||||
|
|
@ -65,11 +130,13 @@ API Reference - Initialisation
|
|||
Implementation Notes
|
||||
--------------------
|
||||
|
||||
Knowledge about the regions of memory in the chip comes from the "soc" component, which contains memory layout information for the chip.
|
||||
Knowledge about the regions of memory in the chip comes from the "soc" component, which contains memory layout information for the chip, and the different capabilities of each region. Each region's capabilities are prioritised, so that (for example) dedicated DRAM and IRAM regions will be used for allocations ahead of the more versatile D/IRAM regions.
|
||||
|
||||
Each contiguous region of memory contains its own memory heap. The heaps are created using the `multi_heap <API Reference - Multi Heap API>`_ functionality. multi_heap allows any contiguous region of memory to be used as a heap.
|
||||
|
||||
The heap capabilities allocator uses knowledge of the memory regions to initialize each individual heap. When you call a function in the heap capabilities API, it will find the most appropriate heap for the allocation (based on desired capabilities, available space, and preferences for each region's use) and then call the multi_heap function to use the heap situation in that particular region.
|
||||
The heap capabilities allocator uses knowledge of the memory regions to initialize each individual heap. Allocation functions in the heap capabilities API will find the most appropriate heap for the allocation (based on desired capabilities, available space, and preferences for each region's use) and then calling :cpp:func:`multi_heap_malloc` or :cpp:func:`multi_heap_calloc` for the heap situated in that particular region.
|
||||
|
||||
Calling ``free()`` involves finding the particular heap corresponding to the freed address, and then calling :cpp:func:`multi_heap_free` on that particular multi_heap instance.
|
||||
|
||||
API Reference - Multi Heap API
|
||||
------------------------------
|
||||
|
|
|
|||
Loading…
Reference in a new issue