Crash_Override/OVMS3-idf
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
OVMS3-idf/docs/build_system.rst
Angus Gratton fce359b240 build system: Add support for embedded arbitrary binary or text files in .rodata 
Simplifies examples of embedding a certificate file or a root cert.

This is a much cruder mechanism than the full flash filesystem we want
eventually, but still sometimes useful.
2016-11-08 11:13:54 +11:00

341 lines
15 KiB
ReStructuredText
Raw Blame History

Build System
************
This document explains the Espressif IoT Development Framework build system and the
concept of "components"
Read this document if you want to know how to organise a new ESP-IDF project.
We recommend using the esp-idf-template_ project as a starting point for your project.
Overview
========
An ESP-IDF project can be seen as an almagation of a number of components.
For example, for a webserver that shows the current humidity, we would
have:
- The ESP32 base libraries (libc, rom bindings etc)
- The WiFi drivers
- A TCP/IP stack
- The FreeRTOS operating system
- A webserver
- A driver for an humidity sensor
- Main code tying it all together
ESP-IDF makes these components explicit and configurable. To do that, when a project
is compiled, the build environment will look up all the components in the
ESP-IDF directories, the project directories and optionally custom other component
directories. It then allows the user to configure compile-time options using
a friendly text-based menu system to customize the ESP-IDF as well as other components
to the requirements of the project. After the components are customized, the
build process will compile everything into an output file, which can then be uploaded
into a board in a way that can also be defined by components.
A project in this sense is defined as a directory under which all the files required
to build it live, excluding the ESP-IDF files and the toolchain. A simple project
tree might look like this::
- myProject/ - build/
- components/ - component1/ - component.mk
- Kconfig
- src1.c
- component2/ - component.mk
- Kconfig
- src1.c
- main/ - src1.c
- src2.c
- Makefile
As we can see, a project consists of a components/ subdirectory containing its
components as well as one or more directories containing the project-specific
sources; by default a single directory called 'main' is assumed. The project
directory will also have a Makefile where the projects name as well as optionally
other options are defined. After compilation, the project directory will contain
a 'build'-directory containing all of the objects, libraries and other generated
files as well as the final binary.
Components also have a custom makefile - ``component.mk``. This contains various definititions
influencing the build process of the component as well as the project it's used
in. Components may also include a Kconfig file defining the compile-time options that are
settable by means of the menu system.
Project Makefile variables that can be set by the programmer::
PROJECT_NAME: Mandatory. Name for the project
BUILD_DIR_BASE: Set the directory where all objects/libraries/binaries end up in.
Defaults to $(PROJECT_PATH)/build
COMPONENT_DIRS: Search path for components. Defaults to the component/ directories
in the ESP-IDF path and the project path.
COMPONENTS: A list of component names. Defaults to all the component found in the
COMPONENT_DIRS directory
EXTRA_COMPONENT_DIRS: Defaults to unset. Use this to add directories to the default
COMPONENT_DIRS.
SRCDIRS: Directories under the project dir containing project-specific sources.
Defaults to 'main'. These are treated as 'lite' components: they do not have
include directories that are passed to the compilation pass of all components and
they do not have a Kconfig option.
Component-specific component.mk variables that can be set by the programmer::
COMPONENT_ADD_INCLUDEDIRS: Relative path to include directories to be added to
the entire project. If an include directory is only needed to compile this
specific component, don't add it here.
COMPONENT_PRIV_INCLUDEDIRS: Relative path to include directories that are only used
when compiling this specific component.
COMPONENT_DEPENDS: Names of any components that need to be compiled before this component.
COMPONENT_ADD_LDFLAGS: LD flags to add for the entire project. Defaults to -l$(COMPONENT_NAME).
Add libraries etc in the current directory as $(abspath libwhatever.a)
COMPONENT_EXTRA_INCLUDES: Any extra include paths used when compiling the component's
source files. These will be prefixed with '-I' and passed to the compiler.
Similar to COMPONENT_PRIV_INCLUDEDIRS, but these paths are passed as-is instead of
expanded relative to the component directory.
COMPONENT_SRCDIRS: Relative directories to look in for sources. Defaults to '.', the current
directory (the root of the component) only. Use this to specify any subdirectories. Note
that specifying this overwrites the default action of compiling everything in the
components root dir; to keep this behaviour please also add '.' as a directory in this
list.
COMPONENT_OBJS: Object files to compile. Defaults to the .o variants of all .c and .S files
that are found in COMPONENT_SRCDIRS.
COMPONENT_EXTRA_CLEAN: Files that are generated using rules in the components Makefile
that also need to be cleaned
COMPONENT_BUILDRECIPE: Recipe to build the component. Optional. Defaults to building all
COMPONENT_OBJS and linking them into lib(componentname).a
COMPONENT_CLEANRECIPE: Recipe to clean the component. Optional. Defaults to removing
all built objects and libraries.
COMPONENT_BUILD_DIR: Equals the cwd of the component build, which is the build dir
of the component (where all the .o etc files should be created).
These variables are already set early on in the Makefile and the values in it will
be usable in component or project Makefiles::
CC, LD, AR, OBJCOPY: Xtensa gcc tools
HOSTCC, HOSTLD etc: Host gcc tools
LDFLAGS, CFLAGS: Set to usable values as defined in ESP-IDF Makefile
PROJECT_NAME: Name of the project, as set in project makefile
PROJECT_PATH: Path to the root of the project folder
COMPONENTS: Name of the components to be included
CONFIG_*: All values set by 'make menuconfig' have corresponding Makefile variables.
Inside your component's component.mk makefile, you can override or add to these variables
as necessary. The changes are isolated from other components (see Makefile.projbuild below
if you want to share these changes with all other components.)
For components, there also are these defines::
COMPONENT_PATH: Absolute path to the root of the source tree of the component we're
compiling
COMPONENT_LIBRARY: The full path to the static library the components compilation pass
is supposed to generate
Make Process
------------
The Make process is always invoked from the project directory by the
user; invoking it anywhere else gives an error. This is what happens if
we build a binary:
The Makefile first determines how it was included. It figures out
various paths as well as the components available to it. It will also
collect the ldflags and includes that the components specify they need.
It does this by running a dummy make on the components with a "get_variable"
target that will output these values.
The Makefile will then create targets to build the lib*.a libraries of
all components and make the elf target depend on this. The main Makefile
invokes Make on the componen.mk of each components inside a sub-mke: this way
the components have full freedom to do whatever is necessary to build
the library without influencing other components. By default, the
component.mk includes the utility makefile $(IDF_PATH)/make/component_common.mk.
This provides default targets and configurations that will work
out-of-the-box for most projects.
KConfig
-------
Each component can also have a Kconfig file, alongside the component.mk, that contains
details to add to "menuconfig" for this component.
Makefile.projbuild
------------------
For components that have parts that need to be evaluated in the top-level
project context, you can create a file called Makefile.projbuild in the
component root directory. These files is included into the project's
top-level Makefile.
For example, if your component needs to add to CFLAGS for the entire
project (not just for its own source files) then you can set
``CFLAGS +=`` in Makefile.projbuild. Note that this isn't necessary for
adding include directories to the project, you can set
``COMPONENT_ADD_INCLUDEDIRS`` (see above) in the component.mk.
KConfig.projbuild
-----------------
There's an equivalent to Makefile.projbuild for KConfig: if you want to include
options at the top-level, not inside the 'components' submenu then create a Kconfig.projbuild and
it will be included in the main menu of menuconfig.
Take good care when (re)defining stuff here: because it's included with all the other
.projbuild files, it's possible to overwrite variables or re-declare targets defined in
the ESP-IDF makefile/Kconfig and other .projbuild files. It's generally better to just
create a KConfig file, if you can.
Writing Component Makefiles
---------------------------
A component consists of a directory which doubles as the name for the
component: a component named 'httpd' lives in a directory called 'httpd'
Because components usually live under the project directory (although
they can also reside in an other folder), the path to this may be
something like /home/myuser/projects/myprojects/components/httpd .
Components can have any name (unique to the project) but the name
cannot contain spaces (esp-idf does not support spaces in paths).
One of the things that most components will have is a component.mk makefile,
containing instructions on how to build the component. Because the
build environment tries to set reasonable defaults that will work most
of the time, component.mk can be very small.
Simplest component.mk
=====================
At the minimum, component.mk will just include the ESP-IDF component "common" makefile,
which adds common component functionality::
include $(IDF_PATH)/make/component_common.mk
This will take all the .c and .S files in the component root and compile
them into object files, finally linking them into a library.
Adding source directories
=========================
By default, subdirectories are ignored. If your project has sources in subdirectories
instead of in the root of the component then you can tell that to the build
system by setting COMPONENT_SRCDIRS::
COMPONENT_SRCDIRS := src1 src2
include $(IDF_PATH)/make/component_common.mk
This will compile all source files in the src1/ and src2/ subdirectories
instead.
Specifying source files
=======================
The standard component.mk logic adds all .S and .c files in the source
directories as sources to be compiled unconditionally. It is possible
to circumvent that logic and hardcode the objects to be compiled by
manually setting the COMPONENT_OBJS variable to the name of the
objects that need to be generated::
COMPONENT_OBJS := file1.o file2.o thing/filea.o thing/fileb.o anotherthing/main.o
include $(IDF_PATH)/make/component_common.mk
Adding conditional configuration
================================
The configuration system can be used to conditionally compile some files
dependending on the options selected in ``make menuconfig``:
Kconfig::
config FOO_ENABLE_BAR
bool "Enable the BAR feature."
help
This enables the BAR feature of the FOO component.
Makefile::
COMPONENT_OBJS := foo_a.o foo_b.o $(if $(CONFIG_FOO_ENABLE_BAR),foo_bar.o foo_bar_interface.o)
include $(IDF_PATH)/make/component_common.mk
Source Code Generation
======================
Some components will have a situation where a source file isn't supplied
with the component itself but has to be generated from another file. Say
our component has a header file that consists of the converted binary
data of a BMP file, converted using a hypothetical tool called bmp2h. The
header file is then included in as C source file called graphics_lib.c::
COMPONENT_EXTRA_CLEAN := logo.h
graphics_lib.o: logo.h
logo.h: $(COMPONENT_PATH)/logo.bmp
bmp2h -i $^ -o $@
include $(IDF_PATH)/make/component_common.mk
In this example, graphics_lib.o and logo.h will be generated in the
current directory (the build directory) while logo.bmp comes with the
component and resides under the component path. Because logo.h is a
generated file, it needs to be cleaned when make clean is called which
why it is added to the COMPONENT_EXTRA_CLEAN variable.
Embedding Binary Data
=====================
Sometimes you have a file with some binary or text data that you'd like to make available to your component - but you don't want to reformat the file as C source.
You can set a variable COMPONENT_EMBED_FILES in component.mk, giving the names of the files to embed in this way::
COMPONENT_EMBED_FILES := server_root_cert.der
Or if the file is a string, you can use the variable COMPONENT_EMBED_TXTFILES. This will embed the contents of the text file as a null-terminated string::
COMPONENT_EMBED_TXTFILES := server_root_cert.pem
The file's contents will be added to the .rodata section in flash, and are available via symbol names as follows::
extern const uint8_t server_root_cert_pem_start[] asm("_binary_server_root_cert_pem_start");
extern const uint8_t server_root_cert_pem_end[] asm("_binary_server_root_cert_pem_end");
The names are generated from the full name of the file, as given in COMPONENT_EMBED_FILES. Characters /, ., etc. are replaced with underscores. The _binary prefix in the symbol name is added by objcopy and is the same for both text and binary files.
For an example of using this technique, see examples/04_https_request - the certificate file contents are loaded from the text .pem file at compile time.
Cosmetic Improvements
=====================
The above example will work just fine, but there's one last cosmetic
improvement that can be done. The make system tries to make the make
process somewhat easier on the eyes by hiding the commands (unless you
run make with the V=1 switch) and this does not do that yet. Here's an
improved version that will output in the same style as the rest of the
make process::
COMPONENT_EXTRA_CLEAN := test_tjpgd_logo.h
graphics_lib.o: logo.h
logo.h: $(COMPONENT_PATH)/logo.bmp
$(summary) BMP2H $@
$(Q) bmp2h -i $^ -o $@
include $(IDF_PATH)/make/component_common.mk
Fully Overriding The Component Makefile
---------------------------------------
Obviously, there are cases where all these recipes are insufficient for a
certain component, for example when the component is basically a wrapper
around another third-party component not originally intended to be
compiled under this build system. In that case, it's possible to forego
the build system entirely by setting COMPONENT_OWNBUILDTARGET and
possibly COMPONENT_OWNCLEANTARGET and defining your own build- and clean
target. The build target can do anything as long as it creates
$(COMPONENT_LIBRARY) for the main file to link into the project binary,
and even that is not necessary: if the COMPONENT_ADD_LDFLAGS variable
is set, the component can instruct the linker to do anything else as well.
.. _esp-idf-template: https://github.com/espressif/esp-idf-template
Reference in a new issue View git blame Copy permalink