| .. _build_overview: |
| |
| Build Overview |
| ############## |
| |
| The Zephyr build process can be divided into two main phases: a configuration |
| phase (driven by CMake) and a build phase (driven by Make or Ninja). |
| |
| .. _build_configuration_phase: |
| |
| Configuration Phase |
| ******************* |
| |
| The configuration phase begins when the user invokes *CMake*, |
| specifying a source application directory and a board target. |
| |
| .. figure:: build-config-phase.svg |
| :align: center |
| :alt: Zephyr's build configuration phase |
| :figclass: align-center |
| :width: 80% |
| |
| CMake begins by processing the :file:`CMakeLists.txt` file in the application |
| directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr |
| top-level directory, which in turn refers to :file:`CMakeLists.txt` files |
| throughout the build tree (directly and indirectly). Its primary output is a |
| set of Makefiles or Ninja files to drive the build process, but the CMake |
| scripts also do some processing of their own: |
| |
| Devicetree |
| :file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source |
| include*) files are collected from the target's architecture, SoC, board, |
| and application directories. |
| |
| :file:`*.dtsi` files are included by :file:`*.dts` files via the C |
| preprocessor (often abbreviated *cpp*, which should not be confused with |
| C++). The C preprocessor is also used to merge in any devicetree |
| :file:`*.overlay` files, and to expand macros in :file:`*.dts`, |
| :file:`*.dtsi`, and :file:`*.overlay` files. |
| |
| The preprocessed devicetree sources (stored in :file:`*.dts.pre.tmp`) are |
| parsed by :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to |
| generate a :file:`devicetree_unfixed.h` header with preprocessor macros. |
| |
| As a debugging aid, :file:`gen_defines.py` writes the final devicetree to |
| :file:`zephyr.dts`. This file is just for reference. It is not used |
| anywhere. |
| |
| The ``dtc`` devicetree compiler also gets run on the preprocessed devicetree |
| sources to catch any extra warnings and errors generated by it. The output |
| from ``dtc`` is unused otherwise. |
| |
| The above is just a brief overview. For more information on devicetree, see |
| :ref:`dt-guide`. |
| |
| Devicetree fixups |
| Files named :file:`dts_fixup.h` from the target’s architecture, SoC, board, |
| and application directories are concatenated into a single |
| :file:`devicetree_fixups.h` file. :file:`dts_fixup.h` files are used to |
| rename generated macros to names expected by the source code. |
| |
| Source code accesses preprocessor macros generated from devicetree by |
| including the :zephyr_file:`devicetree.h <include/devicetree.h>` header, |
| which includes :file:`devicetree_unfixed.h` and :file:`devicetree_fixups.h`. |
| |
| Kconfig |
| :file:`Kconfig` files define available configuration options for for the |
| target architecture, SoC, board, and application, as well as dependencies |
| between options. |
| |
| Kconfig configurations are stored in *configuration files*. The initial |
| configuration is generated by merging configuration fragments from the board |
| and application (e.g. :file:`prj.conf`). |
| |
| The output from Kconfig is an :file:`autoconf.h` header with preprocessor |
| assignments, and a :file:`.config` file that acts both as a saved |
| configuration and as configuration output (used by CMake). |
| |
| Information from devicetree is available to Kconfig, through the functions |
| defined in :zephyr_file:`kconfigfunctions.py |
| <scripts/kconfig/kconfigfunctions.py>`. |
| |
| See :ref:`the Kconfig section of the manual <kconfig>` for more information. |
| |
| Build Phase |
| *********** |
| |
| The build phase begins when the user invokes ``make`` or ``ninja``. Its |
| ultimate output is a complete Zephyr application in a format suitable for |
| loading/flashing on the desired target board (:file:`zephyr.elf`, |
| :file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually, |
| into four stages: the pre-build, first-pass binary, final binary, and |
| post-processing. |
| |
| Pre-build occurs before any source files are compiled, because during |
| this phase header files used by the source files are generated. |
| |
| Pre-build |
| ========= |
| |
| Offset generation |
| Access to high-level data structures and members is sometimes |
| required when the definitions of those structures is not |
| immediately accessible (e.g., assembly language). The generation of |
| *offsets.h* (by *gen_offset_header.py*) facilitates this. |
| |
| System call boilerplate |
| The *gen_syscall.py* and *parse_syscalls.py* scripts work |
| together to bind potential system call functions with their |
| implementations. |
| |
| .. figure:: build-build-phase-1.svg |
| :align: center |
| :alt: Zephyr's build stage I |
| :figclass: align-center |
| :width: 80% |
| |
| First-pass binary |
| ================= |
| |
| Compilation proper begins with the first-pass binary. Source files (C |
| and assembly) are collected from various subsystems (which ones is |
| decided during the configuration phase), and compiled into archives |
| (with reference to header files in the tree, as well as those |
| generated during the configuration phase and the pre-build stage). |
| |
| If memory protection is enabled, then: |
| |
| Partition grouping |
| The gen_app_partitions.py script scans all the |
| generated archives and outputs linker scripts to ensure that |
| application partitions are properly grouped and aligned for the |
| target’s memory protection hardware. |
| |
| Then *cpp* is used to combine linker script fragments from the target’s |
| architecture/SoC, the kernel tree, optionally the partition output if |
| memory protection is enabled, and any other fragments selected during |
| the configuration process, into a *linker.cmd* file. The compiled |
| archives are then linked with *ld* as specified in the |
| *linker.cmd*. |
| |
| In some configurations, this is the final binary, and the next stage |
| is skipped. |
| |
| .. figure:: build-build-phase-2.svg |
| :align: center |
| :alt: Zephyr's build stage II |
| :figclass: align-center |
| :width: 80% |
| |
| Final binary |
| ============ |
| |
| In some configurations, the binary from the previous stage is |
| incomplete, with empty and/or placeholder sections that must be filled |
| in by, essentially, reflection. When :ref:`usermode_api` is enabled: |
| |
| Kernel object hashing |
| The *gen_kobject_list.py* scans the *ELF DWARF* |
| debug data to find the address of the all kernel objects. This |
| list is passed to *gperf*, which generates a perfect hash function and |
| table of those addresses, then that output is optimized by |
| *process_gperf.py*, using known properties of our special case. |
| |
| Then, the link from the previous stage is repeated, this time with the |
| missing pieces populated. |
| |
| .. figure:: build-build-phase-3.svg |
| :align: center |
| :alt: Zephyr's build stage III |
| :figclass: align-center |
| :width: 80% |
| |
| |
| Post processing |
| =============== |
| |
| Finally, if necessary, the completed kernel is converted from *ELF* to |
| the format expected by the loader and/or flash tool required by the |
| target. This is accomplished in a straightforward manner with *objdump*. |
| |
| .. figure:: build-build-phase-4.svg |
| :align: center |
| :alt: Zephyr's build final stage |
| :figclass: align-center |
| :width: 80% |
| |
| |
| .. _build_system_scripts: |
| |
| Supporting Scripts and Tools |
| **************************** |
| |
| The following is a detailed description of the scripts used during the build process. |
| |
| .. _gen_syscalls.py: |
| |
| :zephyr_file:`scripts/gen_syscalls.py` |
| ======================================== |
| |
| .. include:: ../../../scripts/gen_syscalls.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _gen_kobject_list.py: |
| |
| :zephyr_file:`scripts/gen_kobject_list.py` |
| ========================================== |
| |
| .. include:: ../../../scripts/gen_kobject_list.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _gen_offset_header.py: |
| |
| :zephyr_file:`scripts/gen_offset_header.py` |
| =========================================== |
| |
| .. include:: ../../../scripts/gen_offset_header.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _parse_syscalls.py: |
| |
| :zephyr_file:`scripts/parse_syscalls.py` |
| ======================================== |
| |
| |
| .. include:: ../../../scripts/parse_syscalls.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _gen_idt.py: |
| |
| :zephyr_file:`arch/x86/gen_idt.py` |
| ================================== |
| |
| .. include:: ../../../arch/x86/gen_idt.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _gen_gdt.py: |
| |
| :zephyr_file:`arch/x86/gen_gdt.py` |
| =================================== |
| |
| .. include:: ../../../arch/x86/gen_gdt.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _gen_relocate_app.py: |
| |
| :zephyr_file:`scripts/gen_relocate_app.py` |
| =========================================== |
| |
| .. include:: ../../../scripts/gen_relocate_app.py |
| :start-after: """ |
| :end-before: """ |
| |
| .. _process_gperf.py: |
| |
| :zephyr_file:`scripts/process_gperf.py` |
| ======================================== |
| |
| .. include:: ../../../scripts/process_gperf.py |
| :start-after: """ |
| :end-before: """ |