pw_ide/docs.rst - pigweed/pigweed - Git at Google

 .. _module-pw_ide:

 ------
 pw_ide
 ------
 This module provides tools for supporting code editor and IDE features for
 Pigweed projects.

 Usage
 =====

 Setup
 -----
 Most of the time, ``pw ide sync`` is all you need to get started.

 .. _module-pw_ide-configuration:

 Configuration
 -------------
 ``pw_ide`` has a built-in default configuration. You can create a configuration
 file if you need to override those defaults.

 A project configuration can be defined in ``.pw_ide.yaml`` in the project root.
 This configuration will be checked into source control and apply to all
 developers of the project. Each user can also create a ``.pw_ide.user.yaml``
 file that overrides both the default and project settings, is not checked into
 source control, and applies only to that checkout of the project. All of these
 files have the same schema, in which these options can be configured:

 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.working_dir
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.compdb_gen_cmd
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.compdb_search_paths
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.target_inference
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.targets_include
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.targets_exclude
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.default_target
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.cascade_targets
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.sync
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.clangd_additional_query_drivers
 .. autoproperty:: pw_ide.settings.PigweedIdeSettings.editors

 C++ Code Intelligence
 ---------------------
 `clangd <https://clangd.llvm.org/>`_ is a language server that provides C/C++
 code intelligence features to any editor that supports the language server
 protocol (LSP). It uses a
 `compilation database <https://clang.llvm.org/docs/JSONCompilationDatabase.html>`_,
 a JSON file containing the compile commands for the project. Projects that have
 multiple targets and/or use multiple toolchains need separate compilation
 databases for each target toolchain. ``pw_ide`` provides tools for managing
 those databases.

 Assuming you have one or more compilation databases that have been generated by
 your build system, start with:

 .. code-block:: bash

    pw ide sync

 This command will:

 - Find every compilation database in your build directory

 - Analyze each database

   - If a database is internally consistent (i.e., it only contains valid
     compile commands for a single target), it will use that database as-is for
     the target toolchain that database pertains to. This is the typical case for
     CMake builds.

   - Otherwise, if a database contains commands for multiple target toolchains
     and/or contains invalid compile commands, the database will be processed,
     yielding one new compilation database for each target toolchain. Those
     databases will be used instead of the original.

 - Link each target to its respective compilation database

 Now, you can list the available target toolchains with:

 .. code-block:: bash

    pw ide cpp --list

 Then set the target toolchain that ``clangd`` should use with:

 .. code-block:: bash

    pw ide cpp --set <selected target name>

 ``clangd`` will now work as designed since it is configured to use a compilation
 database that is consistent to just a single target toolchain.

 ``clangd`` must be run with arguments that provide the Pigweed environment paths
 to the correct toolchains and sysroots. One way to do this is to launch your
 editor from the terminal in an activated environment (e.g. running ``vim`` from
 the terminal), in which case nothing special needs to be done as long as your
 toolchains are in the Pigweed environment or ``$PATH``. But if you launch your
 editor outside of the activated environment (e.g. launching Visual Studio Code
 from your GUI shell's launcher), you can generate the command that invokes
 ``clangd`` with the right arguments with:

 .. code-block:: bash

    pw ide cpp --clangd-command

 Python Code Intelligence
 ------------------------
 Any Python language server should work well with Pigweed projects as long as
 it's configured to use the Pigweed virtual environment. You can output the path
 to the virtual environment on your system with:

 .. code-block:: bash

    pw ide python --venv

 Docs Code Intelligence
 ----------------------
 The `esbonio <https://github.com/swyddfa/esbonio>`_ language server will provide
 code intelligence for RestructuredText and Sphinx. It works well with Pigweed
 projects as long as it is pointed to Pigweed's Python virtual environment. For
 Visual Studio Code, simply install the esbonio extension, which will be
 recommended to you after setting up ``pw_ide``. Once it's installed, a prompt
 will ask if you want to automatically install esbonio in your Pigweed Python
 environment. After that, give esbonio some time to index, then you're done!

 Command-Line Interface Reference
 --------------------------------
 .. argparse::
    :module: pw_ide.cli
    :func: _build_argument_parser
    :prog: pw ide

 Design
 ======

 Supporting ``clangd`` for Embedded Projects
 -------------------------------------------
 There are three main challenges that often prevent ``clangd`` from working
 out-of-the-box with embedded projects:

 #. Embedded projects cross-compile using alternative toolchains, rather than
    using the system toolchain. ``clangd`` doesn't know about those toolchains
    by default.

 #. Embedded projects (particularly Pigweed project) often have *multiple*
    targets that use *multiple* toolchains. Most build systems that generate
    compilation databases put all compile commands in a single database, meaning
    a single file can have multiple, conflicting compile commands. ``clangd``
    will typically use the first one it finds, which may not be the one you want.

 #. Pigweed projects have build steps that use languages other than C/C++. These
    steps are not relevant to ``clangd`` but many build systems will include them
    in the compilation database anyway.

 To deal with these challenges, ``pw_ide`` processes the compilation database you
 provide, yielding one or more compilation databases that are valid, consistent,
 and specific to a particular target toolchain. This enables code intelligence
 and navigation features that reflect that build.

 After processing a compilation database, ``pw_ide`` knows what target toolchains
 are available and provides tools for selecting which target toolchain is active.
 These tools can be integrated into code editors, but are ultimately CLI-driven
 and editor-agnostic. Enabling code intelligence in your editor may be as simple
 as configuring its language server protocol client to use the ``clangd`` command
 that ``pw_ide`` can generate for you.

 When to provide additional configuration to support your use cases
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 The default configuration for ``clangd`` in ``pw_ide`` should work without
 additional configuration as long as you're using only toolchains provided by
 Pigweed and your native host toolchain. If you're using other toolchains, keep
 reading.

 ``clangd`` needs two pieces of information to use a toolchain:

 #. A path to the compiler, which will be taken from the compile command.

 #. The same compiler to be reflected in the
    `query driver <https://releases.llvm.org/10.0.0/tools/clang/tools/extra/docs/clangd/Configuration.html>`_
    argument provided when running ``clangd``.

 When using ``pw_ide`` with external toolchains, you only need to add a path to
 the compiler to ``clangd_additional_query_drivers`` in your project's
 ``pw_ide.yaml`` file. When processing a compilation database, ``pw_ide`` will
 use the query driver globs to find your compiler and configure ``clangd`` to
 use it.

 Compiler wrappers
 ^^^^^^^^^^^^^^^^^
 If you're using ``ccache`` or any other wrapper command that is configured
 using ``ccache``'s' ``KEY=VALUE`` pattern, it will work out of the box.

 Selected API Reference
 ^^^^^^^^^^^^^^^^^^^^^^
 .. automodule:: pw_ide.cpp
    :members: CppCompileCommand, CppCompilationDatabase, CppCompilationDatabasesMap, CppIdeFeaturesState, path_to_executable, ClangdSettings

 Automated Support for Code Editors & IDEs
 -----------------------------------------
 ``pw_ide`` provides a consistent framework for automatically applying settings
 for code editors, where default settings can be defined within ``pw_ide``,
 which can be overridden by project settings, which in turn can be overridden
 by individual user settings.

 .. _module-pw_ide-vscode:

 Visual Studio Code
 ^^^^^^^^^^^^^^^^^^
 Running ``pw ide sync`` will automatically generate settings for Visual Studio
 Code. ``pw_ide`` comes with sensible defaults for Pigweed projects, but those
 can be augmented or overridden at the project level or the user level using
 ``pw_project_settings.json`` and ``pw_user_settings.json`` respectively. The
 generated ``settings.json`` file is essentially a build artifact and shouldn't
 be committed to source control.

 The same pattern applies to ``tasks.json``, which provides Visual Studio Code
 tasks for ``pw_ide`` commands. Access these by opening the command palette
 (Ctrl/Cmd-Shift-P), selecting ``Tasks: Run Task``, then selecting the desired
 task.

 The same pattern also applies to ``launch.json``, which is used to define
 configurations for running and debugging your project. Create a
 ``pw_project_launch.json`` with configurations that conform to the Visual Studio
 Code `debugger configuration format <https://code.visualstudio.com/docs/editor/debugging>`_.

 .. tip::

    What's the difference between "Change C++ Code Analysis Target" and "Set C++
    Code Analyis Target"? "Set" will automatically restart the ``clangd``
    language server for you to pick up the changed target immediately, while
    "Change" will not.

 Selected API Reference
 ^^^^^^^^^^^^^^^^^^^^^^
 .. automodule:: pw_ide.editors
    :members: EditorSettingsDefinition, EditorSettingsFile, EditorSettingsManager

 .. automodule:: pw_ide.vscode
    :members: VscSettingsType, VscSettingsManager
	.. _module-pw_ide:

	------
	pw_ide
	------
	This module provides tools for supporting code editor and IDE features for
	Pigweed projects.

	Usage
	=====

	Setup
	-----
	Most of the time, ``pw ide sync`` is all you need to get started.

	.. _module-pw_ide-configuration:

	Configuration
	-------------
	``pw_ide`` has a built-in default configuration. You can create a configuration
	file if you need to override those defaults.

	A project configuration can be defined in ``.pw_ide.yaml`` in the project root.
	This configuration will be checked into source control and apply to all
	developers of the project. Each user can also create a ``.pw_ide.user.yaml``
	file that overrides both the default and project settings, is not checked into
	source control, and applies only to that checkout of the project. All of these
	files have the same schema, in which these options can be configured:

	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.working_dir
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.compdb_gen_cmd
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.compdb_search_paths
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.target_inference
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.targets_include
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.targets_exclude
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.default_target
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.cascade_targets
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.sync
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.clangd_additional_query_drivers
	.. autoproperty:: pw_ide.settings.PigweedIdeSettings.editors

	C++ Code Intelligence
	---------------------
	`clangd <https://clangd.llvm.org/>`_ is a language server that provides C/C++
	code intelligence features to any editor that supports the language server
	protocol (LSP). It uses a
	`compilation database <https://clang.llvm.org/docs/JSONCompilationDatabase.html>`_,
	a JSON file containing the compile commands for the project. Projects that have
	multiple targets and/or use multiple toolchains need separate compilation
	databases for each target toolchain. ``pw_ide`` provides tools for managing
	those databases.

	Assuming you have one or more compilation databases that have been generated by
	your build system, start with:

	.. code-block:: bash

	pw ide sync

	This command will:

	- Find every compilation database in your build directory

	- Analyze each database

	- If a database is internally consistent (i.e., it only contains valid
	compile commands for a single target), it will use that database as-is for
	the target toolchain that database pertains to. This is the typical case for
	CMake builds.

	- Otherwise, if a database contains commands for multiple target toolchains
	and/or contains invalid compile commands, the database will be processed,
	yielding one new compilation database for each target toolchain. Those
	databases will be used instead of the original.

	- Link each target to its respective compilation database

	Now, you can list the available target toolchains with:

	.. code-block:: bash

	pw ide cpp --list

	Then set the target toolchain that ``clangd`` should use with:

	.. code-block:: bash

	pw ide cpp --set <selected target name>

	``clangd`` will now work as designed since it is configured to use a compilation
	database that is consistent to just a single target toolchain.

	``clangd`` must be run with arguments that provide the Pigweed environment paths
	to the correct toolchains and sysroots. One way to do this is to launch your
	editor from the terminal in an activated environment (e.g. running ``vim`` from
	the terminal), in which case nothing special needs to be done as long as your
	toolchains are in the Pigweed environment or ``$PATH``. But if you launch your
	editor outside of the activated environment (e.g. launching Visual Studio Code
	from your GUI shell's launcher), you can generate the command that invokes
	``clangd`` with the right arguments with:

	.. code-block:: bash

	pw ide cpp --clangd-command

	Python Code Intelligence
	------------------------
	Any Python language server should work well with Pigweed projects as long as
	it's configured to use the Pigweed virtual environment. You can output the path
	to the virtual environment on your system with:

	.. code-block:: bash

	pw ide python --venv

	Docs Code Intelligence
	----------------------
	The `esbonio <https://github.com/swyddfa/esbonio>`_ language server will provide
	code intelligence for RestructuredText and Sphinx. It works well with Pigweed
	projects as long as it is pointed to Pigweed's Python virtual environment. For
	Visual Studio Code, simply install the esbonio extension, which will be
	recommended to you after setting up ``pw_ide``. Once it's installed, a prompt
	will ask if you want to automatically install esbonio in your Pigweed Python
	environment. After that, give esbonio some time to index, then you're done!

	Command-Line Interface Reference
	--------------------------------
	.. argparse::
	:module: pw_ide.cli
	:func: _build_argument_parser
	:prog: pw ide

	Design
	======

	Supporting ``clangd`` for Embedded Projects
	-------------------------------------------
	There are three main challenges that often prevent ``clangd`` from working
	out-of-the-box with embedded projects:

	#. Embedded projects cross-compile using alternative toolchains, rather than
	using the system toolchain. ``clangd`` doesn't know about those toolchains
	by default.

	#. Embedded projects (particularly Pigweed project) often have multiple
	targets that use multiple toolchains. Most build systems that generate
	compilation databases put all compile commands in a single database, meaning
	a single file can have multiple, conflicting compile commands. ``clangd``
	will typically use the first one it finds, which may not be the one you want.

	#. Pigweed projects have build steps that use languages other than C/C++. These
	steps are not relevant to ``clangd`` but many build systems will include them
	in the compilation database anyway.

	To deal with these challenges, ``pw_ide`` processes the compilation database you
	provide, yielding one or more compilation databases that are valid, consistent,
	and specific to a particular target toolchain. This enables code intelligence
	and navigation features that reflect that build.

	After processing a compilation database, ``pw_ide`` knows what target toolchains
	are available and provides tools for selecting which target toolchain is active.
	These tools can be integrated into code editors, but are ultimately CLI-driven
	and editor-agnostic. Enabling code intelligence in your editor may be as simple
	as configuring its language server protocol client to use the ``clangd`` command
	that ``pw_ide`` can generate for you.

	When to provide additional configuration to support your use cases
	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	The default configuration for ``clangd`` in ``pw_ide`` should work without
	additional configuration as long as you're using only toolchains provided by
	Pigweed and your native host toolchain. If you're using other toolchains, keep
	reading.

	``clangd`` needs two pieces of information to use a toolchain:

	#. A path to the compiler, which will be taken from the compile command.

	#. The same compiler to be reflected in the
	`query driver <https://releases.llvm.org/10.0.0/tools/clang/tools/extra/docs/clangd/Configuration.html>`_
	argument provided when running ``clangd``.

	When using ``pw_ide`` with external toolchains, you only need to add a path to
	the compiler to ``clangd_additional_query_drivers`` in your project's
	``pw_ide.yaml`` file. When processing a compilation database, ``pw_ide`` will
	use the query driver globs to find your compiler and configure ``clangd`` to
	use it.

	Compiler wrappers
	^^^^^^^^^^^^^^^^^
	If you're using ``ccache`` or any other wrapper command that is configured
	using ``ccache``'s' ``KEY=VALUE`` pattern, it will work out of the box.

	Selected API Reference
	^^^^^^^^^^^^^^^^^^^^^^
	.. automodule:: pw_ide.cpp
	:members: CppCompileCommand, CppCompilationDatabase, CppCompilationDatabasesMap, CppIdeFeaturesState, path_to_executable, ClangdSettings

	Automated Support for Code Editors & IDEs
	-----------------------------------------
	``pw_ide`` provides a consistent framework for automatically applying settings
	for code editors, where default settings can be defined within ``pw_ide``,
	which can be overridden by project settings, which in turn can be overridden
	by individual user settings.

	.. _module-pw_ide-vscode:

	Visual Studio Code
	^^^^^^^^^^^^^^^^^^
	Running ``pw ide sync`` will automatically generate settings for Visual Studio
	Code. ``pw_ide`` comes with sensible defaults for Pigweed projects, but those
	can be augmented or overridden at the project level or the user level using
	``pw_project_settings.json`` and ``pw_user_settings.json`` respectively. The
	generated ``settings.json`` file is essentially a build artifact and shouldn't
	be committed to source control.

	The same pattern applies to ``tasks.json``, which provides Visual Studio Code
	tasks for ``pw_ide`` commands. Access these by opening the command palette
	(Ctrl/Cmd-Shift-P), selecting ``Tasks: Run Task``, then selecting the desired
	task.

	The same pattern also applies to ``launch.json``, which is used to define
	configurations for running and debugging your project. Create a
	``pw_project_launch.json`` with configurations that conform to the Visual Studio
	Code `debugger configuration format <https://code.visualstudio.com/docs/editor/debugging>`_.

	.. tip::

	What's the difference between "Change C++ Code Analysis Target" and "Set C++
	Code Analyis Target"? "Set" will automatically restart the ``clangd``
	language server for you to pick up the changed target immediately, while
	"Change" will not.

	Selected API Reference
	^^^^^^^^^^^^^^^^^^^^^^
	.. automodule:: pw_ide.editors
	:members: EditorSettingsDefinition, EditorSettingsFile, EditorSettingsManager

	.. automodule:: pw_ide.vscode
	:members: VscSettingsType, VscSettingsManager