doc/guides/west/repo-tool.rst - third_party/github/zephyrproject-rtos/zephyr - Git at Google

 .. _west-multi-repo:

 Multiple Repository Management
 ##############################

 This page introduces basic concepts related to West and its multiple repository
 management features, and gives an overview of the associated commands. See
 :ref:`west-history` and `Zephyr issue #6770`_ for additional discussion,
 rationale, and motivation.

 .. note::

    West's multi-repo commands are meant to augment Git in minor ways for
    multi-repo work, not to replace it. For tasks that only operate on one
    repository, just use plain Git commands.

 .. _west-installation:

 Introduction
 ************

 West's built-in commands allow you to work with projects composed of multiple
 Git repositories installed under a common parent directory, which we call a
 *west installation*. This works similarly to `Git Submodules
 <https://git-scm.com/book/en/v2/Git-Tools-Submodules>`_ and Google's `repo
 <https://gerrit.googlesource.com/git-repo/>`_.

 A west installation is the result of running the ``west init`` command, which
 is described in more detail below. This command can either create a new
 installation, or convert a standalone "mono-repo" zephyr repository into a full
 west installation. For upstream Zephyr, the installation looks like this:

 .. code-block:: none

    zephyrproject
    ├── .west
    │   ├── config
    │   └── west
    ├── zephyr
    │   ├── west.yml
    │   └── [... other files ...]
    ├── modules
    │   └── lib
    │       └── tinycbor
    ├── net-tools
    └── [ ... other projects ...]

 Above, :file:`zephyrproject` is the name of the west installation's root
 directory. This name is just an example -- it could be anything, like ``z``,
 ``my-zephyr-installation``, etc.  The file :file:`.west/config` is the
 installation's :ref:`local configuration file <west-config>`. The directory
 :file:`.west/west` is a clone of the west repository itself; more details on
 why that is currently needed are given in the next section.

 Every west installation contains exactly one *manifest repository*, which is a
 Git repository containing a file named :file:`west.yml`, which is the *west
 manifest*. The location of the manifest repository is given by the
 :ref:`manifest.path configuration option <west-config-index>` in the local
 configuration file. The manifest file, along with west's configuration files,
 controls the installation's behavior. For upstream Zephyr, :file:`zephyr` is
 the manifest repository, but you can configure west to use any Git repository
 in the installation as the manifest repository. The only requirement is that it
 contains a valid manifest file. See :ref:`west-manifests` for more details on
 what this means.

 Both of the :file:`tinycbor` and :file:`net-tools` directories are *projects*
 managed by west, and configured in the manifest file. A west installation can
 contain arbitrarily many projects. As shown above, projects can be located
 anywhere in the installation. They don't have to be subdirectories of the
 manifest directory, and they can be inside of arbitrary subdirectories inside
 the installation's root directory. By default, the Zephyr build system uses
 west to get the locations of all the projects in the installation, so any code
 they contain can be used by applications. This behavior can be overridden using
 the ``ZEPHYR_MODULES`` CMake variable; see
 :zephyr_file:`cmake/zephyr_module.cmake` for details.

 Finally, any repository managed by a west installation can contain
 :ref:`extension commands <west-extensions>`, which are extra west commands
 provided by that project. This includes the manifest repository and any project
 repository.

 .. _west-struct:

 West Structure
 **************

 West is currently split in two:

 * Bootstrapper: Installed by ``pip3 install west``, which provides the ``west``
   binary and the ``west init`` command.
 * Per-installation clone: this is the west repository cloned into each
   installation, which provides the built-in commands.

 .. note::

    This "bootstrapper" / "everything else" separation is similar to the model
    used by Google's ``repo`` tool, but unfortunately in retrospect was not a
    good strategy for west.

    In future versions, the ``west`` binary and all built-in commands (including
    ``init``) will be installed by ``pip3 install west``. Besides eliminating
    complexity, this will also make it possible to use :ref:`West's APIs
    <west-apis-west>` from any Python file, not just extension
    commands.

    Updating west will still be possible manually, e.g. with ``pip3
    install --upgrade west``. If necessary, it will also still be possible to
    use different versions of west on the same computer through Python virtual
    environments.

 Bootstrapper
 ============

 The bootstrapper module is distributed using `PyPI`_ and installed using
 :file:`pip3`. A launcher named ``west`` is placed by :file:`pip3` in the user's
 ``PATH``. This the only entry point to west.  It implements a single command:
 ``west init``. This command needs to be run first to use the rest of
 functionality included in ``west``, by creating a west installation. The
 command ``west init`` does the following:

 * Clones west itself in a :file:`.west/west` folder in the installation.
 * Clones the manifest repository in the folder specified by the manifest file's
   ``self.path`` section.
 * Creates an initial local configuration file.

 Once ``west init`` has been run, the bootstrapper will delegate the handling of
 any west commands other than ``init`` to the cloned west repository. This means
 that there is a single bootstrapper instance installed at any time (unless you
 use virtual environments), which can then be used to initialize as many
 installations as needed, each of which can have a different version of west.

 .. _west-struct-installation:

 Per-Installation Clone
 ======================

 A west installation, as described above, contains a clone of the west
 repository in :file:`.west/west`. This is where the built-in command
 implementations are currently provided. The rest of :ref:`West's APIs
 <west-apis-west>` are also currently provided to extension commands by this
 repository. So that west can update itself, the built-in ``west update`` and
 ``west selfupdate`` commands fetch and update the :file:`.west/west` repository.

 The ``manifest-rev`` branch
 ***************************

 West creates a branch named ``manifest-rev`` in each project, pointing to the
 commit the project's revision resolves to. The branch is updated whenever
 project data is fetched by ``west update``. Other multi-repo commands also use
 ``manifest-rev`` as a reference for the upstream revision as of the most recent
 update. See :ref:`west-multi-repo-cmds`, below, for more information.

 ``manifest-rev`` is a normal Git branch, but if you delete or otherwise modify
 it, west will recreate and/or reset it as if with ``git reset --hard`` on the
 next update (though ``git update-ref`` is used internally). For this reason, it
 is normally a **bad idea to modify it yourself**. ``manifest-rev`` was added to
 allow SHAs as project revisions in the manifest, and to give a consistent
 reference for the current upstream revision regardless of how the manifest
 changes over time.

 .. note::

    West does not create a ``manifest-rev`` branch in the manifest repository,
    since west does not manage the manifest repository's branches or revisions.

 .. _west-multi-repo-cmds:

 Multi-Repo Commands
 *******************

 This section gives a quick overview of the multi-repo commands, split up by
 functionality. Some commands loosely mimic the corresponding Git command, but
 in a multi-repo context (e.g. ``west diff`` shows local changes on all
 repositories).

 Project arguments can be the names of projects in the manifest, or their paths
 within the installation. Passing no project arguments to commands that accept a
 list of projects usually means to use all projects in the manifest.

 .. note::

    For detailed help, see each command's ``--help`` output (e.g.  ``west diff
    --help``).

 Main Commands
 =============

 The ``west init`` and ``west update`` multi-repo commands are the most
 important to understand.

 - ``west init [-l] [-m URL] [--mr REVISION] [PATH]``: create a west
   installation in directory :file:`PATH` (i.e. :file:`.west` etc. will be
   created there). If the ``PATH`` argument is not given, the current working
   directory is used. This command does not clone any of the projects in the
   manifest; that is done the next time ``west update`` is run.

   This command can be invoked in two ways:

   1. If you already have a local clone of the zephyr repository and want to
      create a west installation around it, you can use the ``-l`` switch to
      pass its path to west, as in: ``west init -l path/to/zephyr``.

   2. Otherwise, omit ``-l`` to create a new installation from a remote manifest
      repository. You can give the manifest URL using the ``-m`` switch, and its
      revision using ``--mr``. For example, invoking west with: ``west init -m
      https://github.com/zephyrproject-rtos/zephyr --mr v1.15.0`` would clone
      the upstream official zephyr repository at the tagged release v1.15.0
      (``-m`` defaults to https://github.com/zephyrproject-rtos/zephyr, and
      ``--mr`` defaults to ``master``).

 - ``west update [--rebase] [--keep-descendants] [--exclude-west] [PROJECT
   ...]``: clone and update the specified projects (default: all projects) based
   on the current :term:`west manifest`.

   This command parses the manifest, clones any project repositories that are
   not already present locally, and checks out the project revisions specified
   in the manifest file, updating ``manifest-rev`` branches along the way.

   For safety, ``west update`` uses ``git checkout --detach`` to check out a
   detached ``HEAD`` at the manifest revision for each updated project, leaving
   behind any branches which were already checked out. This is typically a safe
   operation that will not modify any of your local branches. See the help for
   the ``--rebase`` / ``-r`` and ``--keep-descendants`` / ``-k`` options for
   ways to influence this.

   By default, ``west update`` also updates the west repository in the
   installation. To prevent this, use ``--exclude-west``.

 .. _west-multi-repo-misc:

 Miscellaneous Commands
 ======================

 West has a few more commands for managing the multi-repo, which are briefly
 discussed here.

 - ``west list``: Lists project information from the manifest (URL, revision,
   path, etc.), along with other manifest-related information.

 - ``west manifest --freeze [-o outfile]``: Save a "frozen" representation of
   the current manifest; all ``revision`` fields are converted to SHAs based on
   the current ``manifest-rev`` branches.

 - ``west diff [PROJECT ...]``: Runs a multi-repo ``git diff``
   for the specified projects (default: all cloned projects).

 - ``west status [PROJECT ...]``: Like ``west diff``, for
   running ``git status``.

 - ``west forall -c COMMAND [PROJECT ...]``: Runs the shell command ``COMMAND``
   within the top-level repository directory of each of the specified projects
   (default: all cloned projects). If ``COMMAND`` consists of more than one
   word, it must be quoted to prevent it from being split up by the shell.

   To run an arbitrary Git command in each project, use something like ``west
   forall -c 'git <command> --options'``. Note that ``west forall`` can be used
   to run any command, though, not just Git commands.

 - ``west selfupdate``: Updates the west repository in the installation.

 .. _PyPI:
    https://pypi.org/project/west/

 .. _Zephyr issue #6770:
    https://github.com/zephyrproject-rtos/zephyr/issues/6770
	.. _west-multi-repo:

	Multiple Repository Management
	##############################

	This page introduces basic concepts related to West and its multiple repository
	management features, and gives an overview of the associated commands. See
	:ref:`west-history` and `Zephyr issue #6770`_ for additional discussion,
	rationale, and motivation.

	.. note::

	West's multi-repo commands are meant to augment Git in minor ways for
	multi-repo work, not to replace it. For tasks that only operate on one
	repository, just use plain Git commands.

	.. _west-installation:

	Introduction
	************

	West's built-in commands allow you to work with projects composed of multiple
	Git repositories installed under a common parent directory, which we call a
	west installation. This works similarly to `Git Submodules
	<https://git-scm.com/book/en/v2/Git-Tools-Submodules>`_ and Google's `repo
	<https://gerrit.googlesource.com/git-repo/>`_.

	A west installation is the result of running the ``west init`` command, which
	is described in more detail below. This command can either create a new
	installation, or convert a standalone "mono-repo" zephyr repository into a full
	west installation. For upstream Zephyr, the installation looks like this:

	.. code-block:: none

	zephyrproject
	├── .west
	│ ├── config
	│ └── west
	├── zephyr
	│ ├── west.yml
	│ └── [... other files ...]
	├── modules
	│ └── lib
	│ └── tinycbor
	├── net-tools
	└── [ ... other projects ...]

	Above, :file:`zephyrproject` is the name of the west installation's root
	directory. This name is just an example -- it could be anything, like ``z``,
	``my-zephyr-installation``, etc. The file :file:`.west/config` is the
	installation's :ref:`local configuration file <west-config>`. The directory
	:file:`.west/west` is a clone of the west repository itself; more details on
	why that is currently needed are given in the next section.

	Every west installation contains exactly one manifest repository, which is a
	Git repository containing a file named :file:`west.yml`, which is the *west
	manifest*. The location of the manifest repository is given by the
	:ref:`manifest.path configuration option <west-config-index>` in the local
	configuration file. The manifest file, along with west's configuration files,
	controls the installation's behavior. For upstream Zephyr, :file:`zephyr` is
	the manifest repository, but you can configure west to use any Git repository
	in the installation as the manifest repository. The only requirement is that it
	contains a valid manifest file. See :ref:`west-manifests` for more details on
	what this means.

	Both of the :file:`tinycbor` and :file:`net-tools` directories are projects
	managed by west, and configured in the manifest file. A west installation can
	contain arbitrarily many projects. As shown above, projects can be located
	anywhere in the installation. They don't have to be subdirectories of the
	manifest directory, and they can be inside of arbitrary subdirectories inside
	the installation's root directory. By default, the Zephyr build system uses
	west to get the locations of all the projects in the installation, so any code
	they contain can be used by applications. This behavior can be overridden using
	the ``ZEPHYR_MODULES`` CMake variable; see
	:zephyr_file:`cmake/zephyr_module.cmake` for details.

	Finally, any repository managed by a west installation can contain
	:ref:`extension commands <west-extensions>`, which are extra west commands
	provided by that project. This includes the manifest repository and any project
	repository.

	.. _west-struct:

	West Structure
	**************

	West is currently split in two:

	* Bootstrapper: Installed by ``pip3 install west``, which provides the ``west``
	binary and the ``west init`` command.
	* Per-installation clone: this is the west repository cloned into each
	installation, which provides the built-in commands.

	.. note::

	This "bootstrapper" / "everything else" separation is similar to the model
	used by Google's ``repo`` tool, but unfortunately in retrospect was not a
	good strategy for west.

	In future versions, the ``west`` binary and all built-in commands (including
	``init``) will be installed by ``pip3 install west``. Besides eliminating
	complexity, this will also make it possible to use :ref:`West's APIs
	<west-apis-west>` from any Python file, not just extension
	commands.

	Updating west will still be possible manually, e.g. with ``pip3
	install --upgrade west``. If necessary, it will also still be possible to
	use different versions of west on the same computer through Python virtual
	environments.

	Bootstrapper
	============

	The bootstrapper module is distributed using `PyPI`_ and installed using
	:file:`pip3`. A launcher named ``west`` is placed by :file:`pip3` in the user's
	``PATH``. This the only entry point to west. It implements a single command:
	``west init``. This command needs to be run first to use the rest of
	functionality included in ``west``, by creating a west installation. The
	command ``west init`` does the following:

	* Clones west itself in a :file:`.west/west` folder in the installation.
	* Clones the manifest repository in the folder specified by the manifest file's
	``self.path`` section.
	* Creates an initial local configuration file.

	Once ``west init`` has been run, the bootstrapper will delegate the handling of
	any west commands other than ``init`` to the cloned west repository. This means
	that there is a single bootstrapper instance installed at any time (unless you
	use virtual environments), which can then be used to initialize as many
	installations as needed, each of which can have a different version of west.

	.. _west-struct-installation:

	Per-Installation Clone
	======================

	A west installation, as described above, contains a clone of the west
	repository in :file:`.west/west`. This is where the built-in command
	implementations are currently provided. The rest of :ref:`West's APIs
	<west-apis-west>` are also currently provided to extension commands by this
	repository. So that west can update itself, the built-in ``west update`` and
	``west selfupdate`` commands fetch and update the :file:`.west/west` repository.

	The ``manifest-rev`` branch
	***************************

	West creates a branch named ``manifest-rev`` in each project, pointing to the
	commit the project's revision resolves to. The branch is updated whenever
	project data is fetched by ``west update``. Other multi-repo commands also use
	``manifest-rev`` as a reference for the upstream revision as of the most recent
	update. See :ref:`west-multi-repo-cmds`, below, for more information.

	``manifest-rev`` is a normal Git branch, but if you delete or otherwise modify
	it, west will recreate and/or reset it as if with ``git reset --hard`` on the
	next update (though ``git update-ref`` is used internally). For this reason, it
	is normally a bad idea to modify it yourself. ``manifest-rev`` was added to
	allow SHAs as project revisions in the manifest, and to give a consistent
	reference for the current upstream revision regardless of how the manifest
	changes over time.

	.. note::

	West does not create a ``manifest-rev`` branch in the manifest repository,
	since west does not manage the manifest repository's branches or revisions.

	.. _west-multi-repo-cmds:

	Multi-Repo Commands
	*******************

	This section gives a quick overview of the multi-repo commands, split up by
	functionality. Some commands loosely mimic the corresponding Git command, but
	in a multi-repo context (e.g. ``west diff`` shows local changes on all
	repositories).

	Project arguments can be the names of projects in the manifest, or their paths
	within the installation. Passing no project arguments to commands that accept a
	list of projects usually means to use all projects in the manifest.

	.. note::

	For detailed help, see each command's ``--help`` output (e.g. ``west diff
	--help``).

	Main Commands
	=============

	The ``west init`` and ``west update`` multi-repo commands are the most
	important to understand.

	- ``west init [-l] [-m URL] [--mr REVISION] [PATH]``: create a west
	installation in directory :file:`PATH` (i.e. :file:`.west` etc. will be
	created there). If the ``PATH`` argument is not given, the current working
	directory is used. This command does not clone any of the projects in the
	manifest; that is done the next time ``west update`` is run.

	This command can be invoked in two ways:

	1. If you already have a local clone of the zephyr repository and want to
	create a west installation around it, you can use the ``-l`` switch to
	pass its path to west, as in: ``west init -l path/to/zephyr``.

	2. Otherwise, omit ``-l`` to create a new installation from a remote manifest
	repository. You can give the manifest URL using the ``-m`` switch, and its
	revision using ``--mr``. For example, invoking west with: ``west init -m
	https://github.com/zephyrproject-rtos/zephyr --mr v1.15.0`` would clone
	the upstream official zephyr repository at the tagged release v1.15.0
	(``-m`` defaults to https://github.com/zephyrproject-rtos/zephyr, and
	``--mr`` defaults to ``master``).

	- ``west update [--rebase] [--keep-descendants] [--exclude-west] [PROJECT
	...]``: clone and update the specified projects (default: all projects) based
	on the current :term:`west manifest`.

	This command parses the manifest, clones any project repositories that are
	not already present locally, and checks out the project revisions specified
	in the manifest file, updating ``manifest-rev`` branches along the way.

	For safety, ``west update`` uses ``git checkout --detach`` to check out a
	detached ``HEAD`` at the manifest revision for each updated project, leaving
	behind any branches which were already checked out. This is typically a safe
	operation that will not modify any of your local branches. See the help for
	the ``--rebase`` / ``-r`` and ``--keep-descendants`` / ``-k`` options for
	ways to influence this.

	By default, ``west update`` also updates the west repository in the
	installation. To prevent this, use ``--exclude-west``.

	.. _west-multi-repo-misc:

	Miscellaneous Commands
	======================

	West has a few more commands for managing the multi-repo, which are briefly
	discussed here.

	- ``west list``: Lists project information from the manifest (URL, revision,
	path, etc.), along with other manifest-related information.

	- ``west manifest --freeze [-o outfile]``: Save a "frozen" representation of
	the current manifest; all ``revision`` fields are converted to SHAs based on
	the current ``manifest-rev`` branches.

	- ``west diff [PROJECT ...]``: Runs a multi-repo ``git diff``
	for the specified projects (default: all cloned projects).

	- ``west status [PROJECT ...]``: Like ``west diff``, for
	running ``git status``.

	- ``west forall -c COMMAND [PROJECT ...]``: Runs the shell command ``COMMAND``
	within the top-level repository directory of each of the specified projects
	(default: all cloned projects). If ``COMMAND`` consists of more than one
	word, it must be quoted to prevent it from being split up by the shell.

	To run an arbitrary Git command in each project, use something like ``west
	forall -c 'git <command> --options'``. Note that ``west forall`` can be used
	to run any command, though, not just Git commands.

	- ``west selfupdate``: Updates the west repository in the installation.

	.. _PyPI:
	https://pypi.org/project/west/

	.. _Zephyr issue #6770:
	https://github.com/zephyrproject-rtos/zephyr/issues/6770