.. _docs-automated-analysis:

==================
Automated analysis
==================

The correctness and style of Pigweed's source code is continuously verified
using a suite of automated tools. We also make it easy to use the same tools
to verify the code of projects using Pigweed.

-------
Summary
-------

On presubmit or in CI we verify Pigweed using:

* pylint
* mypy
* clang-tidy
* AddressSanitizer (asan)
* ThreadSanitizer (tsan)
* UndefinedBehaviorSanitizer (ubsan)
* OSS-Fuzz

The rest of this document discusses these tools and their configuration in
greater detail, and how to use them in your own project.

--------------
Analysis tools
--------------

Static analysis
===============

PyLint
------

`PyLint`_ is a customizable Python linter. Pigweed complies with almost all
the default checks; see `.pylintrc`_ for details. PyLint detects problems such
as overly broad catch statements, unused arguments/variables, and mutable
default parameter values.

For upstream Pigweed, PyLint can be run with ``ninja python.lint.pylint`` or
``ninja python.lint``.  It's also included in a variety of presubmit steps,
like ``static_analysis`` and ``python_checks.gn_python_check``.  See the
`Enabling analysis for your project`_ section to learn how to run PyLint on
your Pigweed-based project.

.. _PyLint: https://pylint.org/
.. _.pylintrc: https://cs.opensource.google/pigweed/pigweed/+/main:.pylintrc

Mypy
----

Python 3 allows for `type annotations`_ for variables, function arguments, and
return values. Most, but not all, of Pigweed's Python code has type
annotations, and these annotations have caught real bugs in code that didn't
yet have unit tests. `Mypy`_ is an analysis tool that enforces these
annotations.

Mypy helps find bugs like when a string is passed into a function that expects
a list of strings---since both are iterables this bug might otherwise be hard
to track down.

Mypy can be run with ``ninja python.lint.mypy`` or ``ninja python.lint``. It's
also included in a variety of presubmit steps, like ``static_analysis`` and
``python_checks.gn_python_check``.

.. _type annotations: https://docs.python.org/3/library/typing.html
.. _Mypy: http://mypy-lang.org/

clang-tidy
----------

`clang-tidy`_ is a C++ "linter" and static analysis tool. It identifies
bug-prone patterns (e.g., use after move), non-idiomatic usage (e.g., creating
``std::unique_ptr`` with ``new`` rather than ``std::make_unique``), and
performance issues (e.g., unnecessary copies of loop variables).

While powerful, clang-tidy defines a very large number of checks, many of which
are special-purpose (e.g., only applicable to FPGA HLS code, or code using the
`Abseil`_ library) or have high false positive rates. Pigweed enables over 50
checks which are relevant to an embedded C/C++ library and have good
signal-to-noise ratios. The full list of Pigweed's checks is in `.clang-tidy`_.

We do not currently enable the `Clang Static Analyzers`_ because they suffer
from false positives, and their findings are time-consuming to manually verify.

clang-tidy can be run with ``ninja static_analysis`` or ``pw presubmit --step
static_analysis``.

.. _clang-tidy: https://clang.llvm.org/extra/clang-tidy/
.. _Abseil: https://abseil.io/
.. _.clang-tidy: https://cs.opensource.google/pigweed/pigweed/+/main:.clang-tidy
.. _Clang Static Analyzers: https://clang-analyzer.llvm.org/available_checks.html


Clang sanitizers
================

.. note::
   Running sanitizers in presubmit for all Pigweed code is work in progress.
   See https://bugs.pigweed.dev/514 for details.

We run all of Pigweed's unit tests with the additional instrumentation
described in this section. For more detail about these sanitizers, see the
`Github documentation`_.

* asan: `AddressSanitizer`_ detects memory errors such as out-of-bounds access
  and use-after-free.
* msan: `MemorySanitizer`_ detects reads of uninitialized memory.
* tsan: `ThreadSanitizer`_ detects data races.
* ubsan: `UndefinedBehaviorSanitizer`_ is a fast undefined behavior detector.
  We use the default ``-fsanitize=undefined`` option.

The exact configurations we use for these sanitizers are in
`pw_toolchain/host_clang/BUILD.gn <https://cs.opensource.google/pigweed/pigweed/+/main:pw_toolchain/host_clang/BUILD.gn>`_.
You can see the current status of the sanitizer builds in the `Pigweed CI
console`_, as ``pigweed-linux-san-*``.

.. _Github documentation: https://github.com/google/sanitizers
.. _AddressSanitizer: https://clang.llvm.org/docs/AddressSanitizer.html
.. _MemorySanitizer: https://clang.llvm.org/docs/MemorySanitizer.html
.. _Pigweed CI console: https://ci.chromium.org/p/pigweed/g/pigweed/console
.. _ThreadSanitizer: https://clang.llvm.org/docs/ThreadSanitizer.html
.. _UndefinedBehaviorSanitizer: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html


Fuzzers
=======
`Fuzz testing`_ detects errors in software by providing it with randomly
generated inputs.  We use `OSS-fuzz`_ to continuously uncover potential
vulnerabilities in Pigweed.  `Dashboard with Pigweed's latest results`_. See
the `pw_fuzzer <module-pw_fuzzer>`_ module documentation for more details.

.. _Dashboard with Pigweed's latest results: https://oss-fuzz-build-logs.storage.googleapis.com/index.html#pigweed
.. _Fuzz testing: https://en.wikipedia.org/wiki/Fuzzing
.. _OSS-fuzz: https://github.com/google/oss-fuzz

.. _Enabling analysis for your project:

----------------------------------
Enabling analysis for your project
----------------------------------

PyLint and Mypy
===============

PyLint and Mypy can be configured to run every time your project is built by
adding ``python.lint`` to your default build group. (You can also add one or both
individually using ``python.lint.mypy`` and ``python.lint.pylint``.) Likewise,
these can be added to individual presubmit steps (`examples`_). You can also
directly include the `python_checks.gn_python_lint`_ presubmit step.

.. _examples: https://cs.opensource.google/search?q=file:pigweed_presubmit.py%20%22python.lint%22&sq=&ss=pigweed%2Fpigweed
.. _python_checks.gn_python_lint: https://cs.opensource.google/pigweed/pigweed/+/main:pw_presubmit/py/pw_presubmit/python_checks.py?q=file:python_checks.py%20gn_python_lint&ss=pigweed%2Fpigweed

clang-tidy
==========

`pw_toolchain/static_analysis_toolchain.gni`_ provides the
``pw_static_analysis_toolchain`` template that can be used to create a build
group performing static analysis. See :ref:`module-pw_toolchain` documentation
for more details. This group can then be added as a presubmit step using
pw_presubmit.

You can place a ``.clang-tidy`` file at the root of your repository to control
which checks are executed. See the `clang documentation`_ for a discussion of how
the tool chooses which ``.clang-tidy`` files to apply when run on a particular
source file.

.. _pw_toolchain/static_analysis_toolchain.gni: https://cs.opensource.google/pigweed/pigweed/+/main:pw_toolchain/static_analysis_toolchain.gni
.. _clang documentation: https://clang.llvm.org/extra/clang-tidy/

Clang sanitizers
================

.. note::
   This section is under construction.

Fuzzers
=======
See the `pw_fuzzer <module-pw_fuzzer>`_ module documentation.