pw_hdlc_lite: Completing the docs for this module
Change-Id: I95aefa605a3496476df209404b56066ec08e1ca8
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/15462
Commit-Queue: Shane Gomindes <shaneajg@google.com>
Reviewed-by: Wyatt Hepler <hepler@google.com>
diff --git a/docs/BUILD.gn b/docs/BUILD.gn
index f087940..1296b3f 100644
--- a/docs/BUILD.gn
+++ b/docs/BUILD.gn
@@ -80,6 +80,7 @@
"$dir_pw_doctor:docs",
"$dir_pw_env_setup:docs",
"$dir_pw_fuzzer:docs",
+ "$dir_pw_hdlc_lite:docs",
"$dir_pw_hex_dump:docs",
"$dir_pw_kvs:docs",
"$dir_pw_log:docs",
diff --git a/pw_hdlc_lite/docs.rst b/pw_hdlc_lite/docs.rst
index 1314330..fbb283c 100644
--- a/pw_hdlc_lite/docs.rst
+++ b/pw_hdlc_lite/docs.rst
@@ -16,11 +16,233 @@
Dependencies
============
+* ``pw_bytes``
+* ``pw_log``
* ``pw_preprocessor``
+* ``pw_result``
+* ``pw_rpc``
* ``pw_status``
* ``pw_span``
-* ``pw_sys_io``
* ``pw_stream``
+* ``pw_sys_io``
+
+HDLC-Lite Overview
+==================
+High-Level Data Link Control (HDLC) is a data link layer protocol which uses
+synchronous serial transmissions for communication between two devices. Unlike
+the standard HDLC protocol which uses six fields of embedded information, the
+HDLC-Lite protocol is a minimal version that only uses the bare essentials.
+
+The HDLC-Lite data frame in ``pw_hdlc_lite`` uses a start and end frame
+delimiter (0x7E), the escaped binary payload and the CCITT-CRC16 value.
+It looks like:
+
+.. code-block:: text
+
+ [More frames]
+ _________________________________________ _______
+ | | | | | |...| | |
+ | | | | | |...| | |
+ |_|______________________________|__|_|_|...|___|_|
+ F Payload CRC F F CRC F
+
+Basic Overview
+==============
+The ``pw_hdlc_lite`` module provides a simple, reliable packet-oriented
+transport that uses the HDLC-Lite protocol to send and receive data to and from
+embedded devices. This is especially needed for making RPC calls on devices
+during testing because the ``pw_rpc`` module does not handle the transmission of
+RPCs. This module enables the transmission of RPCs and other bytes through a
+serial connection.
+
+There are essentially two main functions of the ``pw_hdlc_lite`` module:
+
+ * **Encoding** the data by escaping the bytes of the payload, calculating the
+ CCITT-CRC16 value, constructing a data frame and sending the
+ resulting data packet through serial.
+ * **Decoding** the data by unescaping the received bytes, verifying the
+ CCITT-CRC16 value and returning the successfully decoded packets.
+
+**Why use the ``pw_hdlc_lite`` module?**
+
+ * Enables the transmission of RPCs and other data between devices over serial
+ * Resilient to corruption and data loss.
+ * Light-weight, simple and easy to use.
+ * Supports streaming to transport without buffering - e.g. protocol buffers
+ have length-prefix.
+
+Protocol Description
+====================
+
+Encoding and sending data
+-------------------------
+This module first writes an initial frame delimiter byte (0x7E) to indicate the
+beginning of the frame. Before sending any of the payload data through serial,
+the special bytes are escaped accordingly:
+
+ +-----------------------+----------------------+
+ |Unescaped Special Bytes| Escaped Special Bytes|
+ +=======================+======================+
+ | 0x7E | 0x7D5E |
+ +-----------------------+----------------------+
+ | 0x7D | 0x7D5D |
+ +-----------------------+----------------------+
+
+The bytes of the payload are escaped and written in a single pass. The
+CCITT-CRC16 value is calculated, escaped and written after. After this, a final
+frame delimiter byte (0x7E) is written to mark the end of the frame.
+
+Decoding received bytes
+-----------------------
+Packets may be received in multiple parts, so we need to store the received data
+in a buffer until the ending frame delimiter (0x7E) is read. When the
+pw_hdlc_lite decoder receives data, it unescapes it and adds it to a buffer.
+When the frame is complete, it calculates and verifies the CCITT-CRC16 bytes and
+does the following:
+
+* If correctly verified, the decoder returns the decoded packet.
+* If the checksum verification fails, the data packet is discarded.
+
+During the decoding process, the decoder essentially transitions between 3
+states, where each state indicates the method of decoding that particular byte:
+
+NO_PACKET --> PACKET_ACTIVE --> (ESCAPE | NO_PACKET).
+
+API Usage
+=========
+
+Encoder
+-------
+The Encoder API invloves a single function that encodes the data using the
+HDLC-Lite protocol and sends the data through the serial.
+
+C++
+^^^
+In C++, this function is called ``EncodeAndWritePayload`` and it accepts a
+ConstByteSpan called payload and an object of type Writer& as arguments. It
+returns a Status object that indicates if the write was successful. This
+implementation uses the ``pw_checksum`` module to compute the CRC16 value. Since
+the function writes a starting and ending frame delimiter byte at the beginnning
+and the end of frames, it is safe to encode multiple spans. The usage of this
+function is as follows:
+
+.. code-block:: cpp
+
+ #include "pw_hdlc_lite/encoder.h"
+ #include "pw_hdlc_lite/sys_io_stream.h"
+
+ int main() {
+ pw::stream::SerialWriter serial_writer;
+ constexpr std::array<byte, 1> test_array = { byte(0x41) };
+ auto status = EncodeAndWritePayload(test_array, serial_writer);
+ }
+
+In the example above, we expect the encoder to send the following bytes:
+
+- **0x7E** - Initial Frame Delimiter
+- **0x41** - Payload
+- **0x15** - LSB of the CCITT-CRC16 value
+- **0xB9** - MSB of the CCITT-CRC16 value
+- **0x7E** - End Frame Delimiter
+
+Python
+^^^^^^
+In Python, the function is called ``encode_and_write_payload`` and it accepts
+the payload as a byte object and a callable to which is used to write the data.
+This function does not return anything, and uses the binascii library function
+called crc_hqx to compute the CRC16 bytes. Like the C++ function, the Python
+function also writes a frame delimiter at the beginnning and the end of frames
+so it is safe to encode multiple spans consecutively. The usage of this function
+is as follows:
+
+.. code-block:: python
+
+ import serial
+ from pw_hdlc_lite import encoder
+
+ ser = serial.Serial()
+ encoder.encode_and_write_payload(b'A', ser.write)
+
+We expect this example to give us the same result as the C++ example above since
+it encodes the same payload.
+
+Decoder
+-------
+The Decoder API involves a Decoder class whose main functionality is a function
+that unescapes the received bytes, adds them to a buffer and returns the
+successfully decoded packets. A class is used so that the user can call the
+decoder object's adding bytes functionality on the currently received bytes
+instead of waiting on the entire packet to arrive.
+
+C++
+^^^
+The main functionality of the C++ ``Decoder`` class is the 'AddByte' function
+which accepts a single byte as an argument, unescapes it and adds it to the
+buffer. If the byte is the ending frame delimiter flag (0x7E) it attempts to
+decode the packet and returns a ``Result`` object indicating the success of the
+operation:
+
+ * The returned ``pw::Result`` object will have status ``Status::OK`` and
+ value ConstByteSpan containing the most recently decoded packet if it finds
+ the end of the data-frame and successfully verifies the CRC.
+ * The returned ``pw::Result`` object will have status ``Status::UNAVAILABLE``
+ if it doesnt find the end of the data-frame during that function call.
+ * The returned ``pw::Result`` object will have status ``Status::DATA_LOSS``
+ if it finds the end of the data-frame, but the CRC-verification fails. It
+ also returns this status if the packet in question does not have the
+ 2-byte CRC in it.
+ * The returned ``pw::Result`` object will have status
+ ``Status::RESOURCE_EXHAUSTED`` if the decoder buffer runs out of space.
+
+Here's a C++ example of reading individual bytes from serial and then using the
+decoder object to decode the received data:
+
+.. code-block:: cpp
+
+ #include "pw_hdlc_lite/decoder.h"
+ #include "pw_sys_io/sys_io.h"
+
+ int main() {
+ byte data;
+ while (true) {
+ if (!pw::sys_io::ReadByte(&data).ok()) {
+ // Log serial reading error
+ }
+ auto decoded_packet = decoder.AddByte(data);
+
+ if (decoded_packet.ok()) {
+ // Use decoded_packet to get access to the most recently decoded packet
+ }
+ }
+ }
+
+Python
+^^^^^^
+The main functionality of the Python ``Decoder`` class is the ``add_bytes``
+generator which unescapes the bytes object argument and adds them to a buffer
+until it encounters the ending frame delimiter (0x7E) flag. The generator yields
+the decoded packets as byte objects upon successfully verification of the CRC16
+value of the received bytes. If the CRC verification fails it raises a
+CrcMismatchError exception.
+
+Below is an example of the usage of the decoder class to decode bytes read from
+serial:
+
+.. code-block:: python
+
+ import serial
+ from pw_hdlc_lite import decoder
+
+ ser = serial.Serial()
+ decode = decoder.Decoder()
+
+ while true:
+ byte = ser.read(1)
+
+ for decoded_packet in decode.add_bytes(byte):
+ # Do something with the decoded packet
+
+Like the C++ example, this reads individual bytes and adds them to the decoder.
Features
========
@@ -28,9 +250,19 @@
pw::stream::SerialWriter
------------------------
The ``SerialWriter`` class implements the ``Writer`` interface by using sys_io
-to write data over a communication channel.
+to write data over a serial connection. This Writer object is used by the C++
+encoder to send the encoded bytes to the device.
+Roadmap & Status
+================
-Future work
-^^^^^^^^^^^
-- Adding the code for the Encoder and Decoder.
+- **Additional fields** - As it currently stands, ``pw_hdlc_lite`` uses only
+ three fields of control bytes: starting frame delimiter, 2-byte CRC and an
+ ending frame delimiter. However, if we decided to send larger, more
+ complicated RPCs and if wanted to stream log debug and error messages, we will
+ require additional fields of data to ensure the different packets are sent
+ correctly. Thus, in the future, we plan to add additional channel and sequence
+ byte fields that could enable separate channels for pw_rpc, QoS etc.
+
+- **Higher performance** - We plan to improve the overall performance of the
+ decoder and encoder implementations by using SIMD/NEON.
