pw_hdlc_lite/docs.rst - pigweed/pigweed - Git at Google

 .. _chapter-pw-hdlc:

 .. default-domain:: cpp

 .. highlight:: sh

 ------------
 pw_hdlc_lite
 ------------
 pw_hdlc_lite is a module that enables serial communication between devices
 using the HDLC-Lite protocol.

 Compatibility
 =============
 C++17

 Dependencies
 ============
 * ``pw_bytes``
 * ``pw_log``
 * ``pw_preprocessor``
 * ``pw_result``
 * ``pw_rpc``
 * ``pw_status``
 * ``pw_span``
 * ``pw_stream``
 * ``pw_sys_io``

 HDLC-Lite Overview
 ==================
 High-Level Data Link Control (HDLC) is a data link layer protocol which uses
 synchronous serial transmissions for communication between two devices. Unlike
 the standard HDLC protocol which uses six fields of embedded information, the
 HDLC-Lite protocol is a minimal version that only uses the bare essentials.

 The HDLC-Lite data frame in ``pw_hdlc_lite`` uses a start and end frame
 delimiter (0x7E), the escaped binary payload and the CCITT-CRC16 value.
 It looks like:

 .. code-block:: text

                                         [More frames]
     _________________________________________   _______
     | |                              |  | | |...|   | |
     | |                              |  | | |...|   | |
     |_|______________________________|__|_|_|...|___|_|
      F         Payload               CRC F F     CRC F

 Basic Overview
 ==============
 The ``pw_hdlc_lite`` module provides a simple, reliable packet-oriented
 transport that uses the HDLC-Lite protocol to send and receive data to and from
 embedded devices. This is especially needed for making RPC calls on devices
 during testing because the ``pw_rpc`` module does not handle the transmission of
 RPCs. This module enables the transmission of RPCs and other bytes through a
 serial connection.

 There are essentially two main functions of the ``pw_hdlc_lite`` module:

   * **Encoding** the data by escaping the bytes of the payload, calculating the
     CCITT-CRC16 value, constructing a data frame and sending the
     resulting data packet through serial.
   * **Decoding** the data by unescaping the received bytes, verifying the
     CCITT-CRC16 value and returning the successfully decoded packets.

 **Why use the ``pw_hdlc_lite`` module?**

   * Enables the transmission of RPCs and other data between devices over serial
   * Resilient to corruption and data loss.
   * Light-weight, simple and easy to use.
   * Supports streaming to transport without buffering - e.g. protocol buffers
     have length-prefix.

 Protocol Description
 ====================

 Encoding and sending data
 -------------------------
 This module first writes an initial frame delimiter byte (0x7E) to indicate the
 beginning of the frame. Before sending any of the payload data through serial,
 the special bytes are escaped accordingly:

             +-----------------------+----------------------+
             |Unescaped Special Bytes| Escaped Special Bytes|
             +=======================+======================+
             |       0x7E            |       0x7D5E         |
             +-----------------------+----------------------+
             |       0x7D            |       0x7D5D         |
             +-----------------------+----------------------+

 The bytes of the payload are escaped and written in a single pass. The
 CCITT-CRC16 value is calculated, escaped and written after. After this, a final
 frame delimiter byte (0x7E) is written to mark the end of the frame.

 Decoding received bytes
 -----------------------
 Packets may be received in multiple parts, so we need to store the received data
 in a buffer until the ending frame delimiter (0x7E) is read. When the
 pw_hdlc_lite decoder receives data, it unescapes it and adds it to a buffer.
 When the frame is complete, it calculates and verifies the CCITT-CRC16 bytes and
 does the following:

 * If correctly verified, the decoder returns the decoded packet.
 * If the checksum verification fails, the data packet is discarded.

 During the decoding process, the decoder essentially transitions between 3
 states, where each state indicates the method of decoding that particular byte:

 NO_PACKET --> PACKET_ACTIVE --> (ESCAPE | NO_PACKET).

 API Usage
 =========

 Encoder
 -------
 The Encoder API invloves a single function that encodes the data using the
 HDLC-Lite protocol and sends the data through the serial.

 C++
 ^^^
 In C++, this function is called ``EncodeAndWritePayload`` and it accepts a
 ConstByteSpan called payload and an object of type Writer& as arguments. It
 returns a Status object that indicates if the write was successful. This
 implementation uses the ``pw_checksum`` module to compute the CRC16 value. Since
 the function writes a starting and ending frame delimiter byte at the beginnning
 and the end of frames, it is safe to encode multiple spans. The usage of this
 function is as follows:

 .. code-block:: cpp

   #include "pw_hdlc_lite/encoder.h"
   #include "pw_hdlc_lite/sys_io_stream.h"

   int main() {
       pw::stream::SerialWriter serial_writer;
       constexpr std::array<byte, 1> test_array = { byte(0x41) };
       auto status = EncodeAndWritePayload(test_array, serial_writer);
   }

 In the example above, we expect the encoder to send the following bytes:

 - **0x7E** - Initial Frame Delimiter
 - **0x41** - Payload
 - **0x15** - LSB of the CCITT-CRC16 value
 - **0xB9** - MSB of the CCITT-CRC16 value
 - **0x7E** - End Frame Delimiter

 Python
 ^^^^^^
 In Python, the function is called ``encode_and_write_payload`` and it accepts
 the payload as a byte object and a callable to which is used to write the data.
 This function does not return anything, and uses the binascii library function
 called crc_hqx to compute the CRC16 bytes. Like the C++ function, the Python
 function also writes a frame delimiter at the beginnning and the end of frames
 so it is safe to encode multiple spans consecutively. The usage of this function
 is as follows:

 .. code-block:: python

   import serial
   from pw_hdlc_lite import encoder

   ser = serial.Serial()
   encoder.encode_and_write_payload(b'A', ser.write)

 We expect this example to give us the same result as the C++ example above since
 it encodes the same payload.

 Decoder
 -------
 The Decoder API involves a Decoder class whose main functionality is a function
 that unescapes the received bytes, adds them to a buffer and returns the
 successfully decoded packets. A class is used so that the user can call the
 decoder object's adding bytes functionality on the currently received bytes
 instead of waiting on the entire packet to arrive.

 C++
 ^^^
 The main functionality of the C++ ``Decoder`` class is the 'AddByte' function
 which accepts a single byte as an argument, unescapes it and adds it to the
 buffer. If the byte is the ending frame delimiter flag (0x7E) it attempts to
 decode the packet and returns a ``Result`` object indicating the success of the
 operation:

   * The returned ``pw::Result`` object will have status ``Status::OK`` and
     value ConstByteSpan containing the most recently decoded packet if it finds
     the end of the data-frame and successfully verifies the CRC.
   * The returned ``pw::Result`` object will have status ``Status::UNAVAILABLE``
     if it doesnt find the end of the data-frame during that function call.
   * The returned ``pw::Result`` object will have status ``Status::DATA_LOSS``
     if it finds the end of the data-frame, but the CRC-verification fails. It
     also returns this status if the packet in question does not have the
     2-byte CRC in it.
   * The returned ``pw::Result`` object will have status
     ``Status::RESOURCE_EXHAUSTED`` if the decoder buffer runs out of space.

 Here's a C++ example of reading individual bytes from serial and then using the
 decoder object to decode the received data:

 .. code-block:: cpp

   #include "pw_hdlc_lite/decoder.h"
   #include "pw_sys_io/sys_io.h"

   int main() {
     byte data;
     while (true) {
       if (!pw::sys_io::ReadByte(&data).ok()) {
         // Log serial reading error
       }
       auto decoded_packet = decoder.AddByte(data);

       if (decoded_packet.ok()) {
        // Use decoded_packet to get access to the most recently decoded packet
       }
     }
   }

 Python
 ^^^^^^
 The main functionality of the Python ``Decoder`` class is the ``add_bytes``
 generator which unescapes the bytes object argument and adds them to a buffer
 until it encounters the ending frame delimiter (0x7E) flag. The generator yields
 the decoded packets as byte objects upon successfully verification of the CRC16
 value of the received bytes. If the CRC verification fails it raises a
 CrcMismatchError exception.

 Below is an example of the usage of the decoder class to decode bytes read from
 serial:

 .. code-block:: python

   import serial
   from pw_hdlc_lite import decoder

   ser = serial.Serial()
   decode = decoder.Decoder()

   while true:
     byte = ser.read(1)

     for decoded_packet in decode.add_bytes(byte):
       # Do something with the decoded packet

 Like the C++ example, this reads individual bytes and adds them to the decoder.

 Features
 ========

 pw::stream::SerialWriter
 ------------------------
 The ``SerialWriter`` class implements the ``Writer`` interface by using sys_io
 to write data over a serial connection. This Writer object is used by the C++
 encoder to send the encoded bytes to the device.

 Roadmap & Status
 ================

 - **Additional fields** - As it currently stands, ``pw_hdlc_lite`` uses only
   three fields of control bytes: starting frame delimiter, 2-byte CRC and an
   ending frame delimiter. However, if we decided to send larger, more
   complicated RPCs and if wanted to stream log debug and error messages, we will
   require additional fields of data to ensure the different packets are sent
   correctly. Thus, in the future, we plan to add additional channel and sequence
   byte fields that could enable separate channels for pw_rpc, QoS etc.

 - **Higher performance** - We plan to improve the overall performance of the
   decoder and encoder implementations by using SIMD/NEON.
	.. _chapter-pw-hdlc:

	.. default-domain:: cpp

	.. highlight:: sh

	------------
	pw_hdlc_lite
	------------
	pw_hdlc_lite is a module that enables serial communication between devices
	using the HDLC-Lite protocol.

	Compatibility
	=============
	C++17

	Dependencies
	============
	* ``pw_bytes``
	* ``pw_log``
	* ``pw_preprocessor``
	* ``pw_result``
	* ``pw_rpc``
	* ``pw_status``
	* ``pw_span``
	* ``pw_stream``
	* ``pw_sys_io``

	HDLC-Lite Overview
	==================
	High-Level Data Link Control (HDLC) is a data link layer protocol which uses
	synchronous serial transmissions for communication between two devices. Unlike
	the standard HDLC protocol which uses six fields of embedded information, the
	HDLC-Lite protocol is a minimal version that only uses the bare essentials.

	The HDLC-Lite data frame in ``pw_hdlc_lite`` uses a start and end frame
	delimiter (0x7E), the escaped binary payload and the CCITT-CRC16 value.
	It looks like:

	.. code-block:: text

	[More frames]
	_________________________________________ _______
	\| \| \| \| \| \|...\| \| \|
	\| \| \| \| \| \|...\| \| \|
	\|_\|______________________________\|__\|_\|_\|...\|___\|_\|
	F Payload CRC F F CRC F

	Basic Overview
	==============
	The ``pw_hdlc_lite`` module provides a simple, reliable packet-oriented
	transport that uses the HDLC-Lite protocol to send and receive data to and from
	embedded devices. This is especially needed for making RPC calls on devices
	during testing because the ``pw_rpc`` module does not handle the transmission of
	RPCs. This module enables the transmission of RPCs and other bytes through a
	serial connection.

	There are essentially two main functions of the ``pw_hdlc_lite`` module:

	* Encoding the data by escaping the bytes of the payload, calculating the
	CCITT-CRC16 value, constructing a data frame and sending the
	resulting data packet through serial.
	* Decoding the data by unescaping the received bytes, verifying the
	CCITT-CRC16 value and returning the successfully decoded packets.

	Why use the ``pw_hdlc_lite`` module?

	* Enables the transmission of RPCs and other data between devices over serial
	* Resilient to corruption and data loss.
	* Light-weight, simple and easy to use.
	* Supports streaming to transport without buffering - e.g. protocol buffers
	have length-prefix.

	Protocol Description
	====================

	Encoding and sending data
	-------------------------
	This module first writes an initial frame delimiter byte (0x7E) to indicate the
	beginning of the frame. Before sending any of the payload data through serial,
	the special bytes are escaped accordingly:

	+-----------------------+----------------------+
	\|Unescaped Special Bytes\| Escaped Special Bytes\|
	+=======================+======================+
	\| 0x7E \| 0x7D5E \|
	+-----------------------+----------------------+
	\| 0x7D \| 0x7D5D \|
	+-----------------------+----------------------+

	The bytes of the payload are escaped and written in a single pass. The
	CCITT-CRC16 value is calculated, escaped and written after. After this, a final
	frame delimiter byte (0x7E) is written to mark the end of the frame.

	Decoding received bytes
	-----------------------
	Packets may be received in multiple parts, so we need to store the received data
	in a buffer until the ending frame delimiter (0x7E) is read. When the
	pw_hdlc_lite decoder receives data, it unescapes it and adds it to a buffer.
	When the frame is complete, it calculates and verifies the CCITT-CRC16 bytes and
	does the following:

	* If correctly verified, the decoder returns the decoded packet.
	* If the checksum verification fails, the data packet is discarded.

	During the decoding process, the decoder essentially transitions between 3
	states, where each state indicates the method of decoding that particular byte:

	NO_PACKET --> PACKET_ACTIVE --> (ESCAPE \| NO_PACKET).

	API Usage
	=========

	Encoder
	-------
	The Encoder API invloves a single function that encodes the data using the
	HDLC-Lite protocol and sends the data through the serial.

	C++
	^^^
	In C++, this function is called ``EncodeAndWritePayload`` and it accepts a
	ConstByteSpan called payload and an object of type Writer& as arguments. It
	returns a Status object that indicates if the write was successful. This
	implementation uses the ``pw_checksum`` module to compute the CRC16 value. Since
	the function writes a starting and ending frame delimiter byte at the beginnning
	and the end of frames, it is safe to encode multiple spans. The usage of this
	function is as follows:

	.. code-block:: cpp

	#include "pw_hdlc_lite/encoder.h"
	#include "pw_hdlc_lite/sys_io_stream.h"

	int main() {
	pw::stream::SerialWriter serial_writer;
	constexpr std::array<byte, 1> test_array = { byte(0x41) };
	auto status = EncodeAndWritePayload(test_array, serial_writer);
	}

	In the example above, we expect the encoder to send the following bytes:

	- 0x7E - Initial Frame Delimiter
	- 0x41 - Payload
	- 0x15 - LSB of the CCITT-CRC16 value
	- 0xB9 - MSB of the CCITT-CRC16 value
	- 0x7E - End Frame Delimiter

	Python
	^^^^^^
	In Python, the function is called ``encode_and_write_payload`` and it accepts
	the payload as a byte object and a callable to which is used to write the data.
	This function does not return anything, and uses the binascii library function
	called crc_hqx to compute the CRC16 bytes. Like the C++ function, the Python
	function also writes a frame delimiter at the beginnning and the end of frames
	so it is safe to encode multiple spans consecutively. The usage of this function
	is as follows:

	.. code-block:: python

	import serial
	from pw_hdlc_lite import encoder

	ser = serial.Serial()
	encoder.encode_and_write_payload(b'A', ser.write)

	We expect this example to give us the same result as the C++ example above since
	it encodes the same payload.

	Decoder
	-------
	The Decoder API involves a Decoder class whose main functionality is a function
	that unescapes the received bytes, adds them to a buffer and returns the
	successfully decoded packets. A class is used so that the user can call the
	decoder object's adding bytes functionality on the currently received bytes
	instead of waiting on the entire packet to arrive.

	C++
	^^^
	The main functionality of the C++ ``Decoder`` class is the 'AddByte' function
	which accepts a single byte as an argument, unescapes it and adds it to the
	buffer. If the byte is the ending frame delimiter flag (0x7E) it attempts to
	decode the packet and returns a ``Result`` object indicating the success of the
	operation:

	* The returned ``pw::Result`` object will have status ``Status::OK`` and
	value ConstByteSpan containing the most recently decoded packet if it finds
	the end of the data-frame and successfully verifies the CRC.
	* The returned ``pw::Result`` object will have status ``Status::UNAVAILABLE``
	if it doesnt find the end of the data-frame during that function call.
	* The returned ``pw::Result`` object will have status ``Status::DATA_LOSS``
	if it finds the end of the data-frame, but the CRC-verification fails. It
	also returns this status if the packet in question does not have the
	2-byte CRC in it.
	* The returned ``pw::Result`` object will have status
	``Status::RESOURCE_EXHAUSTED`` if the decoder buffer runs out of space.

	Here's a C++ example of reading individual bytes from serial and then using the
	decoder object to decode the received data:

	.. code-block:: cpp

	#include "pw_hdlc_lite/decoder.h"
	#include "pw_sys_io/sys_io.h"

	int main() {
	byte data;
	while (true) {
	if (!pw::sys_io::ReadByte(&data).ok()) {
	// Log serial reading error
	}
	auto decoded_packet = decoder.AddByte(data);

	if (decoded_packet.ok()) {
	// Use decoded_packet to get access to the most recently decoded packet
	}
	}
	}

	Python
	^^^^^^
	The main functionality of the Python ``Decoder`` class is the ``add_bytes``
	generator which unescapes the bytes object argument and adds them to a buffer
	until it encounters the ending frame delimiter (0x7E) flag. The generator yields
	the decoded packets as byte objects upon successfully verification of the CRC16
	value of the received bytes. If the CRC verification fails it raises a
	CrcMismatchError exception.

	Below is an example of the usage of the decoder class to decode bytes read from
	serial:

	.. code-block:: python

	import serial
	from pw_hdlc_lite import decoder

	ser = serial.Serial()
	decode = decoder.Decoder()

	while true:
	byte = ser.read(1)

	for decoded_packet in decode.add_bytes(byte):
	# Do something with the decoded packet

	Like the C++ example, this reads individual bytes and adds them to the decoder.

	Features
	========

	pw::stream::SerialWriter
	------------------------
	The ``SerialWriter`` class implements the ``Writer`` interface by using sys_io
	to write data over a serial connection. This Writer object is used by the C++
	encoder to send the encoded bytes to the device.

	Roadmap & Status
	================

	- Additional fields - As it currently stands, ``pw_hdlc_lite`` uses only
	three fields of control bytes: starting frame delimiter, 2-byte CRC and an
	ending frame delimiter. However, if we decided to send larger, more
	complicated RPCs and if wanted to stream log debug and error messages, we will
	require additional fields of data to ensure the different packets are sent
	correctly. Thus, in the future, we plan to add additional channel and sequence
	byte fields that could enable separate channels for pw_rpc, QoS etc.

	- Higher performance - We plan to improve the overall performance of the
	decoder and encoder implementations by using SIMD/NEON.