Clarification: forward and backward compatibility
The import-and-save and load-and-check parts of the tests don't have to be
actually the same test cases.
Introduce the terms “forward compatibility” and “backward compatibility” and
relate them to import-and-save and load-and-check actions.
These are clarifications of intent that do not represent an intended change
in the strategy or intended coverage.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
diff --git a/docs/architecture/testing/psa-storage-format-testing.md b/docs/architecture/testing/psa-storage-format-testing.md
index ca29f19..161ae2b 100644
--- a/docs/architecture/testing/psa-storage-format-testing.md
+++ b/docs/architecture/testing/psa-storage-format-testing.md
@@ -20,10 +20,14 @@
### Save-and-compare approach
-Importing and saving a key is deterministic. Therefore we can ensure the stability of the storage format by creating test cases under a version V of Mbed TLS, where the test case parameters include both the parameters to pass to key creation and the expected state of the storage after the key is created. The test case creates a key as indicated by the parameters, then compares the actual state of the storage with the expected state. In addition, the test case also loads the key and checks that it has the expected data and metadata.
+Importing and saving a key is deterministic. Therefore we can ensure the stability of the storage format by creating test cases under a version V of Mbed TLS, where the test case parameters include both the parameters to pass to key creation and the expected state of the storage after the key is created. The test case creates a key as indicated by the parameters, then compares the actual state of the storage with the expected state.
+
+In addition, the test case also loads the key and checks that it has the expected data and metadata. Import-and-save testing and load-and-check testing can be split into separate test functions with the same payloads.
If the test passes with version V, this means that the test data is consistent with what the implementation does. When the test later runs under version W ≥ V, it creates and reads back a storage state which is known to be identical to the state that V would have produced. Thus, this approach validates that W can read storage states created by V.
+Note that it is the combination of import-and-save passing on version V and load-and-check passing on version W with the same data that proves that version W can read back what version V wrote. From the perspective of a particular version of the library, the import-and-save tests guarantee forward compatibility while the load-and-check tests guarantee backward compatibility.
+
Use a similar approach for files other than keys where possible and relevant.
### Keeping up with storage format evolution
