This commit includes:
- use subprocess.check_output to report error and capture return
value
- add comment as a reminder for option --list-test-case
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
uniform_title is used to print identical format of $TITLE between
--list-test-case and run_client. In such way, no matter how $TITLE
is developed, --list-test-case will in the same format of test case
description as stored in OUTCOME.CSV.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description is printed by different block of code. This
causes code maintenance harder since we need to maintain two parts
of code with same functionality. print_test_title is used to
control test case description in compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
Test case description in compat.sh is in format of
[ogm]->[ogm] TLSmode, VERIFY CIPHERSUITE_NAME
This program calls compat.sh to list all potential test case
descriptions then checks test case duplication.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
The option --list-test-case lists all potential test cases without
executing them. The test case description is identical with $TITLE
during test case execution.
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
* Turn the warnings produced when finding non-executed tests that
are not in the allow list into errors.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
Introduce the --require-full-coverage in analyze_outcomes.py so that
when analyze_outcomes.py --require-full-coverage is called, those
tests that are not executed and are not in the allowed list issue an
error instead of a warning.
Note that it is useful to run analyze_outcomes.py on incomplete test
results, so this error mode needs to remain optional in the long
term.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The allow list explicits which test cases are allowed to not be
executed when testing. This may be, for example, because a feature
is yet to be developed but the test for that feature is already in
our code base.
Signed-off-by: Tomás González <tomasagustin.gonzalezorlando@arm.com>
The latest and earliest clang/GCC compilers are now used through
variables instead of symlinks and also the all.sh script is updated
to support options for overriding the default values.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The component functions in all.sh will be listed using
compgen instead of sed so this check is not needed.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
Running clang-16 on mbedtls reports warnings of type "-Wstrict-prototypes".
This patch fixes these warnings by adding void to functions with no
arguments. The generate_test_code.py is modified to insert void into test
functions with no arguments in *.function files.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The Ubuntu 16.04 and 22.04 docker images have been updated with
earliest and latest versions of gcc and clang respectively.
This patch adds the necessary component and support functions
required for the CI to run these compilers.
For FreeBSD we invoke the function by name so a condition is added
to disable the existing test_clang_opt function for linux.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
The seed file is part of the stable interface of PSA_CRYPTO_INJECT_ENTROPY,
because it has to survive a library upgrade on a device. So check that its
existence and content are as expected at each point in the tested life cycle.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This function was written before the PSA storage layer switched to the PSA
ITS API as its storage abstraction. Now we can just call PSA ITS functions
unconditionally.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Until now, we were never enabling this option in any test.
MBEDTLS_PSA_INJECT_ENTROPY requires MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES, so
it cannot be enabled in the full config and it gets its own component.
Test with MBEDTLS_USE_PSA_CRYPTO enabled, since MBEDTLS_PSA_INJECT_ENTROPY
is a very PSA feature (which can break non-PSA applications), and Mbed
OS (for whch MBEDTLS_PSA_INJECT_ENTROPY was designed) enables
MBEDTLS_USE_PSA_CRYPTO when it enables MBEDTLS_PSA_INJECT_ENTROPY.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When MBEDTLS_PSA_INJECT_ENTROPY is enabled, we disable standard entropy
sources, so mbedtls_entropy_func() doesn't work out of the box. Disable
tests that rely on it. MBEDTLS_PSA_INJECT_ENTROPY is intended for PSA-only
environments anyway, so it doesn't matter if some legacy features don't work
normally.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The seed file must exist before running tests. Because the location is
somewhat platform- and configuration-dependent, and to be friendly to
developers who run test suites individually and aren't familiar with this
feature, rely on the test framework code rather than on test scripts to
create the seed file.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The build option MBEDTLS_PSA_INJECT_ENTROPY requires some extra platform
functions, for historical reasons. To enable us to test this option, provide
a version of these functions for testing.
(These versions would actually work in production, but providing them in the
library in a way that doesn't break existing users might be slightly tricky,
so it's out of scope of this commit.)
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
ECP group compare function should not check the value of T. We only need
to assert the value of T after the ECP group copy function is called.
Signed-off-by: Gowtham Suresh Kumar <gowtham.sureshkumar@arm.com>
