* Move to the correct location.
* Adjust the package name for auxiliary modules.
* Adjust the hack to import a module from scripts.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add a test suite intended to report configuration options in the outcome
file: we're only interested in SKIP vs PASS.
Add a few test cases for some interesting combinations of options. The
selection here is just for illustration purposes, more will be added later.
A subsequent commit will automatically generate test cases for single options.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Filtering on cipher suites that have RSA in their name excludes a few old
RSA-based cipher suites whose name doesn't contain RSA.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
We were only requesting 3DES cipher suites (which is weirdly restrictive
since the configuration also includes AES), but DES is in the default
exclusion list for compat.sh, so we ended up having no acceptable cipher
suites. Fix this.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
With GnuTLS servers, 3DES-CBC cipher suites are enabled by default under our
GNUTLS_LEGACY (3.3.8), but disabled by default under more recent versions
including the one we use by default on the CI (3.4.6). Even modern
versions (I checked 3.7.2) support 3DES if explicitly enabled. So
unconditionally enable 3DES-CBC for GnuTLS.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Alert if all tests are filtered out or skipped: that probably indicates a
test script that set up an unintended configuration or an overly strict
filter. You can pass `--min 0` to bypass this check. You can pass `--min`
with a larger value to require that many test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Including `mbedtls/check_config.h` from `mbedtls/config.h` is optional. If
done, `limits.h` gets included. If not done, we were missing the inclusion
of `limits.h` in several source files. Fix this and add a test build that
doesn't include `mbedtls/check_config.h`.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Replace more sample PSK by longer (GnuTLS-compatible) strings, taking care
of keeping distinct PSK distinct for wrong-PSK tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This allows many tests to pass with the system openssl and gnutls-*. As
before, not all test cases will pass due to differences between versions and
build options.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
ssl-opt.sh uses a 3-byte PSK in many test cases. Unfortunately GnuTLS >=3.4.0
rejects a PSK that is less than 4 bytes long:
> Error setting the PSK credentials: The request is invalid.
Use a longer PSK throughout ssl-opt. Only the test cases involving GnuTLS
need to change, but it's easier to do a global search-and-replace, and it's
easier to not have to worry about mismatches in constructed test cases
later, so replace everything.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When given a PSK key but no username, gnutls-cli prompts for a password.
Prevent that by passing --pskusername with the same identity that
ssl_server2 uses by default.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Only s_server has a -nocert option, s_client doesn't. Fixes OpenSSL client
test cases in PSK-only builds.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
ECDSA has two variants: deterministic (PSA_ALG_DETERMINISTIC_ECDSA) and
randomized (PSA_ALG_ECDSA). The two variants are different for signature but
identical for verification. Mbed TLS accepts either variant as the algorithm
parameter for verification even when only the other variant is supported,
so we need to handle this as a special case when generating not-supported
test cases.
In this commit:
* Automatically generated not-supported test cases for ECDSA now require
both variants to be disabled.
* Add manually written not-supported test cases for the signature
operation when exactly one variant is supported.
* Add manually written positive test cases for the verification
operation when exactly one variant is supported.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Following "PSA sign/verify: more uniform error on an unsupported hash", some
error cases are detected earlier, so there is some sloppiness in test case
dependencies that is not longer acceptable.
* In test_suite_psa_crypto, one test case for a hash+sign algorithm now
returns NOT_SUPPORTED rather than INVALID_ARGUMENT when the hash is not
supported and the key is invalid.
* In test_suite_psa_crypto_se_driver_hal_mocks, some test cases now error
out before reaching the mocks rather than after when they attempt to
use an unsupported hash.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
It makes little sense to run full compat.sh in the non-ASan component
but only partial compat.sh in the ASan component.
Actually, the non-ASan component doesn't need compat.sh at all, it's
more than covered with ssl-opt.sh and test_suite_ssl already.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
Now run everything we can with the default version in one go, then
everything that needs legacy, then next.
Don't rely on the default value of -e (EXCLUDE), use explicit values
everywhere - this makes it obvious that we are running everything.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
When we generate a test case for a mechanism that is not implemented,
comment out the test case rather than giving it a never-fulfilled
dependency. That way we don't create test cases that cannot be executed.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In automatically generated PSA test cases, we detect cryptographic
mechanisms that are not implemented, and skip the corresponding test cases.
Originally this detection was intended for mechanisms for which the PSA_WANT
symbols were not implemented, but then it morphed into skipping mechanisms
that are declared in crypto_values.h but not actually implemented. So it no
longer makes sense to skip the test cases for which a negative
dependency (!PSA_WANT_xxx) is not implemented.
This causes more not-supported test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use psa_information.TestCase for operation failure test cases.
This changes the generated output in two ways:
* Not-implemented mechanisms now have a `DEPENDENCY_NOT_IMPLEMENTED_YET_xxx`
dependency in addition to the never-fulfilled `PSA_WANT_xxx` dependency.
This does not affect when test cases run.
* ECC test cases now have correct dependency symbols, e.g.
`PSA_WANT_ECC_SECP_R1_192` instead of `PSA_WANT_ECC_FAMILY_SECP_R1`. This
is a bug fix: ECC test cases were formerly never executed because of
incorrect dependency symbols.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use psa_information.TestCase for not-supported test cases for key import and
generation.
No change to the generated output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use psa_information.TestCase for positive test cases for key generation.
The caller remains responsible for tweaking dependencies for some key
types (public keys for which the test is a negative case, RSA which requires
an additional dependency).
No change to the generated output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In automatically generated PSA test cases, annotate the test cases that are
expected to be never executed due to a dependency that is not implemented.
This was already done for not-supported test cases and for key generation,
but not for positive test cases of key usage.
You can audit which mechanisms are detected as not-implemented with
```
grep -hEo 'DEPENDENCY_NOT_IMPLEMENTED_YET_\w+' tests/suites/*.data | sort -u
```
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Create a class for test cases with automatically inferred dependencies,
suitable for PSA crypto API test cases. As of this commit, only basic cases
that use a key are supported. Subsequent commits will address negative tests
and key generation tests that require fancier handling of dependencies.
No change to the generated output.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The output is less nice, since it no longer mostly matches the order in
which symbols are used in the test case arguments. But this makes the output
more canonical, so it will be easier to notice if semantic changes occur in
subsequent commits.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Indicate which dependencies are not implemented. This makes it feasible to
audit the not-implemented detection.
The generated test cases that are detected as never-executed now have one or
more `DEPENDENCY_NOT_IMPLEMENTED_YET_PSA_WANT_xxx` instead of a single
`DEPENDENCY_NOT_IMPLEMENTED`. This does not affect when the test cases run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Skipping DTLS 1.2 with old versions was already done, but now properly
test support only once and use the results.
Skipping single-DES with new versions is new, but helps finding the right
incantation.
Note that historically, this script's policy was that it's the user's
job to find the right value of -e (EXCLUDE) for their version for
OpenSSL & config. Now it's a weird mix of that and the script doing some
detection and skipping.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This is officially a 1.2-only ciphersuite, but we also support it with
1.0 and 1.1. However we don't support it with SSLv3 (see definition in
ssl_ciphersuites.c: mininum minor version is 1, that is TLS 1.0).
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
No sure if it'd possible to build a version of OpenSSL that supports
both by tuning the config, but pretty sure improving testing for
single-DES ciphersuites is not the best use of our time in 2024.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The main fix here is that m->O and m->G should use `M_CIPHERS`. In order
to apply that though, we need to change the structure with a new for
loop and case statement. The new structure matches what's done when
actually running tests.
Note: this issue only exists in 2.28. In 3.x we now use the standard
name for display everywhere, but in 2.28 we use the name as seen by the
client for display.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
We always want to call 'filter' regardless of the values of FILTER
and EXCLUDE because it also performs standard-defined filtering like
removing RC4 ciphersuites with DTLS.
(AFAICS, not calling 'filter' when we thought it was not needed was
just a performance optimisation.)
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
The guards for the declaration and use were not consistent. This could
be resolved either way; I chose the way that matches development.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
A previous commit backported the declaration of this field as part of
conflict resolution in the hope of preventing further conflicts in the
future. But if we backport it we should do it properly. This commit
finishes the job.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
