Wycheproof

Monday Google announced the release of Project Wycheproof which are unit tests to verify cryptographic algorithms. It comes with tests for several version of Bouncy Castle, Spongy Castle and for the installation of OpenJDK. 


Let's try to make those tests running by following the documentation 

I'll use an Ubuntu VM (ubuntu-16.04-desktop-amd64) on VirtualBox 


1) Clone the repository 

First of all, make a clone of the github repo :

git clone https://github.com/google/wycheproof.git

2) Install Bazel (requisite)

From documentation, Bazel is a requisite to make the tests run. I'll install it on Ubuntu

Bazel documentation : https://bazel.build/versions/master/docs/install.html

$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -

$ sudo apt-get update && sudo apt-get install bazel

3) Run Tests - BouncyCastleAllTests

We will launch the tests for the latest version of Bouncy Castle

Inside wycheproof folder run the command :

$ bazel test BouncyCastleAllTests

Check the result

More details can be found in testlogs/BouncyCastleAllTests/test.log:

We can see that 11 tests are failed.


4) Run Tests - OpenJDKAllTests


To launch tests on OpenJDK:

$ bazel test OpenJDKAllTests

At the time of writing, 5 tests are failed  with OpenJDK 9

1) testEmptyPlaintext(com.google.security.wycheproof.CipherOutputStreamTest)
2) testKeyGenerationAll(com.google.security.wycheproof.EcKeyTest)
3) testWrongOrderEcdh(com.google.security.wycheproof.EcdhTest)
4) testBiasAll(com.google.security.wycheproof.EcdsaTest)
5) testVectorsAll(com.google.security.wycheproof.RsaSignatureTest)


So, that's it for now. Wycheproof is up and running. It is great stuff for people working with cryptography.