Make a test crate to make the runtime-test usable (#1258)

Fixes #833
8 jobs for master in 25 minutes and 4 seconds (queued for 1 second)