SoftHSMv2 performance slightly disappointing

The performance of SoftHSMv2 is not quite on par with SoftHSMv1. Speed comparison on a 4-way dual-core AMD 2200:

SoftHSMv1
1 thread, 1000 signatures per thread, 1180.22 sig/s (RSA 1024 bits)
2 threads, 1000 signatures per thread, 3965.38 sig/s (RSA 1024 bits)
4 threads, 1000 signatures per thread, 10067.78 sig/s (RSA 1024 bits)
8 threads, 1000 signatures per thread, 10480.33 sig/s (RSA 1024 bits)

SoftHSMv2 with Botan
1 thread, 1000 signatures per thread, 393.19 sig/s (RSA 1024 bits)
2 threads, 1000 signatures per thread, 1193.87 sig/s (RSA 1024 bits)
4 threads, 1000 signatures per thread, 965.48 sig/s (RSA 1024 bits)
8 threads, 1000 signatures per thread, 642.17 sig/s (RSA 1024 bits)

SoftHSMv2 with OpenSSL
1 thread, 1000 signatures per thread, 444.88 sig/s (RSA 1024 bits)
2 threads, 1000 signatures per thread, 1999.82 sig/s (RSA 1024 bits)
4 threads, 1000 signatures per thread, 3864.71 sig/s (RSA 1024 bits)
8 threads, 1000 signatures per thread, 1179.35 sig/s (RSA 1024 bits)