Split testing is not supported just yet. However, there is one potential workaround.
The reason you are seeing the same Convertable on every visit is that the script you embedded on each of the page variants is the same for both Convertables. This script is tied to clients and not to individual Convertables. The reason for this is that it makes it easier for the script to be added to a host page just once and then used for many campaigns.
So the workaround! Create the second Convertable using a different client. This Convertable will then have a different script, which you can copy and place on your second page variant. Visitors will then see the second Convertable when they visit the second page variant.
This is very much a hack and far from ideal but I hope it helps you out. Don't worry though A/B testing will be a key part of Convertables in the near future! Please feel free to let me know if there's anything unclear.