Hi community

So we started to A/B test an automatic gender detection service. It allows us to leave away the Mr./Mrs. field at the top of the form. The gender service quite accurately detects the gender based on the first name and IP location.

We’ve run the A/B test on two landing pages for more than a month now. I was absolutely sure, that hiding one form will for sure increase the conversion rate. But, in one case the conversion rate is slightly lower ( at a low confidence ) and in the other case the conversion rate is 23% lower with 97% confidence.

Now I really struggle to find an explanation for this. Apart of the the one field that is hidden in the A/B test everything is exactly the same on both landing pages. On one side there could be psychological reasons that I can’t think of. On the other side maybe technical reasons I can’t test for.

What do you think?