What key metric should be tracked when implementing A/B testing on listing titles to avoid accidentally *decreasingyour overall conversion rate?
When implementing A/B testing on listing titles, the key metric to track to avoid accidentally *decreasingyour overall conversion rate is the *'relevance-adjusted click-through rate' (CTR) and its impact on the 'add-to-cart' rate. Standard CTR only measures the percentage of impressions that result in a click, without accounting for the quality of those clicks. A high CTR alone can be misleading if the clicks are coming from buyers who are not actually interested in purchasing the item. Therefore, it is crucial to assess whether the new title attracts buyers w....
Community Answers
Sign in to open profiles and full community answers.
Hamza Iftikhar
βConversion rate per variant β sales divided by views (or clicks) for each title β not click-through rate. A new title can easily win on CTR by being catchier or broader, pulling in clicks from shoppers who were never going to buy; those junk clicks dilute your listing's buyer-behavior signals and drag the overall conversion rate down, which Cassini reads as declining relevance. Run the test as a true split: track conversion rate and revenue per impression for each title variant over the same period, with enough traffic for statistical significance (at least a full 7-day cycle to smooth weekday effects). Only promote the winner if it beats the control on conversion, not just clicks β and watch for the first few days after switching, because ranking can wobble while Cassini relearns the listing. If neither variant clearly wins on conversion, keep the control; a title change that only moves clicks is a risk with no reward.β
83.0%