Question

When conducting an A/B test on a learning interface, what specific metric must be normalized to ensure that the impact on content retention is statistically valid?

Accepted Answer

When conducting an A/B test on a learning interface, the specific metric that must be normalized is study time or time-on-task. Normalization in this context means adjusting the retention scores of learners so they are evaluated relative to the total time spent interacting with the material. This is necessary because if one group spends significantly longer studying the content than the other, any observed increase in retention might be caused by the extra time spent learning rather than the effectiveness of the interface design itself. By normalizing retention against study time, you calculate a learning efficiency rate, which is the amount of knowledge retained per unit of time. This adjustment ensures the statistical validity of the experiment by isolating the interface&#x27;s influence from the influence of varying exposure durations, preventing a bias known as the time-on-task confound.

Home → All Courses → Education Courses → Learning Technology Engineering → Flashcard

When conducting an A/B test on a learning interface, what specific metric must be normalized to ensure that the impact on content retention is statistically valid?

When conducting an A/B test on a learning interface, the specific metric that must be normalized is study time or time-on-task. Normalization in this context means adjusting the retention scores of learners so they are evaluated relative to the total time spent interac....