Marques ‘MKBHD’ Brownlee has raised concerns over YouTube’s upcoming video A/B testing feature, calling it one of the platform’s biggest changes in recent years.
YouTube revealed video A/B testing during its latest Made on YouTube event alongside several other additions, including custom homepages, new livestream features, and AI-powered Shorts editing tools.
While Brownlee acknowledged that some of YouTube’s recent changes have been useful, he took issue with the platform allowing creators to test different versions of the videos themselves.
“But this latest one, in my opinion, is their craziest, riskiest, and possibly worst idea yet,” Brownlee said.
The feature expands on YouTube’s existing testing tools for thumbnails and titles. Instead of changing how a video is presented to potential viewers, creators will be able to upload different cuts of the video itself and have YouTube test them with real audiences.
MKBHD raises concerns over YouTube video A/B testing
YouTube presented the feature as a way for creators to test different opening hooks or edits and determine which version keeps viewers watching for longer.
According to Brownlee, creators will be able to compare the retention graphs for up to three versions before choosing which cut ultimately remains on the platform.
However, the YouTuber questioned what that could mean for viewers who may unknowingly receive different versions of the same upload.
“As a viewer, if I’m watching a video, do I know if that video is part of an A/B test or not? Probably not,” Brownlee said.
“If I’m reading the comments on a video, how do I know which version of the upload they watched? And if someone’s comment [has a] timestamp, does that break if I’m watching the other version of that video?”
Brownlee said he spoke with YouTube engineers about the feature and learned more about how the company intends to prevent creators from uploading substantially different videos as part of the same test.
YouTube reportedly plans to use a semantic analysis model that examines each uploaded video and determines how similar the different cuts are. Only videos that receive a sufficiently high similarity score will be eligible for A/B testing.
The company also plans to restrict video A/B testing to the launch period of new uploads, meaning creators will not be able to go through their older videos and begin testing replacement cuts.
Despite those restrictions, Brownlee said several questions remain, including how timestamped comments will work and whether viewers will know which version of a video they watched.
He also said it could be possible for someone to watch one version of a video during testing, only to return later and find that a different cut won the test and became the permanent version.
“There’s this sort of unspoken rule of community on YouTube that, like, we’re all having the same experience together. We’re all watching the same video,” Brownlee said.
“So letting people put up two or three different versions of the same video at the same time just kind of fundamentally breaks that.”
Brownlee also argued that creators focusing too heavily on retention could affect how they approach making videos.
“This is being framed as a tool to help you improve your videos, but I think for 99.9% of creators, well, to be literal, it’s not actually helping you make better videos,” he said. “It’s helping you find which version of your video has the best retention, which doesn’t necessarily mean it’s a better video.”
YouTube is expected to roll out video A/B testing more broadly early next year, according to Brownlee, with the similarity requirements designed to prevent creators from testing completely different uploads against one another.
I’m Abhishek Sharma, an author at TigerJek.com. I enjoy exploring games, testing different strategies, and turning what I learn into clear, useful guides. My goal is to help players understand the game better and improve without the usual confusion.
