Chrome Audio is Broken

In which we talk about Chrome's new autoplay policy, the backlash, and where things went wrong.

Don't want to read the whole article? Skip directly to the demo!

The Policy

In Chrome 66 for desktop Google announced a new policy around how the browser plays audio. Upon visiting most sites, autoplay for video and audio content will be blocked. Pages are still able to play video and audio, but only after a user gesture has occurred.

Not all pages have their audio blocked. Chrome ships with a global whitelist of over 1000 sites which are exempted from the policy based on past user metrics. Additionally, user metrics are tracked while browsing new sites. Over time, sites that are muted often by a user will be blacklisted from autoplaying content. Sites where users regularly play video will be whitelisted.

This policy has attracted a great deal of criticism from developers. It's also been subjected to a surprising amount of misinformation surrounding its implementation and rollout. Chrome's audio policy may sound reasonable at first glance, but it has significant problems.

And Google doesn't seem to be in much of a hurry to correct them.

The Users

I'm going to spend a lot of time talking about the web developer community, but before I do we first need to talk about users. As developers we ought to be able to tolerate a fair amount of pain in our daily jobs if it means that our products are more pleasant and useful for end users.

Much of the criticism that I'm about to levy could be overlooked if it could be demonstrated that the current policy was necessary to give users a better experience on the web.

To be blunt, Chrome's current autoplay policy is not good for users. Even if the policy had broken nothing I still believe that no adequately informed user would ever be excited about what the Chromium team released. It is frankly one of the most striking examples of bad design that I've seen from Google in a long while.

The Exceptions

Chrome's initial announcement claimed that it was now blocking over half of unwanted audio plays. If you're just now thinking that statistic actually sounds pretty low, you're not alone. It is very low.

Unfortunately, Chrome's policy is trivially easy for malicious advertisers to bypass. There's a wide range of actions that the browser will interpret as user consent to play audio. These include:

  • Pressing any key on a keyboard (including arrow keys).
  • Having your developer tools open.
  • Clicking anywhere within the page (even just to focus the browser).
  • highlighting text anywhere on the page.

This means that as an advertiser you can be pretty sure that anyone who visits a link will eventually give it permission to play audio. I've built a small example page to demonstrate just how easy this is to take advantage of.

It's only a few lines of code: 

var video = document.querySelector('video');
function play() {
    var playing = video.currentTime > 0 && !video.paused;

    if (!playing) {
        video.play();
        setTimeout(play, 1000);
    }
};

setTimeout(play, 1000);

In addition, navigating to a page from within the same domain will automatically lift autoplay restrictions for that page regardless of whether or not the user interacts with it. If you're using Chrome and have already clicked on the example page above, you may have noticed that I didn't need you to interact with the page in order to play audio.

If you want Chrome's autoplay policy to be enforced at all on my demo you need to either be linked to it from another domain, or you need to manually copy the demo address, open a new tab, and navigate to it directly.

In light of how easy it is to accidentally allow a page to play sound, it's pretty telling that the one group that hasn't put up much of a fuss over Chrome's new policy is advertisers. I suspect this is largely because advertisers know they don't need to care. They can either leave their code untouched and still get a fairly sizable chunk of plays or they can employ any number of dark patterns to force users to give them an interaction before they play.

The UX

Even if autoplay blocking worked as intended, the UX would still be problematic. Google has a nasty habit of removing user controls from a large portion of the software and hardware they release. With autoplay, there's no option or button you can click to whitelist or blacklist a domain; everything is instead algorithmically determined.

It's kind of obviously bad UX — predictable behavior and explicit affordances are the basic building blocks of any decent user interface. But Chrome's design takes this a step farther; not only are autoplay permissions algorithmically determined, the user is also given no obvious mechanism to see what the current permission is.

There's also a questionable assumption at the heart of the algorithm itself: it acts as though high media engagement directly maps to a desire for a domain to autoplay videos. This isn't always true.

A user may regularly watch a lot of videos on CNET or Youtube without wanting those videos to autoplay or blast sound when opening a new tab. Chrome's design leaves them with no controls to exercise that preference.

The consequence of Google being so averse to building user-understandable controls and so eager to tie their policies to whatever new user metric they have lying around is that users are now forced to adapt themselves to Chrome's interface and behaviors, rather than the other way around.

The Choice

From the feature's first announcement the Chromium team has phrased debate over autoplay policies as a fight between developers and users. The reality is that many of the developers complaining about the current policy are also users who hate autoplaying ads. Few of them believe that addressing their concerns would require cheapening the web or disregarding users.

In fact many of the developers who have complained to Google (myself included) would actually like to see a much stricter policy than what Chrome already shipped. We've argued that a 50% block rate on unintended audio simply isn't good enough.

In light of these problems, even if there were no developer issues at all, it would still be worth taking the policy back to the drawing board.

The Developers

Unfortunately there are developer issues. Chrome's autoplay policy somehow manages to strike a balance of being both lax for advertisers and irritating for normal developers at the same time.

Performing a user gesture doesn't automatically resume audio, which means that background sounds and music simply won't play when using the Web Audio API. To get around this issue developers need to put apps and videos behind some kind of splash screen.

This can work for many app genres, but not for all of them. Throwing additional barriers in front of users can often cheapen an end product. This can make experimental and innovative sites feel less like full-fledged apps and more like simple toys.

Audio-based notifications are worth calling out specifically. When a user opens up a new tab on Slack or Facebook, they expect that tab to be able to play audio and alert them of new messages. Instead the new tab will be obliquely muted with no indication to the user. Users may not realize they need to interact with the page before their notifications will start working.

Even if this policy could be adapted into a good experience for every game or app, a large amount of content simply won't be updated — no matter what effort Google puts into trying to get developers to do so.

Legacy content and distributed content make up a huge portion of the web: apps that are either unmaintained or that are archived and hosted by third parties who don't understand the inner workings of the code. Many games have been bundled and shipped through distribution channels that developers no longer have access to.

It is irresponsible for anyone, especially Google, to claim that developers can simply update their apps. The web doesn't work that way.

The Bugs

On top of everything else it's clear that the Chromium team did not adequately test features and websites that would be affected by the policy. On release it wasn't just indie sites that broke: a number of Chrome's own web experiments and showcase pages simply stopped working.

There were entire APIs that were never even considered as possible areas of impact. Controller input flat out doesn't count as a web gesture; meaning that web-based games on dedicated devices or gaming consoles have literally no way to enable audio. A bug alerting the Chromium team to the potential problem has sat unclosed in Chromium's bugtracker for over three years.

Even within the core feature itself there are bugs: the Media Engagement Index (the core of the entire functionality) only takes into account user-initiated plays of audio elements, which means that Web Audio experiences will never be whitelisted regardless of how the user interacts with the page.

The Response

Shortly after Chrome 66 was released, several prominent game developers took to Twitter and the media to complain.

Within a day Google had responded maintaining that the feature was well documented and that developers had already been given a full year to update their code.

Undeterred, developers created an issue requesting that the policy be rolled back until it could be updated to address concerns. It has since become one of the top starred issues within Chromium's tracker.

As more of the tech media started to pick up on the story, the Chromium team abruptly announced that they would be rolling back their changes to the Web Audio API.

This rollback would not apply to autoplay on audio or video tags within HTML itself. It also came with a new deadline: Chrome 70 in October would see the feature reenabled. The reasoning offered for the delay was, "to give Web Audio developers (e.g. gaming, audio applications, some RTC features) more time to update their code."

Developers weren't happy.

The original issue that had been raised was not that developers needed more time to update. Legacy content on the web often can not be updated at all. The issue wasn't that the timing was bad; it was that the policy itself was bad.

Chromium developers offered a small olive branch suggesting that internally they were in discussions about changing the policy. However they could offer no deadlines for when decisions would be made or when communication would resume. None of the original criticisms or suggestions were addressed or even mentioned.

Critics took this as a not-so-subtle hint that the Chromium team still saw this as a developer problem rather than as a set of issues with the policy itself. From their point of view, upon being given criticism the Chromium team had simply dug in their heels and decided to push on.

About a week later Chromium's official Twitter account released a separate update reiterating that the policy would be reintroduced in October. No mention of internal discussions was made this time.

The Distrust

This is where that whitelist comes back into play. The fact that Google exempted over 1000 domains (including Google owned sites like Youtube) from being blocked left developers feeling like Google must have known from the start that their current policy wouldn't work with every site.

Critics began to accuse Google of favoritism. If the restrictions weren't burdensome then why was Google exempting itself? What developers may have initially considered to be an honest mistake they now saw as a blatant effort by Google to either crush competition or to exempt themselves from a policy that was obviously insufficient for many websites.

As if to further push that narrative, the Chromium team has been careful to avoid publicly releasing any of the contents of this whitelist.

To be clear, I have no reason to doubt that the whitelist Chrome is using is based on pure usage statistics. But the list's validity doesn't matter: the fact that it is obfuscated at all still makes it easy for critics to accuse Google of trying to hide malicious intent.

Because the site list is built deterministically without any human customization there's no real need to hide the result. Because Google has been relatively straightforward about how MEI is calculated there's no new information that a malicious actor could learn about how to game the system.

The only practical benefit is in preventing end users from being able to see which sites Google's algorithm favors in practice.

The Platform

It is unfortunate that of all the people Google could have upset, they went with game and app developers. It means that Google's misstep will likely have consequences beyond just the Chrome browser.

The backlash from these communities shouldn't be viewed as an isolated event — it's been brewing for a while. Web audio is already something of a pain point for many developers. Google's stance that queuing audio is no big deal feels like another dismissal of concerns about API complexity as a barrier to entry. Add to that the bugs around controller support and it's not hard to see why game developers feel like Google is either overlooking or ignoring their use cases.

I don't think it can be overstated how harmful episodes like this are for the web as a platform. Web standard committees, browser makers, and web advocates in general have been fighting an uphill battle to convince developers (and particularly game developers) to treat the web as a serious platform that should be considered as a valid target alongside native.

One of the most effective arguments for the web right now is its stability. Our mantra has always been that we do not break the web.

It is a lot of work to build an API that is consistently backwards compatible. Sometimes it even requires making tough, controversial decisions. Most languages and platforms just decide that it isn't worth the effort, instead requiring older users to maintain segregated environments. But the web is different — it's evergreen and thus has a much stricter set of responsibilities.

When leading browsers like Chrome maintain that breaking existing web content is acceptable, one of the most effective tools for moving the web forward is undermined. Why should a developer target a platform as shifting and as varied and as diverse and (frankly) as difficult as the modern web if they can't be sure that the platform is stable?

The Debt

The impression that a policy gives is often more important in the long term than the intentions behind it. Regardless of Google's intentions, their actions will reinforce preconceptions among the ordinary developer community that Google doesn't care about them.

These mistakes create a narrative undercurrent that will undermine Google's future efforts to get developers to trust them when they're forced to make difficult decisions.

When controversy looms, Google regularly falls back on reassurances that internal managers and employees have the public's back. That may very well be true, but without transparency it doesn't matter. Given enough examples of Google dismissing concerns about its pet projects, the public will simply stop believing that the company cares.

The Solutions

I would be remiss if I didn't at this point talk about some possible solutions.

As it turns out there are a number of alternate policies that Chrome could implement that would be better for both developers and users. Developers have been making a variety of suggestions in different forums. A lot of these suggestions have one element in common: mute audio at the browser level rather than messing with Javascript APIs.

My personal favorite solution would also be the easiest change for Chrome to make. Chrome already ships with a set of controls to mute audio output, so why doesn't it just use those?

Existing Chrome audio controls

Pretty much the only thing Chrome would need to do to build a better policy for both users and developers would be to change the default setting on new tabs from "Allow" to "Block."

The Chromium team could even keep all of the other additions if they really wanted to (with the caveat being that most of their additions are really bad for end users). They could override that default setting based on a global whitelist. They could randomly unmute tabs based on poorly defined gestures. They could even use MEI as way to determine the output.

Even if Chrome did reintroduce all of the extra cruft from their current policy (and to be clear, I do believe it would be a mistake to do so) it would still be better than what they're doing right now. At the very least, users would now have a clear indication of whether or not a tab was muted. At the very least, nothing on the web would break.

The Takeaway

There are a lot of lessons to take away from this whole experience ranging from community management, to UX design, to general software development practice:

  • If you want to figure out user intent, ask the user. If you’re guessing user intent, provide the user with mechanisms to explicitly (not just implicitly) correct you.
  • Gather feedback to understand the problem before you come up with a solution.
  • Eat your own dog food. Don’t subject users to a policy you’re not willing to follow yourself.
  • Test your code before you ship it.
  • Don’t respond to criticism by passive-aggressively accusing critics of being lazy or misinformed. Listen before you respond to criticism, not after.
  • And finally, even if it doesn’t always feel like it, we’re all in this together.

That last point is worth zeroing in on. I realize that I've been offering some fairly aggressive criticism both towards Google as a company and towards the development team behind Chrome. But I'm not actually mad at anybody and I don't think any of the other devs who originally commented on this policy are either.

Evolving the web platform is difficult. It doesn't matter how smart you are; eventually you are going to mess something up. The way we protect against these missteps is that we work together as a community. If anyone from the Chrome dev team is reading this, if you take anything away from this article, the one thing you should take away is that you do not have to work alone.

Pretty much every problem that I talked about above could have been avoided by involving the broader web community in this decision process; by treating developer consensus as a quality metric rather than as a delivery target. I firmly believe that this entire article would be different if Chromium devs had begun their decision making process with a real understanding that the web is bigger than Google and that keeping it great requires input from diverse stakeholders.

The most frustrating part of all of this has been that there still is no indication that the Chromium team is interested in collaboration. Even at this point the entire situation could be improved by the Chromium team providing any kind of credible commitment at all to addressing developer concerns or to including developers in the final decision making process.

We know that developing software takes time, we know that decisions can't be made overnight. We just want to know that someone cares.

Instead Google has offered the same response that it always offers when it falls into controversy: "we're looking into it." In other words, "shut up and trust us."

To be blunt, "no you don't."

Footnotes

I would be remiss if I didn't draw attention to the excellent work that Ashley from the Construct team has done here, both in raising awareness of the problem and in reaching out to the Chrome team to fix it. You can read their rundown to get another perspective beyond just mine.

A ton of other people have also put in a lot of work to increase visibility of this issue and suss out the messy details about it. Where possible I've tried to include or link to the people who made me aware of any given piece of information.

If you want to encourage the Chrome team to reconsider this policy, you can star or comment on the main bug, or reach out to @ChromiumDev on Twitter.

And of course if you notice an error in this article please feel free to file an issue or reach out to me directly.