[enhancement] built-in click-to-start in index.html?

[icculus]

We get bug reports from time to time in SDL, that audio isn't working in Emscripten, and of course the problem is not in SDL or Emscripten, but always that the user needs to interact with the page before the browser will allow audio to play.

While it's not a perfect fit for everything, a common solution in games is to put up a "click/tap here to start!" thing in front of the canvas, so the user has to interact with the page before the code even hits main(), so no Emscripten-specific magic has to go into the app itself to deal with audio management.

It would be extremely helpful if the default index.html that emcc outputs has this click-to-start built in, perhaps gated behind a setting in Module to enable/disable it.

It can be extremely daunting to hook into index.html and Module, especially to C programmers that aren't really familiar with HTML or Javascript, especially since index.html is something that seems to magically appear as linker output to the untrained eyes, so I think it would be super-helpful if Emscripten offered this.

(Failing that, I would also be happy with a page in the documentation that says "here's a barebones template you could paste into your index.html to add click-to-play support, and maybe extend youself to make it fancier if you're up to it.")

[sbc100]

Sounds like a reasonable request. Lets take a look and see what it might look like and how universally useful it might be. I wonder if we could somehow detect if such a dialog is needed or not. e.g. but detect the usage of audio APIs in the applications?

I assume you are aware the index.html that emscripten outputs is designed to be instructive only, and not something anyone would ship in production.. but I guess all the more reason to demonstrate how to do certain things in that exmaple?

[sbc100]

Also, BTW, we don't really have any good data on how many folks actually output .html from the linker. Emscripten works with -o foo.wasm and -o foo.js and -o foo.html and only the later (or the explicit -oformat=html) will actually trigger the output of the template html file. Do you know if many folks depend on it / use it?

[icculus]

not something anyone would ship in production

I know, but I bet they ship all the time anyhow. :)

Do you know if many folks depend on it / use it?

I don't have data on this at all, but I assume it's most people's starting point, at a minimum, and probably as far as they go if they just want to throw together something quickly, instead of a real professional production.

(Case in point: me, who threw together a quick test app for the AudioWorklet backend I'm working on for SDL3, prompting this bug report. :) )

but detect the usage of audio APIs in the applications?

My suspicion is a click-to-play interface is useful beyond access to audio output, although that's the most dire need. I could see people wanting applications to wait to start (and not spend CPU, even on startup stuff), if someone has loaded something in a background tab

But even there, there are definitely people that don't want that. It might be reasonable to leave it disabled by default so default behavior doesn't change, but a -sCLICK_TO_PLAY=1 on the linker command line saves the day, but I haven't thought this through at all.

[sbc100]

But even there, there are definitely people that don't want that. It might be reasonable to leave it disabled by default so default behavior doesn't change, but a -sCLICK_TO_PLAY=1 on the linker command line saves the day, but I haven't thought this through at all.

Sounds reasonable, yes. As usual, if we can make it work without adding a new build setting that would be great. But we can figure out the details once we get to the implementation. Speaking of which, I'm not sure when I might have time to look at this, so if anyone feels like making a PR for this that would be great. I will mark this issue as such.

[kripken]

@icculus I definitely understand the problem you raise, and this sounds like the right solution for that situation. I'd be ok with adding this option to the default HTML. However, I believe that code should default to not run, because if someone just ports a console-like application then it could be confusing for them to need to click for it to begin running.

Would improved docs + an opt-in option in the default HTML be enough, do you think?

I also wonder if we can add a runtime message in debug builds that points to those docs. That is, if we could detect that audio is not playing because the user hasn't interacted with the page, then we could show a message for the developer explaining the issue and linking to the solution. I think that's possible, and hope it would be enough to help most people?

[icculus]

So, something I'm trying with SDL today:

libsdl-org/SDL@fd75a4c

This makes the app believe it's playing audio (by having SDL throw away the generated audio in a setInterval function) until the browser actually allows the AudioContext to resume, and then hands off to that, so audio timing stays correct, apps don't fail because a callback they expected never came, buffers don't overflow, etc.

In effect, it makes games behave as they should; they just appear muted until the user interacts with the page.

This solves a lot of the problem, at least for SDL2 and SDL3 apps, but I still think having a built-in click-to-play thing is worthwhile.

However, I believe that code should default to not run

I agree; it would be a surprising change to existing projects. Having a simple switch (either set a flag in Module or a command line option or whatever) to turn it on is completely reasonable, and super-easy when someone has this problem and needs a quick fix.

Would improved docs + an opt-in option in the default HTML be enough, do you think?

Yes, 100%.

I also wonder if we can add a runtime message in debug builds that points to those docs.

I think that's a good idea, too, but it's mostly necessary because Chrome and Firefox already do this and maybe point their own (Javascript-focused) explanation pages.

Chrome:

The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu

Firefox:

An AudioContext was prevented from starting automatically. It must be created or resumed after a user gesture on the page.

Anyhow, I'm not really a master of Javascript, but I can write a basic click-to-play thing in index.html and we can decide what to do from there, if you like.

[sbc100]

This solves a lot of the problem, at least for SDL2 and SDL3 apps, but I still think having a built-in click-to-play thing is worthwhile.

That sounds like a good idea!

[kripken]

Plan sounds good to me too!

And good point about the existing devtool warnings in browsers, I remember those now that you mention them...

[ericoporto]

This makes the app believe it's playing audio (by having SDL throw away the generated audio in a setInterval function) until the browser actually allows the AudioContext to resume, and then hands off to that, so audio timing stays correct, apps don't fail because a callback they expected never came, buffers don't overflow, etc.

Hey, this is useful beyond Emscripten, if you have a desktop Windows computer without any device connected (Headphones, Speakers, HDMI monitor with audio, Bluetooth speaker, just remove all), SDL2 will detect it has no audio and I believe it won't open the audio device - so now you kinda have to work with that. Beyond this, it would be nice to be able to know this thing is in-place - so you can test if an audio device is connected and prompt the player for the alternatives - clicking on the screen in Emscripten or connecting an audio device in Desktop.

[ypwangreg]

I tried the following hack and it works.

In your generated project.html, adding a button called 'clickToRun'

    <span id='controls'>
+      <span><input type="button" value="clickToRun" onclick="clickToRun()"></span>
      <span><input type="checkbox" id="resize">Resize canvas</span>
      <span><input type="checkbox" id="pointerLock" checked>Lock/hide mouse pointer &nbsp;&nbsp;&nbsp;</span>
      <span><input type="button" value="Fullscreen" onclick="Module['canvas'].requestFullscreen()">
      </span>
    </span>

then later in the file, adding the javascript that load your YourPorject.js when the click happened.

   <!--script async type="text/javascript" src="YourPorject.js"></script-->
    <script>
      function clickToRun() {
            var script = document.createElement('script');
            script.setAttribute('type', 'text/javascript');
            script.setAttribute('src', 'YourProject.js');
            script.setAttribute('async', '');
            document.body.appendChild(script);
       }
    </script>

It fits for my simple project but maybe not enough for large complicated project. Just an idea.

I would say that nowadays, the browser is too slow and too complicated to work with. We should have a emscripten-browser that optimizes to run emscripten project only. I just want to run my vncviewer using wasm+HTML5 and how is that needed to deal with this all sorts of the limitation of AudioContext, SharedArrayBuffer, Cross-Origin-Opener-Policy etc..etc. It is a mess and without the performance, this emscripten thing would eventually fade out..