GPUExternalTexture

Importing Video Frames with GPUExternalTexture

Video Textures copied every video frame into a texture. WebGPU can skip the copy: device.importExternalTexture({ source: video }) wraps the current frame as a GPUExternalTexture, without a copy where the platform allows. WGSL reads it as a texture_external through textureSampleBaseClampToEdge() (no mipmaps, no repeat), converting to RGB as it samples. This page plays BookNest's promotional clip and shows a frame plain on the left and as a BookNest duotone on the right:

demos/ch04/video-external.html: a video frame as a GPUExternalTextureHTMLLive
<canvas id="promo" width="1280" height="360" style="width:640px"></canvas>
<video id="clip" src="booknest-promo.webm" muted hidden></video>
<script type="module">
const device = await (await navigator.gpu.requestAdapter()).requestDevice();
const format = navigator.gpu.getPreferredCanvasFormat(), context = promo.getContext('webgpu');
context.configure({ device, format });
const module = device.createShaderModule({ code: `
  @group(0) @binding(0) var video: texture_external;
  @group(0) @binding(1) var linear: sampler;
  struct Out { @builtin(position) position: vec4f, @location(0) uv: vec2f }
  @vertex fn vs(@builtin(vertex_index) i: u32) -> Out {
    let uv = vec2f(f32(i & 1) * 2, f32(i >> 1) * 2);        // one oversized triangle
    return Out(vec4f(uv.x * 2 - 1, 1 - uv.y * 2, 0, 1), uv);
  }
  @fragment fn fs(in: Out) -> @location(0) vec4f {
    let rgb = textureSampleBaseClampToEdge(video, linear, fract(in.uv * vec2f(2, 1))).rgb;
    let luma = dot(rgb, vec3f(0.3, 0.59, 0.11));
    let ink = mix(vec3f(0.05, 0.12, 0.25), vec3f(1.0, 0.93, 0.8), luma);    // duotone
    return vec4f(select(rgb, ink, in.uv.x >= 0.5), 1);      // right half tinted
  }` });
const pipeline = device.createRenderPipeline({ layout: 'auto', vertex: { module },
  fragment: { module, targets: [{ format }] } });
const sampler = device.createSampler({ magFilter: 'linear', minFilter: 'linear' });
await clip.play();                                          // muted, so autoplay is allowed
clip.requestVideoFrameCallback(function frame(now, { mediaTime }) {
  const external = device.importExternalTexture({ source: clip });   // this task only
  const group = device.createBindGroup({ layout: pipeline.getBindGroupLayout(0),
    entries: [{ binding: 0, resource: external }, { binding: 1, resource: sampler }] });
  const encoder = device.createCommandEncoder();
  const pass = encoder.beginRenderPass({ colorAttachments: [{ loadOp: 'clear',
    storeOp: 'store', view: context.getCurrentTexture().createView() }] });
  pass.setPipeline(pipeline), pass.setBindGroup(0, group), pass.draw(3), pass.end();
  device.queue.submit([encoder.finish()]);
  if (mediaTime < 1.5) return clip.requestVideoFrameCallback(frame);   // the next frame
  clip.pause();
  window.__done = true;
});
</script>
Browser output of Listing 4.106
Browser output of 106

The import sits inside the frame callback because an external texture from a <video> expires when the task that imported it ends: resubmitting the last bind group from a setTimeout() failed validation with "External texture ... used in a submit is not active". Import once per frame and rebuild the bind group. While the paused video showed the same frame, a second import returned the same object (=== was true) and un-expired it.