Composite Rendering: The Brilliance Behind Inspiring WebGL Transitions

Hey there! I’m Jeremy, a artistic developer at Lively Principle, a artistic expertise studio centered on crafting significant, impactful digital experiences.

Over the previous few years, I’ve turn out to be more and more involved in how WebGL experiences are structured behind the scenes, particularly on the subject of transitions, layered interfaces, and post-processing results. Rendering a single 3D scene on to the display screen works for easy instances, nevertheless it shortly turns into limiting as complexity grows.

On this walkthrough, I’ll revisit one among my earlier initiatives, Private Log 2024, to discover its implementation, break down my thought course of, and replicate on what I may have performed higher.

Going Past Simply 3D Scenes

Earlier than becoming a member of Lively Principle, I spent lots of time diving into private initiatives to sharpen my abilities and construct up my portfolio. Trying again, there are many issues I want I had understood sooner, insights that might have made an actual distinction within the work I used to be constructing. One idea, particularly, actually stands out and it’s composite rendering in WebGL.

Earlier than I proceed, you will need to know that there are various completely different names for this idea. Composite rendering may be referred to as render-to-texture, FBO compositing or multipass rendering.

At a excessive degree, composite rendering includes rendering a scene into an off-screen texture moderately than rendering it on to the display screen. This intermediate step offers us the power to control the rendered picture and add extra results. If this sounds acquainted, that’s as a result of it’s how post-processing works in Three.js. As an alternative of outputting the scene immediately, we first render it to a render goal, the place results could be utilized and layered on with a number of passes. The processed result’s then rendered both by way of the composer or, in our case, mapped to a aircraft geometry and enhanced additional with a customized shader.

// Arrange Scene A
const sceneA = new THREE.Scene();
const cameraA = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, .1, 1000);
const dice = new THREE.Mesh(
    new THREE.BoxGeometry(1, 1, 1),
    new THREE.MeshBasicMaterial()
);
sceneA.add(cameraA, dice);

// Setup Scene B - For remaining render: render goal's texture on aircraft
const sceneB = new THREE.Scene();
const cameraB = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, .1, 1000);
const aircraft = new THREE.PlaneGeometry(1, 1);
const shader = new THREE.ShaderMaterial({
    vertexShader: compositeVertex,
    fragmentShader: compositeFragment,
    uniforms: {
        uTexture: new THREE.Uniform(),
    },
});
const planeMesh = new THREE.Mesh(aircraft, shader);
sceneB.add(cameraB, planeMesh);

// Arrange Renderer
const renderer = new THREE.WebGLRenderer({
    canvas,
    antialias: true,
})
renderer.setSize(window.innerWidth, window.innerHeight);

// Arrange Render Goal
const renderTarget = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight);

// Render Loop
operate startRender() {
    renderer.setRenderTarget(renderTarget);
    shader.uniforms.uTexture.worth = renderTarget.texture;
    renderer.render(sceneA, cameraA);
    renderer.setRenderTarget(null);
    renderer.render(sceneB, cameraB);

    window.requestAnimationFrame(startRender);
}

startRender();

Having this in your toolkit opens up a brand new degree of artistic freedom. It unlocks a variety of potentialities, from transitioning between scenes to compositing textures and experimenting with extra expressive visible results.

Listed below are some examples that make the most of composite render in several ideas:

1. Lively Principle & Slosh Seltzer: Scrolling and transitioning between a number of sections.
2. Kenta Toshikura: Rendering 3D Scenes as mission thumbnails.
3. Aircord: Layering a number of scenes making a seamless transition between pages.

The Spark of Brilliance

I used to be first launched to composite rendering by way of an article by Backyard Eight that dives into the tech behind the Aircord web site. It supplied a deep rationalization on how they layer a number of scenes and deal with web page transitions, and it turned a private breakthrough shaping how I now construction scenes with out pointless duplication.

For my mission, the scene setup mirrors Backyard Eight’s method, however with fewer layers. Two components are at all times current on display screen: the face geometry and the “UI.” Whereas the UI adapts to every web page, the face geometry stays constant throughout all views with a special type of interplay.

As an alternative of duplicating the face, I arrange a most important scene containing the face geometry and a aircraft serving because the render goal. To deal with responsiveness, I calculate the render goal dimension from the digicam’s vertical subject of view and apply it proportionally to the X and Y scales.

const fovY = 
  (this.digicam.place.z + this.aircraft.place.z) *
  this.digicam.getFilmHeight() /
  this.digicam.getFocalLength();
this.aircraft.scale.set(fovY * digicam.facet, fovY, 1);

With the render goal setup, I can now regulate the render order so it sits behind the face geometry, swap the feel to the corresponding UI, and apply post-processing, which primarily acts as a router for my expertise.

Identical Work, Higher Mind

Whereas the answer above does the job, there’s at all times room to refine it to spice up efficiency, simplify the construction, and permit for extra scalability.

FIRST, moderately than rebuilding the digicam, renderer, and scene scaffolding each single time, we will summary that into reusable scene elements utilizing JavaScript’s extends function. This permits us to outline a shared basis as soon as and construct upon it.

Within the demo, I launched a BaseScene class to encapsulate a normal Three.js scene setup to deal with necessities just like the scene occasion, digicam configuration, and mission utils. On prime of that, I created an FXScene (a reputation shamelessly borrowed from Lively Principle’s inside tooling) designed particularly for scenes that require a render goal.

With this structure, we eradicate repetitive setup code whereas retaining full entry to all shared properties and behaviors. Extra importantly, because the mission grows, we will enrich the bottom courses with new capabilities like extra utilities, shared sources, debugging instruments, and each inheriting scene routinely advantages from these enhancements.

Under is a minimal instance of an FXScene, conceptually much like BaseScene however prolonged with render goal configuration.

import * as THREE from 'three';
import Expertise from '../../Expertise'; // Singleton setup to run your complete Three.js mission

export default class FXScene {
  constructor() {
    this.expertise = new Expertise();
    this.renderer = this.expertise.renderer;

    this.initScene();
    this.initCamera();
    this.initRenderTarget();

    this.sizes = this.expertise.sizes;

    this.sizes.on('resize', () => {
      this.onResize();
    });
  }

  initScene() {
    this.scene = new THREE.Scene();
    this.scene.background = null;
  }

  initCamera() {
    this.digicam = new THREE.PerspectiveCamera(45, this.sizes.width / this.sizes.peak, 1, 15);
    this.digicam.place.set(0, 0, 5);
    this.scene.add(this.digicam);
  }

  initRenderTarget() {
    this.rt = new THREE.WebGLRenderTarget(this.sizes.width, this.sizes.peak, {
      minFilter: THREE.LinearFilter,
      magFilter: THREE.LinearFilter,
      format: THREE.RGBAFormat,
      stencilBuffer: false,
    });
  }

  onResize() {
    this.digicam.facet = this.sizes.width / this.sizes.peak;
    this.digicam.updateProjectionMatrix();
  }
}

SECOND, we will simplify the composite go by rendering a fullscreen quad immediately in clip house, avoiding pointless projection calculations since there’s no want for depth testing or 3D calculations because it’s solely displaying a texture. This includes stripping out all of the matrices within the vertex shader, which you’ll see a big rectangle overlaying the scene with out them. To make sure it stays behind all different objects, you’ll be able to management its render order manually.

// const fovY = digicam.getFilmHeight() / digicam.getFocalLength();
// renderTarget.scale.set(fovY * digicam.facet, fovY, 1);
this.aircraft.renderOrder = -1;

void most important() {
  gl_Position = vec4(place.xy, 1.0, 1.0);
}

NEXT, we will simplify the routing construction and eradicate the enormous if-statement litter. By making a devoted class to handle route adjustments utilizing a lookup desk, we will streamline the logic. The category or operate would take the from and to scene textures and deal with rendering and transitioning between them seamlessly.

// Setup scenes, cameras, and render targets...

constructor() {
  this.currentView = null;
}

operate onViewChange(to) {
  let viewMap= {
    scene1: this.sceneOne.rt.texture,
    scene2: this.sceneTwo.rt.texture
  };

  if (!this.currentView) {
    this.currentView = viewMap['scene1'];
    this.shader.uniforms.uFromTexture.worth = this.currentView;
    this.shader.uniforms.uTransition.worth = 0;
    return;
  }

  if (this.currentView === viewMap[to]) return;

  this.shader.uniforms.uToTexture.worth = viewMap[to];
  this.currentView = viewMap[to];
  gsap.to(this.shader.uniforms.uTransition, {
    worth: 1,
    length: 1,
    onComplete: () => {
        this.shader.uniforms.uFromTexture.worth = viewMap[to]
        this.shader.uniforms.uTransition.worth = 0
    }
  });
}

void most important(){
  vec4 fromTexture = texture2D(uFromTexture, vUv);
  vec4 toTexture = texture2D(uToTexture, vUv);

  vec4 shade = combine(toTexture, fromTexture, uTransition);

  gl_FragColor = shade;
}

By utilizing this technique, we will simply preserve and scale our mission as we want, permitting us to have extra artistic freedom with transitions. Yuri Artiukh on YouTube gives nice examples to comply with:

combine(toTexture, fromTexture, uTransition);
combine(toTexture, fromTexture, step(uTransition, vUv.y));
combine(toTexture, fromTexture, step(uTransition, 0.5 * ( vUv.y + vUv.x )));
combine(toTexture, fromTexture, smoothstep(uTransition, uTransition + 0.3, ( vUv.x + vUv.y ) / 2.));

Lastly, within the authentic mission, there was a delicate however vital subject: the blur operate was unintentionally overwriting the alpha channel of the ultimate output. Reasonably than diving deep into the internal workings of blur itself, a cleaner answer was to maneuver the blur logic immediately into the composite shader we constructed earlier and deal with all post-processing there. The mouse fluid impact can dwell in that very same composite go as effectively.

Consolidating these steps right into a single composite shader reduces the entire variety of render passes, main to raised efficiency total. It additionally centralizes all final-stage impact processing, which improves readability and maintainability, and minimizes factors of failure—since we’re primarily working on textures making debugging much more easy.

void most important() {
  vec4 fromTexture = texture2D(uFromTexture, vUv);
  vec4 toTexture = texture2D(uToTexture, vUv);

  vec4 shade = combine(toTexture, fromTexture, uTransition);

  // Postprocessing for shade
  gl_FragColor = shade;
}

Trying Again, Shifting Ahead

Initially, thanks a lot for taking the time to learn all the best way to the top. Penning this has been an extremely rewarding expertise—revisiting work from a yr in the past and reflecting alone progress has been each humbling and deeply inspiring. I really feel genuinely grateful (and really fortunate) to have the ability to share my work, and to collaborate with such a considerate and gifted workforce at Lively Principle. I’m excited to see how the whole lot I’ve realized thus far will form the subsequent chapter of my journey as a artistic developer.

Cheers!