How one can Create a Liquid Raymarching Scene Utilizing Three.js Shading Language

I’ve all the time been fascinated by shaders. The concept that items of code can create among the most awe-inspiring visuals you see in video video games, motion pictures, and on the internet has pushed me to study as a lot as I can about them.

Throughout that journey, I got here throughout a video by Inigo Quilez referred to as Portray a Character with Maths. It’s a frankly mind-blowing instance of a method referred to as Raymarching. Primarily, it’s a option to construct or render complicated 2D and 3D scenes in a single fragment shader while not having complicated fashions or supplies.

Whereas that instance is absolutely spectacular, additionally it is fairly intimidating! So, to ease us into this idea, we’ll discover issues just like metaballs, these extraordinarily cool-looking gloopy, liquid shapes that appear to soak up into one another in fascinating methods.

Raymarching is a big matter to cowl, however there are some glorious, in-depth sources and tutorials accessible in the event you’re concerned about going lots deeper. For this tutorial, we’re going to base the raymarching methods on this tutorial by Kishimisu: An Introduction to Raymarching, with many references to 3D SDF Assets by Inigo Quilez. When you’re after one thing extra in-depth, I extremely advocate the excellently written Portray with Math: A Mild Examine of Raymarching by Maxime Heckel.

On this tutorial, we are going to construct a easy raymarched scene with fascinating lighting utilizing React Three Fiber (R3F) and Three.js Shader Language (TSL). You’ll need some data of Three.js and React, however the methods right here might be utilized in any shading language resembling GLSL, and any WebGL framework (so, OGL or vanilla is totally attainable).

The Setup

We’re going to be utilizing Three.js Shading Language, a brand new and evolving language that goals to decrease the barrier of entry for creating shaders by offering an approachable setting for many who aren’t so accustomed to issues like GLSL or WGSL.

TSL requires the WebGPURenderer in Three.js for the time being. Which means if WebGPU is on the market, the TSL we write will compile all the way down to WGSL (the shading language utilized in WebGPU) and can fall again to GLSL (WebGL) if wanted. As we’re utilizing R3F, we’ll arrange a really fundamental canvas and scene with a single aircraft, in addition to a uniform that comprises details about the display decision that we’ll use in our raymarched scene. First, we have to arrange the Canvas in R3F:

import { Canvas, CanvasProps } from '@react-three/fiber'
import { useEffect, useState } from 'react'

import { AdaptiveDpr } from '@react-three/drei'

import WebGPUCapabilities from 'three/examples/jsm/capabilities/WebGPU.js'
import WebGPURenderer from 'three/examples/jsm/renderers/webgpu/WebGPURenderer.js'
import { ACESFilmicToneMapping, SRGBColorSpace } from 'three'

const WebGPUCanvas = ({
  webglFallback = true,
  frameloop = 'all the time',
  kids,
  debug,
  ...props
}) => {
  const [canvasFrameloop, setCanvasFrameloop] = useState('by no means')
  const [initialising, setInitialising] = useState(true)

  useEffect(() => {
    if (initialising) return

    setCanvasFrameloop(frameloop)
  }, [initialising, frameloop])

  const webGPUAvailable = WebGPUCapabilities.isAvailable()

  return (
    <Canvas
      {...props}
      id='gl'
      frameloop={canvasFrameloop}
      gl={(canvas) => {
        const renderer = new WebGPURenderer({
          canvas: canvas,
          antialias: true,
          alpha: true,
          forceWebGL: !webGPUAvailable,
        })
        renderer.toneMapping = ACESFilmicToneMapping
        renderer.outputColorSpace = SRGBColorSpace
        renderer.init().then(() => {
          setInitialising(false)
        })

        return renderer
      }}
    >
      <AdaptiveDpr />

      {kids}
    </Canvas>
  )
}

Now that we’ve set this up, let’s create a fundamental part for our scene utilizing a MeshBasicNodeMaterial the place we are going to write our shader code. From right here, all of our code can be written for this materials.

import { useThree } from '@react-three/fiber'

import {
  MeshBasicNodeMaterial,
  uniform,
  uv,
  vec3,
  viewportResolution
} from 'three/nodes'

const raymarchMaterial = new MeshBasicNodeMaterial()

raymarchMaterial.colorNode = vec3(uv(), 1)

const Raymarch = () => {
  const { width, peak } = useThree((state) => state.viewport)

  return (
    <mesh scale={[width, height, 1]}>
      <planeGeometry args={[1, 1]} />
      <primitive object={raymarchMaterial} connect='materials' />
    </mesh>
  )
}

Creating the Raymarching Loop

Raymarching, at its most fundamental, includes stepping alongside rays solid from an origin level (resembling a digicam) in small increments (referred to as marching) and testing for intersections with objects within the scene. This course of continues till an object is hit, or if we attain a most distance from the origin level. As that is dealt with in a fraction shader, this course of occurs for each output picture pixel within the scene. (Notice that each one new capabilities resembling float or vec3 are imports from three/nodes).

const sdf = tslFn(([pos]: any) => {
  // That is our major "scene" the place objects will go, however for now return 0
  return float(0)
})

const raymarch = tslFn(() => {
  // Use frag coordinates to get an aspect-fixed UV
  const _uv = uv().mul(viewportResolution.xy).mul(2).sub(viewportResolution.xy).div(viewportResolution.y)

  // Initialize the ray and its path
  const rayOrigin = vec3(0, 0, -3)
  const rayDirection = vec3(_uv, 1).normalize()

  // Whole distance travelled - be aware that toVar is essential right here so we will assign to this variable
  const t = float(0).toVar()

  // Calculate the preliminary place of the ray - this var is asserted right here so we will use it in lighting calculations later
  const ray = rayOrigin.add(rayDirection.mul(t)).toVar()

  loop({ begin: 1, finish: 80 }, () => {
    const d = sdf(ray) // present distance to the scene

    t.addAssign(d) // "march" the ray

    ray.assign(rayOrigin.add(rayDirection.mul(t))) // place alongside the ray

    // If we're shut sufficient, it is a hit, so we will do an early return
    If(d.lessThan(0.001), () => {
      Break()
    })

    // If we have travelled too far, we will return now and think about that this ray did not hit something
    If(t.greaterThan(100), () => {
      Break()
    })
  })

  // Some very fundamental shading right here - objects which are nearer to the rayOrigin can be darkish, and objects additional away can be lighter
  return vec3(t.mul(0.2))
})()

raymarchMaterial.colorNode = raymarch

What you would possibly discover right here is that we’re not really testing for actual intersections, and we’re not utilizing mounted distances for every of our steps. So, how do we all know if our ray has “hit” an object within the scene? The reply is that the scene is made up of Signed Distance Fields (SDFs).

SDFs are based mostly on the idea of calculating the shortest distance from any level in area to the floor of a form. So, the worth returned by an SDF is constructive if the purpose is outdoors the form, adverse if inside, and nil precisely on the floor.

With this in thoughts, we actually solely want to find out if a ray is “shut sufficient” to a floor for it to be a success. Every successive step travels the gap to the closest floor, so as soon as we cross some small threshold near 0, we’ve successfully “hit” a floor, permitting us to do an early return.

(If we saved marching till the gap was 0, we’d successfully simply preserve working the loop till we ran out of iterations, which—whereas it might get the outcome we’re after—is lots much less environment friendly.)

Including SDF Shapes

Our SDF operate here’s a comfort operate to construct the scene. It’s a spot the place we will add some SDF shapes, manipulating the place and attributes of every form to get the outcome that we would like. Let’s begin with a sphere, rendering it within the middle of the viewport:

const sdSphere = tslFn(([p, r]) => {
  return p.size().sub(r)
})

const sdf = tslFn(([pos]) => {
  // Replace the sdf operate so as to add our sphere right here
  const sphere = sdSphere(pos, 0.3)

  return sphere
})

We are able to change how massive or small it’s by altering the radius, or by altering its place alongside the z axis (so nearer, or additional away from the origin level)

That is the place we will additionally do another cool stuff, like change its place based mostly on time and a sin curve (be aware that each one of those new capabilities resembling sin, or timerLocal are all imports from three/nodes):

const timer = timerLocal(1)

const sdf = tslFn(([pos]) => {
  // Translate the place alongside the x-axis so the form strikes left to proper
  const translatedPos = pos.add(vec3(sin(timer), 0, 0))
  const sphere = sdSphere(translatedPos, 0.5)

  return sphere
})

// Notice: that we will additionally use oscSine() rather than sin(timer), however as it's within the vary
// 0 to 1, we have to remap it to the vary -1 to 1
const sdf = tslFn(([pos]) => {
  const translatedPos = pos.add(vec3(oscSine().mul(2).sub(1), 0, 0))
  const sphere = sdSphere(translatedPos, 0.5)

  return sphere
})

Now we will add a second sphere in the midst of the display that doesn’t transfer, so we will present the way it matches within the scene:

const sdf = tslFn(([pos]: any) => {
  const translatedPos = pos.add(vec3(sin(timer), 0, 0))
  const sphere = sdSphere(translatedPos, 0.5)
  const secondSphere = sdSphere(pos, 0.3)

  return min(secondSphere, sphere)
})

See how we use the min operate right here to mix the shapes once they overlap. This takes two enter SDFs and determines the closest one, successfully making a single area. However the edges are sharp; the place’s the gloopiness? That’s the place some extra math comes into play.

Clean Minimal: The Secret Sauce

Clean Minimal is minimal, however clean! Inigo Quilez’s article is the most effective useful resource for extra details about how this works, however let’s implement it utilizing TSL and see the outcome:

const smin = tslFn(([a, b, k]: any) => {
  const h = max(okay.sub(abs(a.sub(b))), 0).div(okay)
  return min(a, b).sub(h.mul(h).mul(okay).mul(0.25))
})

const sdf = tslFn(([pos]: any) => {
  const translatedPos = pos.add(vec3(sin(timer), 0, 0))
  const sphere = sdSphere(translatedPos, 0.5)
  const secondSphere = sdSphere(pos, 0.3)

  return smin(secondSphere, sphere, 0.3)
})

Right here it’s! Our gloopiness! However the outcome right here is fairly flat, so let’s do some lighting to get a extremely cool look

Including Lighting

Up up to now, we’ve been working with quite simple, flat shading based mostly on the gap to a specific floor, so our scene “seems to be” 3D, however we will make it look actually cool with some lighting

Including lighting is a good way to create depth and dynamism, so let’s add quite a lot of totally different lighting results in TSL. This part is a little bit of an “added additional,” so I gained’t go into each kind of lighting. When you’d wish to study extra in regards to the lighting used right here and shaders usually, right here is a superb paid course that I completely advocate: https://simondev.teachable.com/p/glsl-shaders-from-scratch.

On this demo, we’re going so as to add ambient lighting, hemisphere lighting, diffuse and specular lighting, and a fresnel impact. This seems like lots, however every of those lighting results is simply a few traces every! For a lot of of those methods, we might want to calculate normals, once more because of Inigo Quilez.

const calcNormal = tslFn(([p]) => {
  const eps = float(0.0001)
  const h = vec2(eps, 0)
  return normalize(
    vec3(
      sdf(p.add(h.xyy)).sub(sdf(p.sub(h.xyy))),
      sdf(p.add(h.yxy)).sub(sdf(p.sub(h.yxy))),
      sdf(p.add(h.yyx)).sub(sdf(p.sub(h.yyx))),
    ),
  )
})

const raymarch = tslFn(() => {
  // Use frag coordinates to get an aspect-fixed UV
  const _uv = uv().mul(decision.xy).mul(2).sub(decision.xy).div(decision.y)

  // Initialize the ray and its path
  const rayOrigin = vec3(0, 0, -3)
  const rayDirection = vec3(_uv, 1).normalize()

  // Whole distance travelled - be aware that toVar is essential right here so we will assign to this variable
  const t = float(0).toVar()

  // Calculate the preliminary place of the ray - this var is asserted right here so we will use it in lighting calculations later
  const ray = rayOrigin.add(rayDirection.mul(t)).toVar()

  loop({ begin: 1, finish: 80 }, () => {
    const d = sdf(ray) // present distance to the scene

    t.addAssign(d) // "march" the ray

    ray.assign(rayOrigin.add(rayDirection.mul(t))) // place alongside the ray

    // If we're shut sufficient, it is a hit, so we will do an early return
    If(d.lessThan(0.001), () => {
      Break()
    })

    // If we have travelled too far, we will return now and think about that this ray did not hit something
    If(t.greaterThan(100), () => {
      Break()
    })
  })

  return lighting(rayOrigin, ray)
})()

A traditional is a vector that’s perpendicular to a different vector, so on this case, you’ll be able to consider normals as how gentle will work together with the floor of the item (consider how gentle bounces off a floor). We’ll use these in a lot of our lighting calculations:

const lighting = tslFn(([ro, r]) => {
  const regular = calcNormal(r)
  const viewDir = normalize(ro.sub(r))

  // Step 1: Ambient gentle
  const ambient = vec3(0.2)

  // Step 2: Diffuse lighting - provides our form a 3D look by simulating how gentle displays in all instructions
  const lightDir = normalize(vec3(1, 1, 1))
  const lightColor = vec3(1, 1, 0.9)
  const dp = max(0, dot(lightDir, regular))

  const diffuse = dp.mul(lightColor)

  // Steo 3: Hemisphere gentle - a combination between a sky and floor color based mostly on normals
  const skyColor = vec3(0, 0.3, 0.6)
  const groundColor = vec3(0.6, 0.3, 0.1)

  const hemiMix = regular.y.mul(0.5).add(0.5)
  const hemi = combine(groundColor, skyColor, hemiMix)

  // Step 4: Phong specular - Reflective gentle and highlights
  const ph = normalize(mirror(lightDir.negate(), regular))
  const phongValue = max(0, dot(viewDir, ph)).pow(32)

  const specular = vec3(phongValue).toVar()

  // Step 5: Fresnel impact - makes our specular spotlight extra pronounced at totally different viewing angles
  const fresnel = float(1)
    .sub(max(0, dot(viewDir, regular)))
    .pow(2)

  specular.mulAssign(fresnel)

  // Lighting is a mixture of ambient, hemi, diffuse, then specular added on the finish
  // We're multiplying these all by totally different values to manage their depth

  // Step 1
  const lighting = ambient.mul(0.1)

  // Step 2
  lighting.addAssign(diffuse.mul(0.5))

  // Step 3
  lighting.addAssign(hemi.mul(0.2))

  const finalColor = vec3(0.1).mul(lighting).toVar()

  // Step 4 & 5
  finalColor.addAssign(specular)

  return finalColor
})

The place to go from right here

So we did it! There was lots to study, however the outcome might be spectacular, and from right here there’s a lot that you are able to do with it. Listed here are some issues to strive:

• Add a dice or a rectangle and rotate it.
• Add some noise to the shapes and get gnarly with it.
• Discover different combining capabilities (max).
• Use fract or mod for some fascinating area repetition.

I hope you loved this gentle introduction to raymarching and TSL. If in case you have any questions, let me know on X.

Credit and References