Understanding the WebGL Rendering Pipeline

Building a high-performance drawing application in the browser requires deep understanding of WebGL and careful optimization at every level. This article explores Notidian’s rendering pipeline, from input events to pixels on screen.

Pipeline Overview

Our rendering pipeline consists of five main stages:

1. Input Processing - Capturing and normalizing user input
2. Stroke Generation - Converting points to geometric data
3. GPU Tessellation - Creating renderable triangles
4. Shading - Applying colors, textures, and effects
5. Compositing - Combining layers for final output

Stage 1: Input Processing

Event Handling

We use a unified input system that handles mouse, touch, and stylus:

class InputHandler {
	private eventQueue: InputEvent[] = [];
	private rafId: number;

	handlePointerEvent(event: PointerEvent) {
		// Normalize coordinates to canvas space
		const point = {
			x: (event.clientX / canvas.width) * 2 - 1,
			y: -((event.clientY / canvas.height) * 2 - 1),
			pressure: event.pressure || 0.5,
			tiltX: event.tiltX || 0,
			tiltY: event.tiltY || 0,
			timestamp: performance.now(),
		};

		this.eventQueue.push(point);

		if (!this.rafId) {
			this.rafId = requestAnimationFrame(() => this.processQueue());
		}
	}
}

Prediction and Smoothing

To reduce perceived latency, we predict future points:

function predictNextPoint(history: Point[]): Point {
	if (history.length < 2) return history[history.length - 1];

	const p1 = history[history.length - 2];
	const p2 = history[history.length - 1];
	const velocity = {
		x: p2.x - p1.x,
		y: p2.y - p1.y,
	};

	return {
		x: p2.x + velocity.x * 0.5,
		y: p2.y + velocity.y * 0.5,
		pressure: p2.pressure,
	};
}

Stage 2: Stroke Generation

Curve Fitting

Raw input points are fitted to smooth curves using Catmull-Rom splines:

vec2 catmullRom(vec2 p0, vec2 p1, vec2 p2, vec2 p3, float t) {
  vec2 v0 = (p2 - p0) * 0.5;
  vec2 v1 = (p3 - p1) * 0.5;
  float t2 = t * t;
  float t3 = t2 * t;

  return (2.0 * p1 - 2.0 * p2 + v0 + v1) * t3 +
         (-3.0 * p1 + 3.0 * p2 - 2.0 * v0 - v1) * t2 +
         v0 * t + p1;
}

Variable Width Calculation

Stroke width varies based on pressure and velocity:

function calculateWidth(pressure: number, velocity: number, settings: BrushSettings): number {
	const pressureWidth = lerp(settings.minWidth, settings.maxWidth, pressure);

	const velocityFactor = 1.0 - clamp(velocity / settings.maxVelocity, 0, 1);
	const velocityWidth = pressureWidth * lerp(0.5, 1.0, velocityFactor);

	return smoothstep(previousWidth, velocityWidth, 0.3);
}

Stage 3: GPU Tessellation

Vertex Generation

Strokes are tessellated into triangle strips on the GPU:

#version 300 es
in vec2 a_position;
in float a_pressure;
in float a_angle;

uniform mat4 u_mvpMatrix;
uniform float u_strokeWidth;

out vec2 v_texCoord;
out float v_pressure;

void main() {
  float width = u_strokeWidth * a_pressure;
  vec2 normal = vec2(cos(a_angle), sin(a_angle));

  // Generate two vertices per input point
  int vertexId = gl_VertexID % 2;
  vec2 offset = normal * width * (vertexId == 0 ? -1.0 : 1.0);

  gl_Position = u_mvpMatrix * vec4(a_position + offset, 0.0, 1.0);
  v_texCoord = vec2(float(gl_VertexID / 2) / u_pointCount, vertexId);
  v_pressure = a_pressure;
}

Index Buffer Optimization

We use indexed rendering to reduce vertex data:

function generateIndexBuffer(pointCount: number): Uint16Array {
	const indices = new Uint16Array((pointCount - 1) * 6);
	let idx = 0;

	for (let i = 0; i < pointCount - 1; i++) {
		const base = i * 2;
		// Triangle 1
		indices[idx++] = base;
		indices[idx++] = base + 1;
		indices[idx++] = base + 2;
		// Triangle 2
		indices[idx++] = base + 1;
		indices[idx++] = base + 3;
		indices[idx++] = base + 2;
	}

	return indices;
}

Stage 4: Shading

Fragment Shader

Our fragment shader handles texturing and anti-aliasing:

#version 300 es
precision highp float;

in vec2 v_texCoord;
in float v_pressure;

uniform sampler2D u_brushTexture;
uniform vec4 u_color;
uniform float u_opacity;

out vec4 fragColor;

void main() {
  // Sample brush texture
  vec4 brushColor = texture(u_brushTexture, v_texCoord);

  // Apply pressure-based opacity
  float opacity = u_opacity * v_pressure * brushColor.a;

  // Anti-aliasing at edges
  float edge = min(v_texCoord.y, 1.0 - v_texCoord.y);
  float aa = smoothstep(0.0, 0.02, edge);

  fragColor = vec4(u_color.rgb * brushColor.rgb, opacity * aa);
}

Texture Management

Brush textures are managed efficiently:

class TextureAtlas {
	private atlas: WebGLTexture;
	private regions: Map<string, TextureRegion> = new Map();

	addTexture(name: string, image: ImageData): TextureRegion {
		const region = this.findEmptyRegion(image.width, image.height);

		gl.bindTexture(gl.TEXTURE_2D, this.atlas);
		gl.texSubImage2D(gl.TEXTURE_2D, 0, region.x, region.y, image.width, image.height, gl.RGBA, gl.UNSIGNED_BYTE, image);

		this.regions.set(name, region);
		return region;
	}
}

Stage 5: Compositing

Layer Blending

Layers are composited using framebuffers:

class LayerCompositor {
	private framebuffers: Map<string, WebGLFramebuffer> = new Map();

	composeLayers(layers: Layer[]): void {
		// Clear output framebuffer
		gl.bindFramebuffer(gl.FRAMEBUFFER, this.outputFBO);
		gl.clear(gl.COLOR_BUFFER_BIT);

		for (const layer of layers) {
			if (!layer.visible) continue;

			// Bind layer texture
			gl.activeTexture(gl.TEXTURE0);
			gl.bindTexture(gl.TEXTURE_2D, layer.texture);

			// Set blend mode
			this.setBlendMode(layer.blendMode);

			// Draw layer quad
			gl.useProgram(this.compositeProgram);
			gl.uniform1f(this.opacityLoc, layer.opacity);
			gl.drawArrays(gl.TRIANGLE_STRIP, 0, 4);
		}
	}

	setBlendMode(mode: BlendMode): void {
		switch (mode) {
			case 'multiply':
				gl.blendFunc(gl.DST_COLOR, gl.ONE_MINUS_SRC_ALPHA);
				break;
			case 'screen':
				gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_COLOR);
				break;
			case 'overlay':
				// Custom shader for complex blend modes
				gl.useProgram(this.overlayProgram);
				break;
			default:
				gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);
		}
	}
}

Optimization Techniques

Batch Rendering

Similar strokes are batched to minimize draw calls:

class BatchRenderer {
	private vertexBuffer: Float32Array;
	private bufferOffset: number = 0;

	addStroke(stroke: Stroke): void {
		const vertices = stroke.generateVertices();
		this.vertexBuffer.set(vertices, this.bufferOffset);
		this.bufferOffset += vertices.length;

		if (this.bufferOffset > BATCH_THRESHOLD) {
			this.flush();
		}
	}

	flush(): void {
		if (this.bufferOffset === 0) return;

		gl.bindBuffer(gl.ARRAY_BUFFER, this.vbo);
		gl.bufferSubData(gl.ARRAY_BUFFER, 0, this.vertexBuffer);
		gl.drawArrays(gl.TRIANGLES, 0, this.bufferOffset / VERTEX_SIZE);

		this.bufferOffset = 0;
	}
}

Occlusion Culling

Off-screen content is skipped:

function isVisible(bounds: Rectangle, viewport: Rectangle): boolean {
	return !(
		bounds.right < viewport.left ||
		bounds.left > viewport.right ||
		bounds.bottom < viewport.top ||
		bounds.top > viewport.bottom
	);
}

Progressive Rendering

Complex scenes render in passes:

class ProgressiveRenderer {
	renderScene(priority: RenderPriority): void {
		switch (priority) {
			case RenderPriority.Interactive:
				// Render only active stroke
				this.renderActiveStroke();
				break;
			case RenderPriority.Quality:
				// Render everything at full quality
				this.renderAllLayers();
				break;
			case RenderPriority.Background:
				// Render in chunks during idle time
				requestIdleCallback(() => this.renderChunk());
				break;
		}
	}
}

Performance Metrics

Our pipeline achieves:

• < 2ms input to screen latency
• 60fps with 50,000+ vertices
• < 100MB GPU memory for typical artwork
• < 5% CPU usage during drawing

Future Enhancements

WebGPU Migration

WebGPU will unlock:

• True compute shaders
• Better memory management
• Multi-queue rendering
• Reduced CPU overhead

Advanced Techniques

• Mesh shaders for adaptive tessellation
• Variable rate shading for performance
• Temporal upsampling for 4K displays
• Ray tracing for realistic brushes

Conclusion

Building a WebGL rendering pipeline for a drawing application requires careful attention to every stage of the process. By leveraging GPU acceleration, minimizing CPU-GPU communication, and implementing smart optimization strategies, we’ve created a pipeline that delivers native-like performance in the browser.

The key insights:

1. Move computation to the GPU whenever possible
2. Batch operations to reduce API calls
3. Use level-of-detail for scalability
4. Profile and optimize the critical path

This foundation enables Notidian to handle complex artwork while maintaining responsive, fluid interaction.