History Hologram

Making history physical

History Hologram started at a 48-hour hackathon in Sydney, Hack Club Horizons Crux. The theme was the crux of storytelling, and our team had a slightly awkward mix of hardware, Godot, TypeScript, Go, C and Python experience.

We also had no hardware tools. Naturally, we built the only hardware project at the event.

The idea was to turn a historical story into a small Pepper’s Ghost display. Instead of only reading about ANZAC Cove, you could watch a narrated 3D scene unfold on an iPad and appear inside a clear reflector above it.

Finding the reflector material took far longer than expected. We spent hours comparing acrylic, polypropylene and clear film before leaving the venue to buy something that would bend and still reflect enough light. The organisers did not want to leave, so our hardware trip eventually took most of the event with us. It was not efficient, but it gave us a physical display to build for.

A story engine instead of one animation

We did not want to hardcode one ANZAC scene. Stories are described in YAML as actors and timed moments:

actors:
  - id: hero
    src: ../models/anzac-hero.glb
    position: [0, 0, 0]

moments:
  - at: 0
    label: ARRIVAL
  - at: 12
    label: FIRST LIGHT

The same timeline controls camera movement, models, narration, subtitles, ambience and effects. The React app does not know what ANZAC Cove is. It only knows how to load and play a scene.

That was the most important technical decision because another story can be made by adding assets and editing YAML rather than writing another frontend.

Building the display without a workshop

Pepper’s Ghost is a reflection trick. A clear angled surface over a bright screen reflects an image so it appears to float in space. A four-sided reflector needs the same scene rendered in four synchronized directions around the centre of the iPad.

YAML story
    ↓
timeline
    ↓
React + Three.js
    ↓
four mirrored views
    ↓
iPad + clear reflector

We kept a normal 3D mode for development because balancing a plastic pyramid on an iPad after every change would have been miserable.

Forty-eight hours later

The presentation was not flawless. The venue was too bright, the hologram was hard to see, and the recorded narration did not stay synchronized. The physical effect looked better during testing than it did on stage.

Then we won the GitHub award.

I was genuinely surprised because everyone else could open a website while our demo depended on a web app, a 3D engine, audio, an iPad, plastic, room lighting and hope.

By the end of the weekend we had a reusable 3D story player, an ANZAC Cove scene, synchronized timelines, four-way rendering and a real physical display. The presentation exposed its rough edges, but the underlying system worked and could tell more than one story.

Four outputs, one world

04 / ONE WORLD, TWO OUTPUTS

SceneReflector

The recorded scene uses time-derived actor and camera poses. The same YAML story feeds the normal view and the reflector output.

I render one canvas and mirror its captureStream() to the other three sides. That is a very different cost from rendering four independent 3D worlds. The output changes; the scene stays shared.

That wasn’t the first version. My first reflector view ran four separate WebGL renderers, one per face, each with its own copy of a heavy car model. My commit message for fixing it calls four renderers ridiculous, and it was. Rendering a single 30 fps canvas and mirroring its stream across the other three faces fixed it. I also trimmed the texture payload on that car model, which I’d been treating as free until I looked at what it weighed.

One clock instead of eighteen

The narration, effects and ambience started life as separate audio clips, each playing on its own, and keeping eighteen of them in sync with a timeline you can pause and drag was a losing game. Seek to the middle and some of them landed in the wrong place, and the narration drifting away from the picture is the kind of thing we saw on stage.

The fix was to stop synchronising them. I mastered narration and effects into one soundtrack per story, wrote the subtitle cues out from the speech boundaries, and made the timeline the only clock. Pausing and seeking now move one thing instead of eighteen, and the audio can’t disagree with the picture because there’s only one of it. The old pile of clips would have been fine for one demo and impossible for a second story.

A few smaller cleanups came with it. Scene files are validated with ArkType, so a typo in a YAML file fails loudly at load time instead of as a model that’s silently missing. Loading and caching use ky, swr and p-memoize rather than the loader I’d hand-rolled, because maintaining that myself was daft. And I wrote a small tool to strip a node out of a .glb and bake the result into the asset, after I got tired of one car model bringing its own floor along.

Time, not accumulated frame state

The timeline computes actor and camera poses from scene time. Actor cues replay from the start so additive and absolute transforms compose; transitions interpolate with smoothstep. Camera cues carry position, target and field of view.

This makes seeking more than dragging a progress bar: the renderer must be able to reconstruct what the scene should look like at a chosen time, without needing every previous frame to have played.

SceneWorld selects the chapter and shot, builds an actor-parent graph, and renders lights, actors and postprocessing. During playback, the camera follows the timeline; paused interactive mode yields to OrbitControls.