Liteforms

Liteforms began as an experiment: what happens if you put a large language model behind a holographic character and let someone talk to it out loud? It was a compelling demo. Demos are not products, and the year-long distance between those two things is the work I’m proudest of at Looking Glass Factory.

I was the main engineer on that productization — taking it from a prototype into a shipped application on Mac and Windows, built in Unity/C# against OpenAI’s APIs and Azure Speech. It launched publicly in 2023 and was covered by TechCrunch and PetaPixel, and I supported customer-facing demos for enterprise clients including Accenture.

What productization actually meant

Authoring, not just conversing. The demo had a character. The product needed anyone to be able to make one — personality, voice, appearance, animations, spatial placement. That meant designing and implementing essentially the entire application surface, which is what the screenshots below are.

Latency you can live with. A conversation has a rhythm, and speech-to-text, model inference, and text-to-speech chained end to end will break it if you treat them as sequential blocking calls. Making a hologram feel present rather than laggy is a pipelining and streaming problem long before it is a model-quality problem.

Failure without breaking the illusion. Network calls fail and models produce something unusable. A character that freezes or emits an error string stops being a character. Handling that gracefully — staying in character while recovering — is a design problem that lands squarely on the engineer.

The thing running on the display. Rendering a live, animated, lip-synced avatar on a light field display means paying the multi-view rendering cost continuously, at conversational frame rates, on a customer’s machine.

Below are screenshots of the application UI.