Sin Liang LeeSL Lee
Sin Liang LeeClose

Emailslee3846@gatech.edu

GitHubLinkedIn

← Projects,

FRIDAY

A spatial analysis engine and agent harness that furnishes rooms with AI.

Role
3D room editor, Gaussian-splat rooms, video-to-room mesh pipeline
Team
4 people
Context
HackMIT 2026 — OpenAI 5th Teammate Challenge, 3rd place
Stack
React, TypeScript, PlayCanvas, Gaussian splats, Django REST, OpenAI Agents SDK, Elasticsearch, Open3D
Links
GitHubLive demo
The FRIDAY editor on a skyscraper floor, by a reception desk and floor-to-ceiling windows: an armchair in the room, and the AI panel on the right listing the armchairs found for “an armchair”.
In the skyscraper room: an armchair chosen from the AI panel’s results, previewed where it fits, placed, then turned 90°. Building: “[FREE] London Skyscraper” by 99.Miles, on Sketchfab. CC BY 4.0. Modified: its furniture removed.

What it is

You describe the room you want, by text or by voice. An agent searches a furniture catalog for pieces that have 3D models, checks clearances and placements against the room’s geometry and lays out designs you can walk through and compare.

In the demo, the bedroom workflow produces three designs of five pieces each, within a $1,200 budget.

What I built

  • The 3D room editor: selection, dragging, snapping, centimeter position fields, quarter-turn rotation, removal and validated undo and redo, on a shared centimeter scene contract (GLB models stay in meters and are converted once).
  • The move to first-person Gaussian-splat rooms in PlayCanvas, and the integration of a Gaussian-splat capture of an apartment as a test room, with automatic room downloads.
  • A standalone video-to-room mesh pipeline: frame extraction, optional VGGT inference, scale calibration from a measured reference and Open3D TSDF fusion. It exports a textured GLB, a simplified collision GLB and a dense point cloud. Its accuracy on real phone video is not yet verified.
  • The “Hey Friday” voice input and spoken replies.
    FRIDAY’s request and reply above the editor toolbar. You: “Hey Friday, I have a $1,200 budget. Can you help me design my room? I want a warm couch. I want the bed beside the couch. Could you also add decorations like plants around it? I’ll leave it up to you to choose the decorations. Go ahead.” Reply: “Your first design is placed. Three designs are ready to compare; choose one, then lock it in.” Toolbar: Move, Rotate, Clear room, undo, redo, and Hey Friday, Speaking.
    Detail The request to Friday and its reply, over the editor’s toolbar.

Technical notes

  • The agent runs on the OpenAI Agents SDK, with tools for scene inspection, product discovery, spatial analysis and placement validation.
  • Natural-language search requests are compiled into a structured constraint language. Elasticsearch runs the text queries and the deterministic filters.
  • Rooms combine Gaussian splats or meshes with editable GLB furniture.
  • The apartment room ran at 53.5 FPS, p95 frame time 26.2 ms, over 30 seconds at 1280×720. Its scale is assumed, not calibrated.
The FRIDAY editor inside a furnished Parisian apartment, with a panel comparing three bedroom designs priced at $851, $885 and $905.
Fig. 1 Three bedroom designs in the apartment test room, under the $1,200 demo budget. Scene: “HAUSSMANN APARTMENT” by Stéphane Agullo, on SuperSplat. CC BY 4.0.