Speaker
Description
Furniture replacement in augmented reality (AR) requires both isolating the target object within a photorealistic 3D reconstruction and retrieving visually and stylistically coherent replacements from a large catalogue of 3D assets. This paper presents a context-aware furniture recommendation system built on a curated catalogue derived from the 3D-FUTURE dataset. From the 16,563 models available, we construct a balanced catalogue of 2,400 furniture objects spanning six residential super-categories (Chair, Table, Sofa, Bed, Cabinet/Shelf/Desk, and Lighting), with 400 models per category. Each object is rendered from three canonical viewpoints at 512×512 resolution, and more than 40 heterogeneous style annotations are consolidated into nine canonical style labels to enable consistent compatibility reasoning. A task-specific evaluation of seven open-source vision-language models (VLMs) on closed-set interior style classification shows that all functional models perform within sampling uncertainty of one another, and Qwen2.5-VL-7B is selected for its lowest latency (1.53 s/image). The selected VLM generates natural-language captions for 2,347 catalogue objects, describing shape, material, style, and plausible interior context. Captions and thumbnails are indexed with CLIP ViT-L/14 into a dual-channel (text and image) embedding space. At inference, a user click in the reconstructed scene triggers a VLM query that identifies the room style, object category, and a replacement description, which is matched against the index with a tunable visual-channel weight and re-ranked under a hard category constraint and a soft style-compatibility penalty. The offline pipeline processes the full catalogue in under 15 minutes; the index achieves self-consistency Recall@1 of 1.0 (text) and 0.997 (image), and text-dominant retrieval reaches Category Precision@5 of 0.97. The results show that a catalogue-scale, style-aware furniture recommender for AR can be assembled entirely from publicly available 3D assets and open-source vision-language models.