19–21 Nov 2026
"Alexandru Ioan Cuza" University of Iasi and "Gheorghe Asachi" Technical University of Iași
Europe/Bucharest timezone

Catalogue-Scale, Style-Aware Furniture Recommendation for AR Replacement Using Vision-Language Models and Dual-Channel CLIP Retrieval

20 Nov 2026, 09:45
15m
Room 2

Room 2

Paper presentation Doctoral Symposium Doctoral Symposium

Speaker

Mr Dan Toderici (National University of Science and Technology POLITEHNICA Bucharest)

Description

Furniture replacement in augmented reality (AR) requires both isolating the target object within a photorealistic 3D reconstruction and retrieving visually and stylistically coherent replacements from a large catalogue of 3D assets. This paper presents a context-aware furniture recommendation system built on a curated catalogue derived from the 3D-FUTURE dataset. From the 16,563 models available, we construct a balanced catalogue of 2,400 furniture objects spanning six residential super-categories (Chair, Table, Sofa, Bed, Cabinet/Shelf/Desk, and Lighting), with 400 models per category. Each object is rendered from three canonical viewpoints at 512×512 resolution, and more than 40 heterogeneous style annotations are consolidated into nine canonical style labels to enable consistent compatibility reasoning. A task-specific evaluation of seven open-source vision-language models (VLMs) on closed-set interior style classification shows that all functional models perform within sampling uncertainty of one another, and Qwen2.5-VL-7B is selected for its lowest latency (1.53 s/image). The selected VLM generates natural-language captions for 2,347 catalogue objects, describing shape, material, style, and plausible interior context. Captions and thumbnails are indexed with CLIP ViT-L/14 into a dual-channel (text and image) embedding space. At inference, a user click in the reconstructed scene triggers a VLM query that identifies the room style, object category, and a replacement description, which is matched against the index with a tunable visual-channel weight and re-ranked under a hard category constraint and a soft style-compatibility penalty. The offline pipeline processes the full catalogue in under 15 minutes; the index achieves self-consistency Recall@1 of 1.0 (text) and 0.997 (image), and text-dominant retrieval reaches Category Precision@5 of 0.97. The results show that a catalogue-scale, style-aware furniture recommender for AR can be assembled entirely from publicly available 3D assets and open-source vision-language models.

Author

Mr Dan Toderici (National University of Science and Technology POLITEHNICA Bucharest)

Co-authors

Mr Tiberiu-Gabriel Rodanciuc (National University of Science and Technology POLITEHNICA Bucharest) Mr George-Alexandru Micu (National University of Science and Technology POLITEHNICA Bucharest) Prof. Larisa Dunai (Department of Graphic Engineering, Universitat Politécnica de Valéncia, Spain)

Presentation materials

There are no materials yet.