Visually Cohesive Recommendations

Home decor is never really done - it becomes a reflection of the occupant like clothing. While large investment pieces, like a couch or bed, remain static, customers will often change out smaller items like throw pillows and lamps. Using deep learnt feature vectors, this project provides recommendations personalized to the occupants existing space and changing aesthetic. Aspirational decor possible is now possible in real time without a professional designer.

MARKET - HOME DECOR

Home Decor Market is projected to be at $664 billion by 2020
82% of Millennials think home decor is important and 77% of Millennials have redecorated their home in the past two years

(Source: Allied market research, Furniture today, 2018)

CUSTOMER PROBLEM

Home decorating is hard; it is hard to find pieces that match existing furniture to achieve an aspirational look
Volume of available products; hard to narrow down to relevant results

SOLUTION - real time space refresher

Customers can browse through visually cohesive product groups personalized to their esthetic. By uploading pictures of their big ticket items like couch or bed, customers receive objects that work visually within their living space. An interactive AR experience will help them visualize the suggested design.

Customers generate visually cohesive product groups in their personal aesthetic. Filtering by price and brand further enables them to make confident choices.

visually cohesive objects inspired by the scene

interactive AR to visualize the final design

TECHNICAL SUMMARY

Object detection: Trained a neural network based on YOLO for objects found in home categories
Dataset curation: Styled images from professional interior decorators
Deep-learnt feature extraction to represent each image from a multi-label neural network
K-nearest neighbor algorithm with Euclidean distance
Weighted Likelihood to tailor designer recommendations to customer’s taste
Blending 3D assets into a single 3D asset for interactive AR

Step 1. The object detector detects and recognizes the couch in the customer image.

Step 2. Fetch professionally styled images with a similar couch.

Step 3. Parse the scene for related objects. In this example we detect a coffee table, lamp, chair, ottoman and throw pillows. Extract features of detected objects.

For each product type like coffee table,

Step 4a. Use the detected coffee tables in the styled images as positives to creative positive-negative product pairs. Use coffee tables in other styled images as negatives.

Step 4b. Add additional product pairs if customer chooses an aesthetic type like mid-century. Use mid century coffee tables as positives and other aesthetic coffee tables as negatives in the pair.

Step 4c. Add additional product pairs if customer actively likes or dislikes certain coffee tables in the real time interaction.

Step 4d. Use thompson sampling to optimize to the correct correct table.

Here are some results for the above customer query from a small dataset of 10K products. The recommendations take inspiration from the decor objects in the styled image and a mid century aesthetic.