ReVamp — Virtual Renovation
The virtual world is getting bigger. Everything from rocket simulation to grocery shopping is virtual now, and now a new thing is getting added to this list. Moving your furniture can be a tough job, and trying a new look on a virtual platform can give a new dimension and make it a little easier to do.
There are a few layers when you wish to give your room a new look. Usually, you want to add new furniture or move the old one to a new location, but it gets tricky when you want a perfect setting for your furniture or wants to buy a new one that will perfectly fit in your room’s theme. But we can add a virtual element in this and make visualization easier.
What ReVamp brings to the table?
In ReVamp, you can make a virtual element of your furniture using your camera. You can click on the object, and it will create a virtual overlay that you can move using your camera. It also allows you to use external images so you can use pictures of furniture you wish to buy to make the virtual overlay.
Technical view of ReVamp
I was working on this project and finished a software version of ReVamp
using Python.
Let’s see how ReVamp works.
I used OpenCV to get the camera feed of the device or the IP cam for better quality. Using this, I get a continuous stream of frames that I use to extract the object.
Extracting the object from a picture is tricky so, I tried different approaches to do that.
Flood fill with the color difference
I tried the flood fill algorithm with different color difference formula.
I used Euclidean distance and Red Mean. Euclidean distance was better for computer-generated images, and Red Mean was good for the pictures taken from the camera.
All over the result were not as good as the noise in the picture is left out.
Adding edge Detection
The next big step in the results was adding edge-detection. Now to extract an object, I run a flood fill algorithm with low Delta-e on edge and, then I draw the contour on the resulted CV-MAT. Finally, we get a mask of the object, and applying that on the original image gives us the required virtual overlay of the object.
There were a few flaws in this approach.
Adding convolution filters and contour smoothing
Adding convolution filters and contour smoothing removes the flaw in the previous approach. Before using the above approach, I preprocess the image with a different convolution filter. I tried many filters and, smooth + edge enhance works the best as it removes the noise and sharpens the edge. After this, I also added contour smoothing while creating the mask.
The results were fine on different images. Still, there were outliers images where results were not up to the mark. While I was trying different approaches, I saw an idea similar to mine getting popular. So I started looking at the technologies used in that. There I found BASNet, which provides very good results on object detection and segmentation.
Using BASNet
BASNet is a deep Convolutional neural network used for salient object detection and accurately predict the fine structures with clear boundaries.
Using BASNet gave the best results for creating the overlay.
Deep Convolutional Neural Networks have been adopted for salient object detection and achieved the state-of-the-art performance. Most of the previous works however focus on region accuracy but not on the boundary quality.
And finally, the tricky part of the ReVamp was ready. Now I have to implement this in ReVamp and create a usable tool.
Interface/UI
Now to create an interface, I used the Pygame library of python, I didn’t use other interfacing libraries like Tkinter as I want a little flexible UI with a different feature. So, I create my UI library for Pygame that provided me the flexibility I needed. Further, I used multi treading to make the flow of ReVamp seamless.
I used different techniques to provide the basic functionality of ReVamp like using external images to create the overlay, save/load overlay, basic camera function, and some basic function on the overlay.
You can check more about ReVamp
