Caio Costa: advancing computer vision into new horizons

July 25, 2022

Caio Costa, 22, Class of 2022

Background:

Costa earned a bachelor’s degree from the Department of Computer Science. He also earned an undergraduate certificate from the Center for Statistics and Machine Learning (CSML).

Undergraduate Work:

Costa tackled computer vision for his CSML project, specifically how computers may perceive objects that are partially occluded  from view .

“Scene reconstruction is an important computer vision task in many 3D contexts, including autonomous vehicles and robotics. Occlusions can negatively impact the ability to determine the geometry of objects in a scene,” he said.

He set out to put together a machine learning process that uses light bounced off a surface to determine an object’s shape, a technique known as non-line-of-sight (NLOS) imaging, a burgeoning area of study in the computer vision field.

“It’s kind of a light echo,” he said. “Non-line-of-sight imaging is basically being able to use optimization and machine learning techniques to create a representation of an object that the camera can't directly see by imaging what is essentially the echo of light off of a wall.  We call this wall the relay wall,” he said.

In a common NLOS set-up, which Costa used for his project, there is a relay wall that the camera points at, and then there's another wall that's dividing the camera from the object that it's trying to image, Costa said. The camera can't directly see the object, but it can see the relay wall. Researchers would then emit pulsed laser lights from the camera. These light pulses bounce off the relay wall and then bounce off the hidden object. The bounced light carries information, a light echo. The light echo reaches sensors in the camera with light further away taking longer to reach the sensor. By analyzing the different times when the light echoes arrive back at the sensor, researchers can recreate the geometries of the hidden object, Costa said.

For his project, Costa developed and trained a non-line-of-sight neural renderer that processes transient images, the light echoes collected by sensors. He then used an evaluation metric to measure the performance of his renderer, which proved to be effective. The image he tried to reconstruct for the project was a 3D model of a motorcycle.

In the short term, the process has not been perfected yet for practical use, Costa said. The field is still very much in the beginning stage. But in the long term, after many successive improvements, Costa envisions this type of technology finding a home in an autonomous vehicle. They would use light bouncing off the roadway to imagine any objects that are hidden, such as a pedestrian behind another parked car.

Costa found the CSML curriculum helpful in not just assisting him in the development and completion of his project, but it had also given him tools to look at different situations or old problems with a new lens.

“One of the classes I took this past semester is Roman archaeology and scientific applications in the field, such as chemistry and using spectrometers. I proposed in my final project using machine learning, which is a new approach in the field of archaeology,” he said.

After his graduation, Costa assumed the position of software engineer at Stripe, a financial services company mostly known for payments processing. Before this job, Costa had worked at Meta as a software engineering intern in the previous three summers.

Extracurricular Activities:

Costa was a co-president and officer at the Princeton robotics club and served as a teaching assistant in the Department of Computer Science.

For Fun:

Costa plays board games with friends. One of the board games he enjoys playing is Terraforming Mars, a complicated and large board game that fits his interest in space exploration.