3DB: A Framework for Debugging Computer Vision Models
Guillaume Leclerc, Hadi Salman, Andrew Ilyas, Sai Vemprala, Logan Engstrom, Vibhav Vineet, Kai Xiao, Pengchuan Zhang, Shibani Santurkar, Greg Yang, Ashish Kapoor, Aleksander Madry
We introduce 3DB: an extendable, unified framework for testing and debugging
vision models using photorealistic simulation. We demonstrate, through a wide
range of use cases, that 3DB allows users to discover vulnerabilities in
computer vision systems and gain insights into how models make decisions. 3DB
captures and generalizes many robustness analyses from prior work, and enables
one to study their interplay. Finally, we find that the insights generated by
the system transfer to the physical world.
We are releasing 3DB as a library (this https URL) alongside a
set of example analyses, guides, and documentation: this https URL .