Abstract

It is not widely recognized that people with visual impairments (PVI) want to take their own photographs. This talk presents a practical pathway that enables both taking the shot and understanding the photo by touch. We first introduce VisPhoto, which captures a 360° image at the shutter press and then produces the final photograph as the user desires, effectively removing the need for precise framing at capture time. The workflow combines object detection to surface candidate subjects and speech recognition to capture user intent before generating the final image. We then show how a single photograph can be transformed into a 2.5D tactile relief, enabling users to experience what the image depicts. Monocular depth estimation, a computer vision technique, provides global 3D structure, while luminance-to-height mapping encodes planar details, such as textures and text. The resulting height map is fabricated on a 3D printer as a tactile plate. As an extension, depth-guided detection of foreground meshes with inpainting helps rebuild continuous, easy-to-explore relief surfaces. Together, these components make it feasible for PVI to take photographs independently and understand them through tactile exploration.

Bio

Masakazu Iwamura (Member, IEEE) received the B.E., M.E., and Ph.D. degrees in engineering from Tohoku University, Japan, in 1998, 2000, and 2003, respectively. He is currently a Professor with the Department of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University. His research interests include text and object recognition and visually impaired assistance. He has received numerous awards, including the IAPR/ICDAR Young Investigator Award, in 2011, the Best Paper Awards of IEICE, in 2008 and 2021, the IAPR/ICDAR Best Paper Award, in 2007, the IAPR Nakano (Best Paper) Award, in 2010, the ICFHR Best Paper Award, in 2010, the MVA Best Paper Award, in 2017, and the SIGACCESS Best Paper Award, in 2023. He served as the Vice-Chair of the IAPR Technical Committee 11 (Reading Systems), from 2016 to 2018, an Associate Editor of International Journal of Document Analysis and Recognition, from 2013 to 2023, and an Associate Editor and an Associate Editor-in-Chief of IEICE Transactions on Information and Systems, from 2017 to 2021 and from 2021 to 2023, respectively.