Human-Centric AI & Security Seminar

Abstract:

Equipped with smartphones, tablets, and virtual reality headsets, we can now perform a wide variety of tasks from virtually anywhere. However, this newfound freedom and flexibility come with its own challenges, particularly in terms of personal privacy. One emerging threat is sophisticated machine learning (ML) attacks that break privacy assumptions by extracting previously undetectable patterns carried by our daily actions.

In this talk, I will discuss two of our recent projects on identifying and mitigating real-world privacy threats. First, we identify a general keystroke inference attack in the physical world, where the attacker does not rely on any prior knowledge or labeled data on the target user or their device. Just by analyzing a 10-min video of the target’s typing hands from meters away, the attacker can extract over 90% of their typed content. This is done by applying a self-supervised learning pipeline to the video, which curates labeled data from the video and train DNN models to achieve robust inference results. We then extend this attack into virtual reality. By screen-recording the target’s avatar for 10 min, the attacker can also recover the typed content. This is made possible by the addition of transformer models designed to mitigate digital noise in avatar motion capture. Finally, I will also discuss ongoing projects that explore new forms of privacy attacks in VR systems.

Bio:

Zhuolin Yang is a final year PhD at SAND Lab, University of Chicago. She researches security and privacy issues related to user interfaces. Specifically, she identifies and mitigates new attacks made possible by customized ML solutions. Her work has produced publications at top CS conferences (USENIX Security, CHI) and media coverages by MIT Technology Review and New Scientist.