In 2018, UW Security and Privacy Lab members Ivan Evtimov and Earlence Fernandes (now faculty at Wisconsin), along with UW Prof. Yoshi Kohno and researchers from Samsung Research North America, Stanford University, Stony Brook University, University of California at Berkeley, and University of Michigan , wrote a now widely sited paper on fooling computer vision classifiers and, in doing so, demonstrated the ability to fool a machine learning system into misidentifying a stop sign as, say, a speed limit sign.
The Science Museum in London asked to include the UW Stop Sign in their exhibit titled “Driverless: Who is in Control?”. If you’re in London, please stop by and check it out!
Congratulations to all UW Allen School Security and Privacy Research Lab PhD Graduates — Dr. Camille Cobb, Dr. Kiron Lebeck, Dr. Peter Ney, and Dr. Alex Takakuwa! Congratulations also to graduating Security Lab undergraduate Mitali Palekar, who also won one of the Allen School’s few Outstanding Senior Awards. What an amazing job everyone!
Congratulations to Dr. Alex Takakuwa for successfully defending his PhD dissertation today! Alex’s PhD work focuses on improving various key open challenges in two-factor authentication, and is a result of significant collaboration with Dr. Alexei Czeskis from Google. Alex will continue at UW as a postdoc, incubating a creative new technology idea. Congratulations Dr. Takakuwa!
What makes Grover so effective at spotting fake news is the fact that it was trained to generate fake news itself.
When we hear the term “fake news,” more often than not it refers to false narratives written by people to distort the truth and poison the public discourse. But new developments in natural language generation have raised the prospect of a new potential threat: neural fake news. Generated by artificial intelligence and capable of adopting the particular language and tone of popular publications, this brand of fake news could pose an even greater problem for society due to its ability to emulate legitimate news sources at a massive scale. To fight the emerging threat of fake news authored by AI, a team of researchers at the Allen School and Allen Institute for Artificial Intelligence (AI2) developed Grover, a new model for detecting neural fake news more reliably than existing technologies can.
Until now, the best discriminators could correctly distinguish between real, human-generated news content and AI-generated fake news 73% of the time; using Grover, the rate of accuracy rises to 92%. What makes Grover so effective at spotting fake content is that it learned to be very good at producing that content itself. Given a sample headline, Grover can generate an entire news article written in the style of a legitimate news outlet. In an experiment, the researchers found that the system can also generate propaganda stories in such a way that readers rated them more trustworthy than the original, human-generated versions.
“Our work on Grover demonstrates that the best models for detecting disinformation are the best models at generating it,” explained Yejin Choi, a professor in the Allen School’s Natural Language Processing group and a researcher at AI2. “The fact that participants in our study found Grover’s fake news stories to be more trustworthy than the ones written by their fellow humans illustrates how far natural language generation has evolved — and why we need to try and get ahead of this threat.”
Choi and her collaborators — Allen School Ph.D. students Rowan Zellers, Ari Holtzman, and Hannah Rashkin; postdoctoral researcher Yonatan Bisk; professor and AI2 researcher Ali Farhadi; and professor Franziska Roesner — describe their results in detail in a paper recently published on the preprint site arXiv.org. Although they show that Grover is capable of emulating the style of a particular outlet and even writer — for example, one of the Grover-generated fake news pieces included in the paper is modeled on the writing of columnist Paul Krugman of The New York Times — the researchers point out that even the best examples of neural fake news are still based on learned style and tone, rather than a true understanding of language and the world. So, that Krugman piece and others like it will contain evidence of the true source of the content.
“Despite how fluid the writing may appear, articles written by Grover and other neural language generators contain unique artifacts or quirks of language that give away their machine origin,” explained Zellers, lead author of the paper. “It’s akin to a signature or watermark left behind by neural text generators. Grover knows to look for these artifacts, which is what makes it so effective at picking out the stories that were created by AI.”
The research team, top row from left: Rowan Zellers, Ari Holtzman, Hannah Rashkin, and Yonatan Bisk. Bottom row from left: Ali Farhadi, Franziska Roesner, and Yejin Choi.
Although Grover will naturally recognize its own quirks, which explains the high success rate in the team’s study, the ability to detect evidence of AI-generated fake news is not limited to its own content. Grover is better at detecting fake news written by both human and machine than any system that came before it, in large part because it is more advanced than any neural language model that came before. The researchers believe that their work on Grover is only the first step in developing effective defenses against the machine-learning equivalent of a supermarket tabloid. They plan to release two of their models, Grover-Base and Grover-Large, to the public, and to make the Grover-Mega model and accompanying dataset available to researchers upon request. By sharing the results of this work, the team aims to encourage further discussion and technical innovation around how to counteract neural fake news.
According to Roesner, who co-directs the Allen School’s Security and Privacy Research Laboratory, the team’s approach is a common one in the computer security field: try to determine what adversaries might do and the capabilities they may have, and then develop and test effective defenses. “With recent advances in AI, we should assume that adversaries will develop and use these new capabilities — if they aren’t already,” she explained. “Neural fake news will only get easier and cheaper and better regardless of whether we study it, so Grover is an important step forward in enabling the broader research community to fully understand the threat and to defend the integrity of our public discourse.”
Roesner, Choi and their colleagues believe that models like Grover should be put to practical use in the fight against fake news. Just as sites like YouTube rely on deep neural networks to scan videos and flag those containing illicit content, a platform could employ an ensemble of deep generative models like Grover to analyze text and flag articles that appear to be AI-generated disinformation.
“People want to be able to trust their own eyes when it comes to determining who and what to believe, but it is getting more and more difficult to separate real from fake when it comes to the content we consume online,” Choi said. “As AI becomes more sophisticated, a tool like Grover could be the best defense we have against a proliferation of AI-generated fake news.”
Congratulations to UW Security and Privacy Lab PhD student Ivan Evtimov for successfully completing his Quals written and oral exam today! The Quals exam is a major milestone in the PhD process. Congratulations Ivan!
Congratulations to Dr. Kiron Lebeck for successfully defending his PhD dissertation today! Kiron’s dissertation describes forward-looking and foundational work on computer security for augmented reality systems. Kiron will be joining Niantic as a Software Engineer after graduation. Congratulations Dr. Lebeck!
Congratulations to Dr. Camille Cobb for successfully defending her PhD dissertation today! Camille’s PhD work focuses on privacy and security issues that can arise between user-to-user interactions, e.g., in online dating systems. Camille will be joining CMU as a postdoc following graduation, where she will be working with Prof. Lujo Bauer. Congratulations Dr. Cobb!
Members of the team that examined the privacy and security risks of implantable medical devices in 2008. UW News Office
In March 2008, Allen School researchers and their collaborators at the University of Massachusetts Amherst and Harvard Medical School revealed the results of a study examining the privacy and security risks of a new generation of implantable medical devices. Equipped with embedded computers and wireless technology, new models of implantable cardiac defibrillators, pacemakers, and other devices were designed to make it easier for physicians to automatically monitor and treat patients’ chronic health conditions while reducing the need for more invasive — and more costly — interventions. But as the researchers discovered, the same capabilities intended to improve patient care might also ease the way for adversarial actions that could compromise patient privacy and safety, including the disclosure of sensitive personal information, denial of service, and unauthorized reprogramming of the device itself.
A paper detailing their findings, which earned the Best Paper Award at the IEEE’s 2008 Symposium on Security and Privacy, sent shock waves through the medical community and opened up an entirely new line of computer security research. Now, just over 10 years later, the team has been recognized for its groundbreaking contribution by the IEEE Computer Society Technical Committee on Security and Privacy with a 2019 Test of Time Award.
“We hope our research is a wake-up call for the industry,” professor Tadayoshi Kohno, co-director of the Allen School’s Security and Privacy Research Laboratory, told UW News when the paper was initially published. “In the 1970s, the Bionic Woman was a dream, but modern technology is making it a reality. People will have sophisticated computers with wireless capabilities in their bodies. Our goal is to make sure those devices are secure, private, safe and effective.”
Chest x-ray showing an implanted cardioverter defibrillator (ICD).
To that end, Kohno and Allen School graduate student Daniel Halperin (Ph.D., ‘12), worked with professor Kevin Fu, then a faculty member at University of Massachusetts Amherst, and Fu’s students Thomas Heydt-Benjamin, Shane Clark, Benessa Defend, Will Morgan, and Ben Ransford — who would go on to complete a postdoc at the Allen School — in an attempt to expose potential vulnerabilities and offer solutions. The computer scientists teamed up with cardiologist Dr. William Maisel, then-director of the Medical Device Safety Institute at Beth Israel Deaconess Medical Center and a professor at Harvard Medical School. As far as the team was aware, the collaboration represented the first time that anyone had examined implantable medical device technology through the lens of computer security. Their test case was a commercially available implantable cardioverter defibrillator (ICD) that incorporated a programmable pacemaker capable of short-range wireless communication.
The researchers first partially reverse-engineered the device’s wireless communications protocol with the aid of an oscilloscope and a commodity software radio. They then commenced a series of computer security experiments targeting information stored and transmitted by the device as well as the device itself. With the aid of their software radio, the team found that they were able to compromise the security and privacy of the ICD in a variety of ways. As their goal was to understand and address potential risks without enabling an unscrupulous actor to use their work as a guide, they omitted details from their paper that would facilitate such actions outside of a laboratory setting. On a basic level, they discovered that they could trigger identification of the specific device, including its model and serial number. This, in turn, yielded the ability to elicit more detailed data about a hypothetical patient, including name, diagnosis, and other sensitive details stored on the device. From there, the researchers tested a number of scenarios in which they sought to actively interfere with the device, demonstrating the ability to change a patient’s name, reset the clock, run down the battery, and disable therapies that the device was programmed to deliver. They were also able to bypass the safeguards put in place by the manufacturer to prevent the accidental issuance of electrical shocks to the patient’s heart, thereby potentially triggering shocks to induce hypothetical fibrillation after turning off the ICD’s automatic therapies.
Equipment used in the 2008 study to test the security of a commercially available ICD.
The team set out to not only identify potential flaws in implantable medical technology, but also to offer practical solutions that would empower manufacturers, providers, and patients to mitigate the potential risks. The researchers developed prototypes for three categories of defenses that could ostensibly be refined and built into future ICD models. They dubbed these “zero-power defenses,” meaning they did not need to draw power from the device’s battery to function but instead harvested energy from external radio frequency (RF) signals. The first, zero-power notification, provides the patient with an audible warning in the event of a security-sensitive event. To prevent such events in the first place, the researchers also proposed a mechanism for zero-power authentication, which would enable the ICD to verify it is communicating with an authorized programmer. The researchers complemented these defenses with a third offering, zero-power sensible key exchange. This approach enables the patient to physically sense a key exchange to combat unauthorized eavesdropping of their implanted device.
Upon releasing the results of their work, the team took great pains to point out that their goal was was to aid the industry in getting ahead of potential problems; at the time of the study’s release, there had been no reported cases of a patient’s implanted device having been compromised in a security incident. But, as Kohno reflects today, the key to computer security research is anticipating the unintended consequences of new technologies. It is an area in which the University of Washington has often led the way, thanks in part to Kohno and faculty colleague Franziska Roesner, co-director of the Security and Privacy Research Lab. Other areas in which the Allen School team has made important contributions to understanding and mitigating privacy and security risks include motor vehicles, robotics, augmented and virtual reality, DNA sequencing software, and mobile advertising — to name only a few. Those projects often represent a rich vein of interdisciplinary collaboration involving multiple labs and institutions, which has been a hallmark of the lab’s approach.
Professor Tadayoshi Kohno (left) and Daniel Halperin
“This project is an example of the types of work that we do here at UW. Our lab tries to keep its finger on the pulse of emerging and future technologies and conducts rigorous, scientific studies of the security and privacy risks inherent in those technologies before adversaries manifest,” Kohno explained. “In doing so, our work provides a foundation for securing technologies of critical interest and value to society. Our medical device security work is an example of that. To my knowledge, it was the first work to experimentally analyze the computer security properties of a real wireless implantable medical device, and it served as a foundation for the entire medical device security field.”
Security Lab undergraduate researchers Mitali Palekar and Kimberly Ruth both presented today at the UW Undergraduate Research Symposium. Mitali will present her published work on “Analysis of the Susceptibility of Smart Home Programming Interfaces to End User Error” again next week at the IEEE Workshop on the Internet of Safe Things (SafeThings 2019), and Kimberly will present her published work on “Secure Multi-User Content Sharing for Augmented Reality Applications” again at the 28th USENIX Security Symposium (USENIX Security 2019) in August. Congratulation Mitali and Kimberly on the great work!
Quoting from the Allen School News article on the topic: Chen’s research interests lie at the intersection of technology and crime, physical safety, and at-risk populations. Her recent work has focused on technology and survivors of human trafficking. Chen just wrapped up a study in which she interviewed victim service providers (VSPs) to expose how technology can be utilized to re-victimize survivors of trafficking and understand how VSPs mitigate these risks as they interact with and support survivors. As a result of this work, Chen and her collaborators propose privacy and security guidelines for technologists who wish to partner with VSPs to support and empower trafficking survivors. The study will be presented at the upcoming USENIX Security Symposium in August.
