Cybersafety Threats - from Deception to Aggression

Summary

A number of malicious activities are prospering online and are putting users at risk. In particular, cyber deception and cyber aggression practices are increasing their reach and seriousness, leading to a number of harmful practices such as phishing, disinformation, radicalization, and cyberbullying. Attack strategies include controlling and operating fake or compromised social media accounts, artificially manipulating the reputation of online entities, spreading false information, and manipulating users via psychological principles of influence into performing behaviors that are counter to their best interests and benefit the attackers.

So far, computer science research on cybersafety has looked at the various sub-problems in isolation, mostly relying on algorithms aimed at threat detection, and without considering the implications of the attacks and countermeasures for individual users as well as for society. On the other hand, human factors and social science researchers often consider user interfaces and social interactions without taking full advantage of the algorithmic, data-driven cybersafety research. Moreover, the legal and ethical implications of attacks and countermeasures are often unclear.

The goal of the Dagstuhl Seminar 19302 "Cybersafety Threats - from Deception to Aggression" was to provide a platform for researchers to look at the problem of cybersafety from a holistic and multi-disciplinary perspective. The participants were drawn from a number of disciplines such as computer science, criminology, psychology, and education, with the aim of developing new ideas to understand and mitigate the problems.

At the beginning of the seminar, we asked participants to identify important themes to focus on, and these themes were refined through specific activities and discussions during the first day: Firstly, all participants gave 5-minute talks where they presented their current research related to the seminar, and their expectations and topics they would like to work on during the week. Secondly, we conducted three introductory panels on the topics of Cyber Deception, Cyber Aggression and Propaganda & Disinformation. Each panel consisted of five participants. We took special care to represent different disciplines and different career stages in each panel.

By the beginning of the second day, participants had identified four key themes to study in this area, which we describe in detail in the rest of this section. The participants formed working groups (WGs) for each theme.

Theme 1: Attacker modeling

The working group focused on predicting the next steps of an ongoing attack by means of a probabilistic model. The initial model developed by the group consists of 9 variables: attacker goals, characteristics of the attack (e.g., how long the attack takes, tools employed), consequences, authorization, attribution, expected resilience of the victim, expected characteristics of the victim from attacker's perspective, actual characteristics of the victim, actual responsiveness of the victim. The developed model was verified and refined using two known attacks as case studies: the Internet Worm (1988) and the SpamHaus DDoS attack (2013).

Two most important next steps to refine the model are:

1. Convert the variables into measurable quantities
2. Obtain labeled data on which the model can be trained

The working group started working on a conceptual paper that describes the model, and discussed possible venues for its publication. Several methods of obtaining the data for the model were proposed, such as interviewing CISOs and other defenders, creating financial incentives for organization to share their data, and organizing a stakeholder workshop including not only defenders, but also former attackers who now work as security consultants.

Theme 2: Unintended consequences of countermeasures

This working group focused on an often overlooked aspect of computer security research: the fact that deploying any countermeasure to mitigate malicious online activity can have unexpected consequences and harms to other parties. The members of this working group started by discussing a number of scenarios: intimate partner abuse, CEO fraud, disinformation, online dating fraud, and phishing, and developed a taxonomy of these potential harms. The taxonomy takes into account not only technical issues that might arise from deploying countermeasures but also socio-technical ones such as the displacement effect of attackers moving to other victims, the additional costs incurred by using the countermeasure, and the issues arising from complacency, for example leaving users desensitized by displaying too many alerts to prevent a certain type of attack.

Theme 3: Measuring human behavior from information security (and societal) perspectives

Measuring online behavior is of fundamental importance to gain an accurate understanding of malicious online activities such as cybercrime. The research community, however, does not have well established techniques to accurately measure this behavior, and this can lead to studies presenting largely contradicting results. This working group focused on identifying techniques relevant to measure and model various types of online behavior, from cyberbullying and disinformation to ransomware and phishing. As a final outcome, the working group drafted two methodological frameworks for researchers aiming to study these problems, one focused on socio-technical threats (cyberbullying and disinformation) and one focused on cybersecurity (phishing and malware).

Theme 4: Prevention, detection, response and recovery.

A key challenge when mitigating socio-technical issues is developing the most effective countermeasures. This group focused on developing detection and prevention approaches focusing on threats encountered by adolescents when surfing the Web (e.g., cybergrooming). A common issue here is that adolescents rarely turn to adults for help, and therefore any mitigation based on direct parental oversight has limited effectiveness. To go beyond these issues, the group developed a mitigation strategy based on a "guardian angel" approach. The idea is to let a minor create a "guardian avatar"' that will then advise them on cybersafety practices, with a decreasing level of oversight as the minor grows up. While the children are very young, the guardian avatar will closely supervise them, reporting any suspicious contacts that they have online to a parent or a guardian. Later, as the child enters adolescence, the avatar will gradually take on an advisory role, eventually only providing advice once the adolescent asks for it. The group considered privacy issues and interdisciplinary aspects related to psychology and education, and developed a proposal of how the avatar would work.

Conclusion and Future Work

The seminar produced a number of ideas on how to investigate and mitigate cybersafety threats. It enabled researchers from different disciplines to connect, and set the agenda for potentially impactful research to be carried out in the next years. Joint publications and funding for joint research were discussed in each WG and later in the plenum. For example, WG 3 considered possibilities for a large international grant, such as H2020. The ideas produced as part of theme 4 resulted in the paper "Identifying Unintended Harms of Cybersecurity Countermeasures" to appear at the APWG eCrime Symposium in November 2019.