Automated Detection of Online Harms for Social Media Applications
Claudia Peersman, Minhao Zhang, Rohit Nautiyal – University of Bristol
This project will analyse the current (and future potential) abuse of commercially available PETs for criminal purposes, using specific online harms as primary case examples, and considering the potential role of automated methods to address these. These automated methods will be based on a combination of novel text mining and image/video analysis techniques that are able to flag a range of online harms on social media. Specifically:
- The sale of illegal goods and services, such as drugs, weapons, and online fraud or hacking-related services;
- Child sexual abuse, exploitation and grooming;
- Harassment;
- Sextortion;
- Cyber bullying;
- Trolling, aggression and hate speech;
- Depression and self-harm;
- Radicalisation
A multidisciplinary approach will be used, combining the development of intelligent technologies to tackle online harms with a qualitative approach that will incorporate a series of semi-structured interviews with relevant stakeholders combined with an open-source literature review. Guidance will be developed based on the interviews regarding both potential mechanisms of online harm arising as a result of (a) different PETs and (b) the differential use of these PETs across the specific online harm contexts listed, and suggestions for potential mitigations that consider the wider socio-technical space (e.g., future design, regulatory approaches, usage conditions etc.).