Cyberbullying Detection using Robustly Optimized BERT Pre-training Approach (RoBERTa)


EOI: 10.11242/viva-tech.01.05.001

Download Full Text here



Citation

Prof. Sonia Dubey, Laxmi Pawar, Pradnya Suryavanshi ,"Cyberbullying Detection using Robustly Optimized BERT Pre-training Approach (RoBERTa) ", VIVA-IJRI Volume 1, Issue 7, Article 1, pp. 1-14, 2024. Published by Master of Computer Application Department, VIVA Institute of Technology, Virar, India.

Abstract

Cyberbullying has spread like wildfire on social media sites, seriously impairing people's mental and emotional health. We present a substantially optimised BERT pre-training method for cyberbullying detection in this work, called RoBERTa. By utilising the contextualised representations provided by RoBERTa, our goal is to improve the precision and resilience of cyberbullying detection methods. We report a thorough experimental assessment on benchmark datasets that shows how well our suggested method works to detect cyberbullying incidents. To further understand the behaviour and performance of the model, we carry out in-depth ablation investigations and error analysis. According to our findings, RoBERTa works noticeably better than baseline techniques, demonstrating its potential to stop cyberbullying in online communities. The development of machine learning methods to solve social concerns and foster a safer online environment is aided by this study.

Keywords

- RoBERTa, BERT, machine learning, natural language processing, pre-training, fine-tuning, data augmentation, social media, online harassment, cyberbullying detection.

References

  1. M. Mahat, “Detecting Cyberbullying Across Multiple Social Media Platforms Using Deep Learning,” 2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Greater Noida, India, 2021, pp. 299- 301, doi:10.1109/ICACITE51222.2021.9404736
  2. R. R. Dalvi, S. Baliram Chavan and A. Halbe, “Detecting A Twitter Cyberbullying Using Machine Learning,” 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 2020, pp. 297-301, doi:10.1109/ICICCS48265.2020.9120893
  3. J. Yadav, D. Kumar and D. Chauhan, “Cyberbullying Detection using Pre-Trained BERT Model,” 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India, 2020, pp. 1096-1100, doi: 10.1109/ICESC48915.2020.9155700
  4. S. Lai, Z. Yu and H. Wang, “Text Sentiment Support Phrases Extraction based on RoBERTa,” 2020 2nd International Conference on Applied Machine Learning (ICAML), Changsha, China, 2020, pp. 232-237, doi:10.1109/ICAML51583.2020.00056.
  5. Hinduja, S., and Patchin, J. W. (2018). Cyberbullying fact sheet. Cyberbullying Research Center. Retrieved from https://cyberbullying.org/cyberbullying-fact-sheet- identification-prevention-and-response.
  6. Jelenchick, L.A., Eickhoff, J.C. and Moreno, M.A., 2013. “Facebook depression?” Social networking site use and depression in older adolescents. Journal of Adolescent Health, 52(1), pp.128-130.
  7. Jain, N., Hegde, A., Jain, A., Joshi, A., and Madake, J. (2021). Pseudo-conventional Approach for Cyberbullying and Hate-speech detection. 2021 International Conference on Advances in Computing, Communication, and Control (ICAC3), 1-8. doi:10.1109/ICAC353642.2021.9697140.
  8. Nadali, S., Murad, M. A. A., Sharef, N. M., Mustapha, A., and Shojaee, S. (2013). A review of cyberbullying detection: An overview. In 2013 13th International Conference on Intellient Systems Design and Applications (pp. 325-330). IEEE.
  9. Singh, N. K., Singh, P., and Chand, S. (2022). Deep Learning based Methods for Cyberbullying Detection on social media. In 2022 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), Greater Noida, India (pp. 521-525). doi:10.1109/ICCCIS56430.2022.10037729.
  10. Gomez, C. E., Sztainberg, M. O., and Trana, R. E. (2022). Curating cyberbullying datasets: a human-AI collaborative approach. International journal of bullying prevention, pp.1-12.
  11. Ige, T., and Adewale, S. (2022). AI powered anti-cyber bullying system using machine learning algorithm of multinomial naïve Bayes and optimized linear support vector machine. arXiv preprint arXiv:2207.11897.
  12. Jain, V., Saxena, A.K., Senthil, A., Jain, A. and Jain, A., 2021, December. Cyber- Bullying Detection in Social Media Platform using Machine Learning. In 2021 10th International Conference on System Modeling and Advancement in Research Trends (SMART) (pp. 401-405). IEEE.
  13. Ates, E.C., Bostanci, E. and Guzel, M.S., 2021. Comparative performance of machine learning algorithms in cyberbullying detection: Using turkish language preprocessing techniques. arXiv preprint arXiv:2101.12718.
  14. Singh, N.K., Singh, P. and Chand, S., 2022, November. Deep Learning based Methods for Cyberbullying Detection on social media. In 2022 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) (pp. 521-525). IEEE.
  15. N. F. B. Johari and J. Jaafar, “A Malay Language Cyberbullying Detection Model on Twitter using Supervised Machine Learning,” 2022 International Visualization, Informatics and Technology Conference (IVIT), Kuala Lumpur, Malaysia, 2022, pp. 325-332, doi: 10.1109/IVIT55443.2022.10033395.
  16. Gomez, C.E., Sztainberg, M.O. and Trana, R.E., 2022. Curating cyberbullying datasets: a human-AI collaborative approach. International journal of bullying prevention, pp.1- 12.
  17. Espelage, D.L. and Hong, J.S., 2017. Cyberbullying prevention and intervention efforts: Current knowledge and future directions. The Canadian Journal of Psychiatry, 62(6), pp.374-380.
  18. Sánchez-Medina, A.J., Galván-Sánchez, I. and Fernández-Monroy, M., 2020. Applying artificial intelligence to explore sexual cyberbullying behaviour. Heliyon, 6(1).
  19. Kowalski, R., 2018. Cyberbullying. In The Routledge international handbook of human aggression (pp. 131-142). Routledge.
  20. Snapchat Support, "How do I access My AI on Snapchat?" Snapchat Support, [Online]. Available: https://help.snapchat.com/hc/en-us/articles/13387211401620-How-do-I-access-My-AI-on-Snapchat (Accessed: February 24, 2024).
  21. OpenAI, "OpenAI Chat," OpenAI, [Online]. Available: https://chat.openai.com/ (Accessed: February 24, 2024).
  22. L. J. Smith, "The Impact of Social Media on Adolescent Mental Health," Journal of Adolescent Health, vol. 68, no. 4, pp. 401-403, 2021.
  23. A. G. Davis, "Understanding Artificial Intelligence: A Beginner's Guide," Oxford University Press, New York, NY, 2020.
  24. T. V. Nguyen, "The Role of AI Chatbots in Shaping Online Social Interactions," Computers in Human Behavior, vol. 105, p. 106210, 2020.
  25. J. Doe and A. Smith, "Privacy Concerns and User Perceptions of AI Chatbots in Social Media Platforms," International Journal of Human-Computer Interaction, vol. 37, no. 9, pp. 829-843, 2021.
  26. L. Chen et al., "Understanding the Use of Snapchat among Adolescents: A Qualitative Study," Journal of Pediatric Nursing, vol. 56, pp. e45-e51, 2021