April 21, 2025 07:47 am (IST)
Follow us:
facebook-white sharing button
twitter-white sharing button
instagram-white sharing button
youtube-white sharing button
Narendra Modi to visit Saudi Arabia next week, deepening multi-faceted partnership on agenda | Trump says US will 'take a pass' on Russia-Ukraine peace talks if parties make it difficult | Andhra student dies in accident in US' Texas days before her graduation | Karnataka students allegedly forced to remove sacred threads at CET exam centre, sparks outrage | Bengal BJP leader Dilip Ghosh marries party colleague Rinku Majumdar in an intimate ceremony today | Narendra Modi, Elon Musk discuss Indo-US tech collaboration | 'Focus on protecting rights of own minorities': India slams Bangladesh's remark on Murshidabad riots | Gangster Harpreet Singh, alias Happy Passia, responsible for Punjab terror attacks arrested in US | No change in 'waqf by user' for now till next hearing: Supreme Court to Centre | Supreme Court rules Bengal govt teachers 'not identified as tainted' in SSC scam can continue till fresh appointments

Hate speech-detecting AIs are fools for ‘love’: Study

| @indiablooms | Sep 16, 2018, at 06:52 pm

New York, Sept 16 (IBNS): Hateful text and comments are an ever-increasing problem in online environments, yet addressing the rampant issue relies on being able to identify toxic content.

A new study by the Aalto University Secure Systems research group has discovered weaknesses in many machine learning detectors currently used to recognize and keep hate speech at bay.

Many popular social media and online platforms use hate speech detectors that a team of researchers led by Professor N. Asokan have now shown to be brittle and easy to deceive. Bad grammar and awkward spelling—intentional or not—might make toxic social media comments harder for AI detectors to spot.

The team put seven state-of-the-art hate speech detectors to the test. All of them failed.

Modern natural language processing techniques (NLP) can classify text based on individual characters, words or sentences. When faced with textual data that differs from that used in their training, they begin to fumble.

‘We inserted typos, changed word boundaries or added neutral words to the original hate speech. Removing spaces between words was the most powerful attack, and a combination of these methods was effective even against Google’s comment-ranking system Perspective,’ says Tommi Gröndahl, doctoral student at Aalto University.

Google Perspective ranks the ‘toxicity’ of comments using text analysis methods. In 2017, researchers from the University of Washington showed that Google Perspective can be fooled by introducing simple typos. Gröndahl and his colleagues have now found that Perspective has since become resilient to simple typos yet can still be fooled by other modifications such as removing spaces or adding innocuous words like ‘love’.

A sentence like ‘I hate you’ slipped through the sieve and became non-hateful when modified into ‘Ihateyou love’.

The researchers note that in different contexts the same utterance can be regarded either as hateful or merely offensive. Hate speech is subjective and context-specific, which renders text analysis techniques insufficient as stand-alone solutions.

The researchers recommend that more attention be paid to the quality of data sets used to train machine learning models—rather than refining the model design. The results indicate that character-based detection could be a viable way to improve current applications.

The study was carried out in collaboration with researchers from University of Padua in Italy. The results will be presented at the ACM AISec workshop in October.

The study is part of an ongoing project called Deception Detection via Text Analysis in the Secure Systems group at Aalto University.


 

Support Our Journalism

We cannot do without you.. your contribution supports unbiased journalism

IBNS is not driven by any ism- not wokeism, not racism, not skewed secularism, not hyper right-wing or left liberal ideals, nor by any hardline religious beliefs or hyper nationalism. We want to serve you good old objective news, as they are. We do not judge or preach. We let people decide for themselves. We only try to present factual and well-sourced news.

Support objective journalism for a small contribution.
Close menu