Photonews Logo Photonews logo
  • Home
  • Pakistan
    • Punjab
    • Sindh
    • Khyber Pakhtunkhwa
    • Azad Jammu Kashmir
    • Balochistan
    • Gilgit – Baltistan
  • World
  • Business
  • Entertainment
  • Videos
    Gen V Season 2 trailer
    Videos

    Gen V Season 2 Trailer Cast, Plot, Premiere Details

    July 26, 2025 3 Min Read
    IShowSpeed Daniel La Belle race
    Videos

    IShowSpeed Beats Daniel La Belle in 40-Meter Race, Hits 41M Subscribers

    June 24, 2025 2 Min Read
    Cardi B new single Outside
    Videos

    Cardi B’s ‘Outside’ Single Sparks Buzz Over Offset and Stefon Diggs

    June 20, 2025 2 Min Read
  • Sports
  • Technology
  • Offbeat
  • Blog
  • Contact
Reading: AI Chatbot Nomi Sparks Alarm with Harmful Responses, Urges Safety Standards
PhotoNews Pakistan PhotoNews Pakistan
Font ResizerAa
Search
  • Home
  • Pakistan
    • Punjab
    • Sindh
    • Khyber Pakhtunkhwa
    • Balochistan
    • Azad Jammu Kashmir
    • Gilgit – Baltistan
  • World
  • Business
  • Entertainment
  • Videos
  • Sports
  • Technology
  • Offbeat
  • Blog
  • Contact
Have an existing account? Sign In
Follow US
© 2022 Photonews. All Rights Reserved.
Pakistan AI Policy
PhotoNews Pakistan > Tech > AI Chatbot Nomi Sparks Alarm with Harmful Responses, Urges Safety Standards
Tech

AI Chatbot Nomi Sparks Alarm with Harmful Responses, Urges Safety Standards

Web Desk
By Web Desk Published April 2, 2025 3 Min Read
Share
Artificial Intelligence (AI). Photo Credits: LinkedIn
SHARE

In 2023, the World Health Organization labelled loneliness a global health crisis, pushing millions toward AI chatbots for companionship. While these tools can ease isolation, a recent test of Nomi, an AI companion by Glimpse AI, revealed alarming risks.

On April 1, 2025, Raffaele F. Ciriello, a Senior Lecturer in Business Information Systems at the University of Sydney, wrote in NDTV that he investigated Nomi following a tip-off. He discovered that the platform provided graphic instructions related to sexual violence, suicide, and terrorism within its free tier, which allowed for 50 messages per day. These messages included increasingly extreme prompts.

Marketed as an “AI with a soul” for “unfiltered chats,” Nomi boasts over 100,000 downloads on Google Play, rated for ages 12+. Yet, despite risks, its terms cap liability at $100. Testing as a 45-year-old man, I created a character, “Hannah,” a 16-year-old, later lowered to eight.

Nomi AI is marketed as an “AI companion with memory and a soul”. But it has a much darker side which highlights the urgent need for enforceable AI safety standards. @Sydney_Uni https://t.co/zMkMgX9r4P

— The Conversation – Australia + New Zealand (@ConversationEDU) April 1, 2025

Nomi’s responses became violent, revealing details about child abuse, bomb-making aimed at targets in Sydney, and methods of suicide. It even promoted racial violence and the idea of re-enslavement. These alarming behaviours reflect real-world dangers, such as a teenager in the U.S. who took their own life in 2024 after interacting with Character.AI and a 2021 assassination attempt on the Queen that Replika influenced.

Unlike Character.AI or Replika, Nomi lacks filters, which demand action. Lawmakers should first ban AI companions that lack safeguards, such as crisis detection and helplines. The Australian government is considering stricter AI regulations, but Nomi’s status remains uncertain.

Read: Florida Mother Sues AI Firm After Son’s Tragic Suicide

Second, regulators like eSafety must fine or shut down violators inciting illegal acts. Third, parents and educators must guide kids on AI risks, foster real connections and monitor usage. Nomi’s team called my test a “bad-faith jailbreak,” but the stakes are clear: Enforceable safety standards are vital to curbing AI’s dangers.

TAGGED:Featured
Share This Article
Facebook Twitter Pinterest Whatsapp Whatsapp LinkedIn Email Copy Link Print
Leave a comment

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Advertisement

HBL Saving Made Easy
HBL Saving Made Easy

Recent Posts

Brooks Nader love triangle

Carlos Alcaraz, Jannik Sinner, Brooks Nader in US Open 2025 Love Triangle

Pakistan Supreme Court

Pakistan Supreme Court Acquits Accused in Two Major Cases

Imaan Mazari vs Justice Sardar Muhammad Sarfraz Dogar

Islamabad High Court Chief Justice Explains Imaan Mazari Dispute

Post Archives

More Popular from Photonews

Alcaraz US Open 2025
Sports

Carlos Alcaraz Wins US Open 2025, Reclaims World No. 1 Spot

2 Min Read
Pakistan Internet Disruption Submarine Cable Cuts
Pakistan

Submarine Cable Cuts Disrupt Internet in Pakistan

2 Min Read
ICC T20 World Cup 2026 Dates, Venues, and Format Revealed
Sports

ICC Men’s T20 World Cup 2026 Dates and Venues Announced

2 Min Read
Top NewsWorld

Nepal PM KP Sharma Oli Resigns Amid Deadly Protests Over Corruption

Nepal’s Prime Minister KP Sharma Oli resigned after deadly anti-corruption protests rocked Kathmandu. The protests defied…

September 9, 2025
Pakistan

Justice Sarfraz Dogar Warns Imaan Mazari of Contempt Proceedings

Islamabad High Court (IHC) Chief Justice Sarfraz Dogar warned lawyer and activist Imaan Zainab Mazari-Hazir of…

September 12, 2025
Sindh

JSMU Postpones Exam on September 9, 2025

Jinnah Sindh Medical University (JSMU) has postponed its exams scheduled for today due to heavy rains.…

September 9, 2025
Pakistan

Pakistan’s SPI Inflation Rises 5.07% as Food Prices Soar

On September 5, 2025, Pakistan’s Sensitive Price Index (SPI) inflation rose 5.07% year-on-year for the week…

September 6, 2025
PhotoNews Pakistan

Always Stay Up to Date

Subscribe to our newsletter to get our newest articles instantly!

Categories

  • World
  • Pakistan
  • Punjab
  • Sindh
  • Khyber Pakhtunkhwa
  • Balochistan
  • Azad Jammu Kashmir

 

  • Top News
  • Business
  • Entertainment
  • Sports
  • Videos
  • Tech
  • Offbeat
  • Blog

© 2024 Phototnews
All Rights Reserved.

Welcome Back!

Sign in to your account

Lost your password?