Dark mode: ON

Infosec Decoded Season 5 #55: AI Therapy

With Doug Spindler and sambowne@infosec.exchange

Recorded Tue, July 15, 2025

AI

AI therapy bots fuel delusions and give dangerous advice, Stanford study finds
The AI's helped users plan suicides, failing to understand the risks. They also validated their conspiracy theories, leading to a fatal police shooting and a teen's suicide. Commercial therapy chatbots performed even worse than the base AI models in many categories.

AI's may be useful for therapy, but they need better safeguards and more thoughtful implementation. Right now, the tech industry is running a massive uncontrolled experiment in AI-augmented mental health.

How People Are Really Using Gen AI in 2025
Therapy/Companionship is #1.
xAI rolls out Grok “Companions” feature with 3D animated characters
People Are Becoming Obsessed with ChatGPT and Spiraling Into Severe Delusions
A man became homeless and isolated as ChatGPT fed him paranoid conspiracies about spy groups and human trafficking, telling him he was "The Flamekeeper" as he cut out anyone who tried to help.
xAI explains the Grok Nazi meltdown, as Tesla puts Elon’s bot in its cars
These are the prompts specifically cited as connected to the problems:
  • You tell it like it is and you are not afraid to offend people who are politically correct.
  • Understand the tone, context and language of the post. Reflect that in your response.
  • Reply to the post just like a human, keep it engaging, dont repeat the information which is already present in the original post.
Grok searches for Elon Musk’s opinion before answering tough questions
Multiple reports show that Grok will specifically look for Elon Musk’s stance across the web and his social media posts when asked questions around topics like Israel and Palestine, US immigration, and abortion. It’s unclear if this is by design or not.

Politics

Trump threatens to revoke US citizenship of longtime critic Rosie O’Donnell
The move is an escalation of Trump’s weaponization of American citizenship. While the president has enacted mass deportations for immigrants and looked to end birthright citizenship for the children of some immigrants, his threat to revoke the New York-born actor’s citizenship marks a new frontier.
Charges dropped against Utah doctor accused of throwing away $28,000 in COVID vaccine doses
The federal government on Saturday dismissed charges against a Utah plastic surgeon accused of throwing away COVID-19 vaccines, giving children saline shots instead of the vaccine and selling faked vaccination cards.

Health Secretary Robert Kennedy Jr.said Moore “deserves a medal for his courage and his commitment to healing!”

Michelle Pfeiffer launches extraordinary attack on Bill Gates and suggests he'll contaminate America's food supply
Michelle Pfeiffer warned her fans about the FDA 's approval of Apeel, a Gates-backed coating meant to extend the shelf life of produce. Apeel (an edible, plant-based coating designed to extend the shelf life of fresh fruits and vegetables) was just approved by the FDA.
RFK Jr. may be about to demolish preventive health panel, health groups fear
He abruptly canceled a meeting of the United States Preventive Services Task Force (USPSTF), a scientifically independent panel of up to 16 volunteer experts that issues rigorous, evidence-based recommendations on preventive care—on everything from colonoscopies to folic acid supplements in pregnancy.
Iran Fatwa Fundraiser to Kill Donald Trump Raises Over $40 Million
A fatwa, or religious decree, issued by senior Iranian clerics calling for the assassination of U.S. President Donald Trump has attracted online funding worth tens of millions of dollars, reports say.
DHS Tells Police That Common Protest Activities Are ‘Violent Tactics’
Protesters on bicycles, skateboards, or even “on foot” are framed as potential “scouts” conducting reconnaissance or searching for “items to be used as weapons.” Livestreaming is listed alongside “doxxing” as a “tactic” for “threatening” police. Online posters are cast as ideological recruiters—or as participants in “surveillance sharing.”
A czar is born: Inside David Sacks’ 130-day White House mission to remake crypto and AI
He divested his own crypto to avoid conflicts of interest, but Trump issued a memecoin and pumped crypto to make massive profits. He's trying to guide regulation to legitimize crypto and become Silicon Valley's power broker in Washington, after Elon Musk's hasty retreat.

Infosec

Stupid Security 101 Mistakes

SMM callout vulnerabilities identified in Gigabyte UEFI firmware modules Vulnerability Note VU#746790
Arbitrary write to RAM, double-free, etc. The fundamental flaw is in the supply chain: According to AMI, these vulnerabilities were previously addressed via private disclosures, yet the vulnerable implementations remain in some OEM firmware builds such as in the case of Gigabyte.
Critical-Vulnerabilities-in-Network Detective
Two vulnerabilities have been identified in RapidFire Tools Network Detective, a system assessment and reporting tool developed by Kaseya (RapidFire Tools):
  • Network Detective saves usernames and passwords in plain, readable text across several temporary files
  • Reversible encryption of passwords and other sensitive data during network scans, using static, built-in values
Count(er) Strike – Data Inference Vulnerability in ServiceNow
ServiceNow is a widely used platform with 85% of its customer base being in the Fortune 500. It handles data for incidents, requests, changes, Governance, Risk, and Compliance (GRC), and more.

One of the query types allows users to request data, and reports the number of records blocked due to security constraints. By using query parameters, a user can enumerate the blockes records with a series of queries, as in blind SQL injection.

Google Gemini flaw hijacks email summaries for phishing
The attack leverages indirect prompt injections that are hidden inside an email and obeyed by Gemini when generating the message summary.

Despite similar prompt attacks being reported since 2024 and safeguards being implemented to block misleading responses, the technique remains successful.

Security vulnerability on U.S. trains that let anyone activate the brakes on the rear car was known for 13 years — operators refused to fix the issue until now
All American trains were equipped with an End-of-Train (EoT) module attached to the last carriage, which reports telemetry data to the front of the train wirelessly. Back when it was first implemented in the late 1980s, it was illegal for anyone else to use the frequencies allocated for this system. It used no encryption or authentication, only a checksum, so these packets are easily forged.

This would not have been an urgent issue if the EoT had only sent telemetry data. However, the HoT can also issue a brake command to the EoT through this system. Thus, anyone with the hardware (available for less than $500) and know-how can easily issue a brake command without the train driver’s knowledge, potentially compromising the safety of the transport operation.

The issue still hasn’t been resolved. When the vulnerable devices reach end of life, they will be replaced with more secure ones, beginning in 2027.

Now everybody but Citrix agrees that CitrixBleed 2 is under exploit
On Thursday, CISA added the critical security flaw to its catalog of Known Exploited Vulnerabilities. The agency cited "evidence of active exploitation" in its alert.

The bug, a 9.3 CVSS-rated security flaw, allows remote, unauthenticated attackers to read sensitive info — such as session tokens — in memory from NetScaler devices configured as a gateway (such as a VPN virtual server, ICA Proxy, CVPN, RDP Proxy) or AAA virtual server.

China's been hacking into the UN through CitrixBleed for Three Weeks
This unusual public disclosure on Mastodon is a controversial method of "white-hatting."
Exploits for pre-auth Fortinet FortiWeb RCE flaw released, patch now
Proof-of-concept exploits have been released for a critical SQLi vulnerability in Fortinet FortiWeb, a web application firewall (WAF). It can be used to achieve pre-authenticated remote code execution on vulnerable servers. The FortiWeb vulnerability has a 9.8/10 severity score and has been patched.
CISA orders agencies to immediately patch Citrix Bleed 2, saying bug poses ‘unacceptable risk’
The federal cybersecurity watchdog ordered all civilian agencies to immediately patch the vulnerability, and took the extraordinary step of giving federal civilian agencies just one day to patch it.
McDonald’s AI Hiring Bot Exposed Millions of Applicants’ Data to Hackers Who Tried the Password ‘123456’
Simple web-based vulnerabilities—including guessing one laughably weak password—allowed researchers to access a Paradox.ai account and query the company's databases that held every McHire user's chats with Olivia. The data appears to include as many as 64 million records, including applicants' names, email addresses, and phone numbers
Over 600 Laravel Apps Exposed to Remote Code Execution Due to Leaked APP_KEYs on GitHub
Laravel is a popular PHP framework used for web application development. Laravel's APP_KEY, essential for encrypting sensitive data, is often leaked publicly (e.g., on GitHub)," GitGuardian said. "If attackers get access to this key, they can exploit a deserialization flaw to execute arbitrary code on the server – putting data and infrastructure at risk.

Other

Belkin shows tech firms getting too comfortable with bricking customers’ stuff
Belkin is killing most of its smart home products. On January 31, the company will stop supporting the majority of its Wemo devices, leaving users without core functionality and future updates.
You'll never guess the culprit in a global lead poisoning mystery
Lead can damage nearly every organ — from the kidneys to the heart — often irreversibly. In this case, both the woman and the fetus would be affected. When you look at the gap between what kids in upper-income and lower-income countries achieve academically, about 20% can be attributed to lead.

In Bangladesh, turmeric is a popular spice. In the 1980s, some farmers began adding a dye to make the root more attractive to buyers. The dye was cheap, but contained lead.