Sara Fish • My thoughts on AI risk

My thoughts on AI risk

July 15, 2026

I am deeply concerned about the potential risks from AI. Accordingly, much of my research is motivated by these risks. However, I get the sense that the forms of AI risk I am most worried about differ from those most frequently discussed in AI safety spaces (broadly construed).

In this post, I briefly outline my opinions on AI risk prioritization. I'm not claiming these are super novel or comprehensively defended takes; I am just putting my current personal opinions in one place.

TLDR: I am most worried about (1) the negative effects AI systems will have on their human users, (2) concentration of power in the AI labs and/or governments that nationalize them, (3) AI weakening human-to-human social relationships. I am comparatively less worried about (4-7) CBRN, cyber, rogue AI takeover, and rogue human takeover using an AI.

My reasoning is pretty first-order: (4-7) seem like they will be fixed either way because it is in the business interests of the frontier labs to mitigate these risks, whereas (1), (2), and (3) are at best orthogonal and at worst antithetical to the frontier labs’ business interests.

Thanks to a handful of friends who gave helpful feedback on this post, even though they disagree with it.

The three forms of AI risk that worry me most

(1) Negative effects AI systems will have on their human users. One important, if rare, example is psychosis. Another more broadly-applicable example would be deskilling. Deskilling is tricky to talk about, because some deskilling is natural when adopting new technology. Still, I worry about deskilling due to overreliance on AI, via people "taking the easy way out" and not bothering to improve their human capital. For example, I find it concerning when junior researchers struggle to articulate research ideas or interests without brainstorming help from AI.

Finally, I think overexposure to AI writing (or outputs more generally) is bad for people. AI outputs, by design, reward hack the human mind. AI writing feels bad for my brain in the same way that trashy TV or addictive algorithmic feeds do. If you have ever had the experience of skimming AI writing, thinking it looks good, then reading more carefully, and realizing it's utter nonsense, you know what I mean. I think an increasing number of jobs are going to consist of overseeing AI outputs, and that these workers' mental health will be negatively impacted, due to the nature of AI writing itself.

(2) Concentration of power in the AI labs and/or the governments that nationalize them. One example is AI for math. If OpenAI or Google actually cared about advancing mathematics research for the common good, they would be working hard to make their frontier models accessible to thousands of mathematicians, help build infrastructure to process, verify, and disseminate the glut of resulting new mathematics research, and so on. But they are not. Instead they are poaching top mathematicians to do internal work, and publishing research according to a calculated PR strategy. That is, they are amassing “mathematical power”, because this is better-aligned with their business interests -- which do not value mathematics intrinsically, but rather for its good PR and/or potential emergent alignment.

I would encourage you to think about other examples of causes the labs claim to care about and think through: if they actually cared about this cause, is this the most effective way to work on it? Or is what they are doing just addressing a business interest, and they are framing it in a certain way to have better optics, as they amass power?

(3) AI weakening human-to-human social relationships. "AI girlfriend" situations are obviously one example. More broadly, increasingly, people turn to AI for advice, rather than other people. Even if AI advice is adequate (or better) in the moment, this weakens our bonds with other people in the long run.

Other forms of AI risk that worry me less

(4-7) The typical laundry list of AI risks motivated by existential risk: CBRN (i.e., bio + nuclear), cyber, rogue AI takeover, rogue human takeover using an AI. I think these are still a concern, but I think it is in the business interests of the frontier labs to mitigate these risks. For example, mitigating cyber risk seems to be helpful for maintaining good relations with the U.S. government, which in turn are good for business. We saw this with the rollout of Fable 5.

I think each frontier lab is best-modelled as having some fixed safety target they decide based on their business interests, and they will pay high salaries to recruit researchers to meet this safety target, whether or not those researchers are ideologically aligned with AI safety. Accordingly, I am not convinced that an AI-safety-concerned person will have much of a counterfactual impact working on these problems as an IC at a frontier lab. My impression (based only on rumors) is that xAI is a good example here.

(8-9) Near-term economic impacts of AI, and the terrible information environment surrounding AI. I am also concerned about these risks, but a bit less than (1)-(3).

Conclusion

Here are some overarching principles underlying the above. First, to understand what the frontier labs will work on, I think it is useful to model them in terms of their business interests. Second, I think the critical risks from AI reach far beyond (4-7), contrary to the narrative perpetuated by certain frontier labs. And third, I think the counterfactual impact of ICs working on “safety” in frontier labs is essentially zero.

Based on this, pieces of advice I would give to AI-safety-concerned (junior) people include: (A) avoid positions at frontier labs, or at external organizations that are thinly-veiled vassals of frontier labs, (B) seek out positions at organizations that curtail the labs' power and/or actually strive for improving human welfare, (C) do research on real human subjects, and/or in collaboration with real human experts on relevant topics such as psychology.