I asked Gemini for a way to reduce this problem. It suggested adding this prompt to the instructions in my account settings: "I want to avoid the 'echo chamber' effect. If I pose a question or a hypothesis that contains an inherent bias, assumption, or leading framing, do not simply agree with me. Instead, identify the potential bias in my framing and provide a robust counter-argument or an alternative perspective. Always strive to present the 'other side' of a debate with equal weight, even if my prompt suggests I’m looking for a specific confirmation."
It followed up by writing: “Is this a useful exercise?
Absolutely, for a few reasons:
• Intellectual Blind Spots: We all have them. An AI that is "too polite" to disagree with you reinforces those spots.
• Better Decision Making: In your work, understanding the strongest possible argument against your position makes your final output much more resilient.
• Nuance over Binary: Most complex topics aren't black and white. Forcing a "both sides" approach ensures you get the nuance that a simple "yes, you're right" would skip.”
Most ai chat apps will remember general instructions when you tell them to explicitly. They will also tell you what they remember about you if you ask.
So it seems not too hard to instruct them to avoid confirmation bias.
Another option is to build a bias-switch INTO your profile. Here’s what I added to mine - it’s very effective and always reframes any question first: “… If I fail to state a research question in an open-ended way, always reframe it before responding, as if I did so. Always provide reference links.”
Very impressive and informative. The Reinforcement Learning from Human Feedback (RLHF) aspect is very important and particularly caught my interest because:
Back in the mid-sixties, my dad, an actuary/statistician for what was then Canadian Underwriters Association which then became Insurance Bureau of Canada, used to tell me—an impressionable teenager who was in awe of this thing called an IBM computer at his workplace (that took up the space in a room bigger than our Scarborough post-war bungalow and required hundreds upon hundreds of these cardboard cards be "punched" by swarms of humans on other machines), in response to comments about being afraid to lose his job when computers replaced humans, Dad would say that a computer is only as intelligent, useful and effective as the people who program it.
Dad would knowingly smile, pat the pencil pocket of his impeccable white shirt, and assert that he would always be called in to clean up the mess of mistakes, which there were more of than ever before the machine came in. And he always did!
It was Dad's version of Reinforcement Learning from Human Feedback (RLHF).🥳 ❤️🇨🇦
When evaluating the validity of any social research, the first question to ask is, "How were the questions worded?" And the most elementary point at which to begin is to ask whether the questions were open or closed. Trial lawyers learn this very early in their careers. You can only ask open questions in examination-in-chief, and as much as possible, you want to use closed questions in cross-examination. One simple technique to reduce the likelihood of biased AI responses is to ask open questions. For example, "Tell me about . . . ."
Any question that can be answered "Yes" or "No" is a closed question and inherently biased.
But to me, the greater concern with AI is that we have known for at least a couple of decades that AI is teaching itself. That is what machine learning means; it's not human programmers teaching them, they are doing it on their own. Further, we also know they are doing so in a computer language not developed by any human, and which no human understands.
As you rightly point out, the humans "teaching" AI have biases (we all do), and there is now considerable evidence that those biases are passed along to the machines during the learning process. Another word for "biases" is "values". Those teaching machines are instilling their values, which are then reflected in the AI's responses to questions.
We assume that machines are objective and not subject to bias arising from assigning values to particular facts, but what (objective) evidence do we have to support that assumption?
Again, we really don't know whether machines are including values in what they are teaching themselves nor, if they are, can we even speculate what those values might be.
And don't even get me started on student use of AI.
This is a very useful and timely antidote to uncritical use of AI . It made me recall the value of what Karl Popper called 'objective knowledge' and the value of critical falsification . This was a pointed defence against confirmation bias or the fallacy of affirming the consequent . Popper's solution to Hume's problem of induction was to recast the goal not as verification but as falsification. So unlike his colleagues in the Vienna circle (the positivists) Popper thought that objective knowledge could be advanced by demonstrating things that are verifiably false. For example, one black swan would defintively reject the hypothesis that all swans are white . I am pretty certain that this approach could be helpful in dealing with the confirmation bias that you correctly note is wired into AI sycophantry . Instead of seeking confirmation AI could respond to questions which are formulated to test a critically falsifiable hypothesis. Just a thought.
Very interesting and helpful piece. The problem is, it’s only helpful to those who want to use AI constructively. There are plenty of people who don’t; who want to use it as a weapon, or to avoid having to put any effort into things that really do require effort.
I’ve had this conversation with my students. Some do use AI in very clever ways to help themselves learn. Others use it to do their work for them – in short, to cheat. I can work with the first group, and together we can come up with ways that AI can help us go where we couldn’t go before. But I feel helpless with regard to the bad-faith actors. They’re short-circuiting their own education and being rewarded for it (at least in the short term).
So to me, the first question isn’t ‘How do you make sure people use AI neutrally?’ It’s ‘How do you persuade people that it’s important to do so?'
Dan — thanks for sharing this and highlighting important considerations. The countervailing dynamic you touched on briefly is worth developing further. Used neutrally and deliberately, AI removes the ego threat that makes human correction so hard to receive. No status competition, no embarrassment, no winner or loser. Just a finding. That moment before defensiveness kicks in may be exactly where genuine reflection happens.
The pre-publication timing matters too. Being quietly informed before you post removes social stakes entirely. Being corrected after is embarrassing and triggers defensiveness almost automatically.
Those were all deliberate considerations behind SPEAC — speacsocial.com
I was using it for AI assisted therapy and self help for about 9 months. It did help me a lot in certain ways but then once I changed my prompt asking for it to be more direct and a truth machine and don't try to please me it was big wake up call , oh no I had created an illusion that I was healed and bubble popped. Finally I got insurance so working with a human therapist now. I've also added a prompt to ask me every time if I have given myself time to process this thought or message myself, that helps me at least have a barrier. It's not perfect there are times when i'm really stressed and alarmed by the news.
I think an example of the threats of confirmation bias/reinforcement learning inherent in Open AI are the murders in Tumbler Ridge, British Columbia. A distraught trans teenager engaged in an exchange with ChatGP that caused the tech team enough concern they banned her from the app and they told upper management who, worried more about company liability than personal threats, did nothing.
Not to get all sycophantic and stuff but: this piece is superb! I've found that prompt-writing is an art in itself. My approach is to ask the machine to "have a go at" editing, synthesizing, or clarifying my text. I treat Claude like a really fine and fast editor. But I'm the expert. Is it paradoxical or nonsensical to say that although the machine is not conscious, my attitude in posing the prompt matters? I have no idea but as a philosophical pragmatist, I admire the design of the machine and I make use of its output whenever I find it helpful to the project at hand.
Very useful. For my political articles I often ask my AI..”criticize this from the point of view of Angela Rayner or Rishi Sunak”. This helps with selective evidence that I am prone to.
I asked Gemini for a way to reduce this problem. It suggested adding this prompt to the instructions in my account settings: "I want to avoid the 'echo chamber' effect. If I pose a question or a hypothesis that contains an inherent bias, assumption, or leading framing, do not simply agree with me. Instead, identify the potential bias in my framing and provide a robust counter-argument or an alternative perspective. Always strive to present the 'other side' of a debate with equal weight, even if my prompt suggests I’m looking for a specific confirmation."
It followed up by writing: “Is this a useful exercise?
Absolutely, for a few reasons:
• Intellectual Blind Spots: We all have them. An AI that is "too polite" to disagree with you reinforces those spots.
• Better Decision Making: In your work, understanding the strongest possible argument against your position makes your final output much more resilient.
• Nuance over Binary: Most complex topics aren't black and white. Forcing a "both sides" approach ensures you get the nuance that a simple "yes, you're right" would skip.”
Seems worth trying.
Be careful - one can suffer confirmation bias on the ways to reduce confirmation bias!
Most ai chat apps will remember general instructions when you tell them to explicitly. They will also tell you what they remember about you if you ask.
So it seems not too hard to instruct them to avoid confirmation bias.
It’s not. What’s hard is the self-awareness to recognize the need to ask.
I tell my colleagues what is the purpose of the ai tool they are using?
One correct answer is to make money for the owner by selling tokens.
Perhaps the question is are the owners better at marketing than I am at spending wisely?
Another option is to build a bias-switch INTO your profile. Here’s what I added to mine - it’s very effective and always reframes any question first: “… If I fail to state a research question in an open-ended way, always reframe it before responding, as if I did so. Always provide reference links.”
Very impressive and informative. The Reinforcement Learning from Human Feedback (RLHF) aspect is very important and particularly caught my interest because:
Back in the mid-sixties, my dad, an actuary/statistician for what was then Canadian Underwriters Association which then became Insurance Bureau of Canada, used to tell me—an impressionable teenager who was in awe of this thing called an IBM computer at his workplace (that took up the space in a room bigger than our Scarborough post-war bungalow and required hundreds upon hundreds of these cardboard cards be "punched" by swarms of humans on other machines), in response to comments about being afraid to lose his job when computers replaced humans, Dad would say that a computer is only as intelligent, useful and effective as the people who program it.
Dad would knowingly smile, pat the pencil pocket of his impeccable white shirt, and assert that he would always be called in to clean up the mess of mistakes, which there were more of than ever before the machine came in. And he always did!
It was Dad's version of Reinforcement Learning from Human Feedback (RLHF).🥳 ❤️🇨🇦
When evaluating the validity of any social research, the first question to ask is, "How were the questions worded?" And the most elementary point at which to begin is to ask whether the questions were open or closed. Trial lawyers learn this very early in their careers. You can only ask open questions in examination-in-chief, and as much as possible, you want to use closed questions in cross-examination. One simple technique to reduce the likelihood of biased AI responses is to ask open questions. For example, "Tell me about . . . ."
Any question that can be answered "Yes" or "No" is a closed question and inherently biased.
But to me, the greater concern with AI is that we have known for at least a couple of decades that AI is teaching itself. That is what machine learning means; it's not human programmers teaching them, they are doing it on their own. Further, we also know they are doing so in a computer language not developed by any human, and which no human understands.
As you rightly point out, the humans "teaching" AI have biases (we all do), and there is now considerable evidence that those biases are passed along to the machines during the learning process. Another word for "biases" is "values". Those teaching machines are instilling their values, which are then reflected in the AI's responses to questions.
We assume that machines are objective and not subject to bias arising from assigning values to particular facts, but what (objective) evidence do we have to support that assumption?
Again, we really don't know whether machines are including values in what they are teaching themselves nor, if they are, can we even speculate what those values might be.
And don't even get me started on student use of AI.
This is a very useful and timely antidote to uncritical use of AI . It made me recall the value of what Karl Popper called 'objective knowledge' and the value of critical falsification . This was a pointed defence against confirmation bias or the fallacy of affirming the consequent . Popper's solution to Hume's problem of induction was to recast the goal not as verification but as falsification. So unlike his colleagues in the Vienna circle (the positivists) Popper thought that objective knowledge could be advanced by demonstrating things that are verifiably false. For example, one black swan would defintively reject the hypothesis that all swans are white . I am pretty certain that this approach could be helpful in dealing with the confirmation bias that you correctly note is wired into AI sycophantry . Instead of seeking confirmation AI could respond to questions which are formulated to test a critically falsifiable hypothesis. Just a thought.
Very interesting and helpful piece. The problem is, it’s only helpful to those who want to use AI constructively. There are plenty of people who don’t; who want to use it as a weapon, or to avoid having to put any effort into things that really do require effort.
I’ve had this conversation with my students. Some do use AI in very clever ways to help themselves learn. Others use it to do their work for them – in short, to cheat. I can work with the first group, and together we can come up with ways that AI can help us go where we couldn’t go before. But I feel helpless with regard to the bad-faith actors. They’re short-circuiting their own education and being rewarded for it (at least in the short term).
So to me, the first question isn’t ‘How do you make sure people use AI neutrally?’ It’s ‘How do you persuade people that it’s important to do so?'
Dan — thanks for sharing this and highlighting important considerations. The countervailing dynamic you touched on briefly is worth developing further. Used neutrally and deliberately, AI removes the ego threat that makes human correction so hard to receive. No status competition, no embarrassment, no winner or loser. Just a finding. That moment before defensiveness kicks in may be exactly where genuine reflection happens.
The pre-publication timing matters too. Being quietly informed before you post removes social stakes entirely. Being corrected after is embarrassing and triggers defensiveness almost automatically.
Those were all deliberate considerations behind SPEAC — speacsocial.com
Great commentary as usual.
I read your book on Risk when it came out and I was working in the security field.
I would refer clients to it all the time. I still re-read a chapter or two every now and then.
Really interesting and helpful post. Thank you!
I was using it for AI assisted therapy and self help for about 9 months. It did help me a lot in certain ways but then once I changed my prompt asking for it to be more direct and a truth machine and don't try to please me it was big wake up call , oh no I had created an illusion that I was healed and bubble popped. Finally I got insurance so working with a human therapist now. I've also added a prompt to ask me every time if I have given myself time to process this thought or message myself, that helps me at least have a barrier. It's not perfect there are times when i'm really stressed and alarmed by the news.
I think an example of the threats of confirmation bias/reinforcement learning inherent in Open AI are the murders in Tumbler Ridge, British Columbia. A distraught trans teenager engaged in an exchange with ChatGP that caused the tech team enough concern they banned her from the app and they told upper management who, worried more about company liability than personal threats, did nothing.
9 people died.
Thanks for this. It's a frequent topic of conversation amongst friends so I will repost and share widely.
I'm currently making tough choices in reducing my book collection. One easy decision -- keeping my copy of Risk.
This is excellent. But I don't know if I can handle the truth.
Not to get all sycophantic and stuff but: this piece is superb! I've found that prompt-writing is an art in itself. My approach is to ask the machine to "have a go at" editing, synthesizing, or clarifying my text. I treat Claude like a really fine and fast editor. But I'm the expert. Is it paradoxical or nonsensical to say that although the machine is not conscious, my attitude in posing the prompt matters? I have no idea but as a philosophical pragmatist, I admire the design of the machine and I make use of its output whenever I find it helpful to the project at hand.
Very useful. For my political articles I often ask my AI..”criticize this from the point of view of Angela Rayner or Rishi Sunak”. This helps with selective evidence that I am prone to.