Eh, I'm quite skeptical of the conclusions here. Not that AI's can get to be as good or better as top human superforecasters - that I'll take as given. More like, that this is actually all that helpful.
I think the flaw is right here: "Everyone having an elite-tier forecasting department on call 24/7 is transformative stuff."
No. The bottleneck is almost always not the marginal accuracy of forecasts. That is, people don't make anywhere near optimal use of what they have now, so it doesn't help to make that particular factor somewhat better.
Consider Covid. It's not a secret that there was a probability of a pandemic over the years. Some people repeated warned of such an event, and had governments make some preparations. They were mocked and attacked as wasting money, since low-probability events *usually* don't happen. And when they were proven right, nobody cared. A more accurate points probability will not change this at all.
Basically, having highly refined probabilities usually just doesn't matter.
Real example: What's the probability that the US goes into civil war under Trump? Now tell me what I do differently based on whether it's 2.78% or 3.14%
But I think you’re missing something. Do you have any idea how bad most people are at estimating even basic facts, like “what percentage of the population is Muslim”? Now imagine how off-base they are likely to be with something like “will there be civil war?” By introducing a super-low cost way to ask questions — no money, almost no time almost no effort — AI effectively invites people to start asking lots more questions, like “what percentage of the population is Muslim,” rather than operating on the basis of assumptions. Be surprised a few times when you ask questions and you may realize the value in this. Or even start to enjoy it.
Or go the next step: You ask AI “will there be a civil war?” You get a number. It startles you, one way or another. What’s the natural next question? “How did you make that forecast? What are you considering?” Suddenly, this person is looking at the various factors that may be relevant to the outcome. Which may prompt more questions. Now what’s happening? They’re exploring the whole problem space. And learning like crazy. That’s not nothing.
I speak from personal experience here: I’ve spent my life trying hard to be the curious sort of person who asks questions, but the sheer ease of asking questions with AI has super-charged my own question-asking. My daily total is going through the roof. And I encounter surprises daily.
A lot of this has to do with one’s model of a human, I think. How common is curiosity? Can it be cultivated? How easily? If it were made free and easy to learn, would many people want to learn more? I tend to be pretty optimistic about these questions. I think AI will be a test of my model.
Perfect question. "How common is curiosity?" Not that common, I fear. I find many social conversations peter out just as the topic gets interesting. Some curious person asks a great question and people drift off. Intensity in human interactions seems to scare a lot of people... Anyway, I agree that being super- charged curious is more fun now!
Curiosity isn't a fixed quantity or trait, in one person or one billion. It can be promoted and cultivated. Or stamped out. I expect and hope that for a substantial portion of the population -- perhaps a minority, but still substantial -- AI's lowering of the cost of answers (in terms of time, effort, and money) will promote and cultivate it. I can attest to that myself: I am asking far more questions these days, learning more, and discovering happy surprises that take me in unexpected directions routinely. It's pretty amazing.
With great sorrow, I think I do have an idea how bad people are at basic facts. It was the death of my attempts at tech policy. But in specific, currently I often inveigh against that we have an anti-vax lunatic as Secretary Of Health and now MEASLES IS BACK! This is incontrovertible proof that accurate knowledge is not a limiting factor. It's absurd to think the actual percentage of the population being Muslim matters in a world of "In Springfield, they're eating the dogs. The people that came in. They're eating the cats.".
Note, you didn't answer my question about US civil war - what do I DO once I have the presumably-best-forecast number?
With respect, that sort of cynicism is not merely descriptive (and wrong in description). It is destructive. You are not simply making a forecast. You are contributing to a self-fulfilling prophecy. You worry that everyone but you is impervious to facts and reason? I worry that cynicism like yours will produce a downward spiral making the world look more like your belief that it would have otherwise.
And if you think I didn't answer your question, I suggest you read a little more carefully. (I would also note that you have deliberately chosen a question where the particular value attached isn't likely to be useful. But there is a vast array of questions where the opposite is likely to be true. You know that. Please acknowledge that reality is complex and don't go for whatever is most likely to score points in your contrarian (as always) argument.) In this case, yes, the particular number isn't likely to be useful for most people. But the asking and answering of the question can be of enormous value if it leads someone to ask "what goes into this forecast? What matters to the outcome? How do we judge those factors?" That's exploratory thinking. It's likely to lead, at a minimum, to learning. And in some situations, that learning may itself be useful. At worst, it's learning. And that's good.
As Dwight Eisenhower wrote: "I have found that in preparing for battle, plans are useless, but planning is indispensable." If you understand the paradox of that wise statement, you understand why you are wrong.
Strawman: "You worry that everyone but you is impervious to facts and reason?"
Where did I *ever* write anything like *EVERYONE BUT [ME]*? I'm hardly the only person who has gotten vaccinated. But again, that MEASLES IS BACK - that a disease which had been virtually wiped-out in the US, has been surging, is irrefutuable proof that way too many people are indeed impervious to facts and reason. We aren't talking about complicated politics. We're talking about decades and decades where children didn't die from getting a disease, and now they are dying in part because, once more, we have an anti-vax lunatic as Secretary Of Health. It's about as blunt and clear demonstration possible of the limits of facts and reason.
That's "global". For "local", I have earned my cynicism by much direct experience.
You've moved the goalposts greatly with "if it leads someone ...". I don't claim there is not a single person in a single context where good forecasting could help them. Rather, that it is a very minor and specialized application, and will do nearly nothing in terms of society overall. For the avowedly cynical reason that being right very rarely matters: "Worldly wisdom teaches that it is better for reputation to fail conventionally than to succeed unconventionally." - John Maynard Keynes
[Anti-strawman: "very rarely" != "never, none, there is not a single case"]
OK, Seth, not "everyone but you." You got me! Point scored. Well done, you.
An aside: Seth, you're a smart guy and you contribute here, and I appreciate that, but you are relentlessly oppositional. I don't think I've ever seen you agree with even some some portion of a comment, or modify modestly in light of, etc. If I may: That subtracts from the value of what you do, particularly the value to you. For example, here, you just went to town on me for what suited your opposition but ignored my central point. That's not helpful to you, me, or anyone.
On the substance: As I explained to someone else (I can't keep comments on these threads straight) much depends on our model of people. In my model, people are capable of exploring and learning, and capable of enjoying that process, and if the cost of exploring and learning falls -- in terms of time, effort, and money - they will do more of it. Not all. But in substantial numbers. And even if that's only a minority of the population, that absolutely has the potential to do good things at an aggregate level.
You seemingly have a much bleaker model of people. Maybe you're right. But I worry that cynicism is not only descriptive but destructive. We see that in politics: The sort of cynicism that leads people to say things like "all politicians are liars" actually contributes to the breakdowns that make it more likely that all politicians will become liars.
Regarding "I don't think I've ever seen you agree with even some portion of a comment", I'll vigorously agree here with the portion of your comment that "You seemingly have a much bleaker model of people.". That's completely true 1/2 :-).
I do have very strong views on certain topics, and I'll defend them. I'm not contrarian for contrarian's sake. But if I think an argument is wrong, indeed I don't make accommodating statements, and I know that's a cultural difference. I'll never be much good at an agreeable communication style, it's beyond me, regrets. It's a flaw I freely admit.
I didn't ignore your central point, I understood it. I disagree with it, about amount. My view is that such effects are pretty small overall across all society (*not* absolute zero, but e.g. easily swamped by the devastation the Trump administration has done to US science).
We can't will the world into what we want it to be. If politicians have overwhelming incentives to be liars, deciding we won't say this because it's bad, won't change those overwhelming incentives. In fact, I'd contend making it socially disfavored to say that, provides a very strong weapon for the liars to use, as e.g. they can then turn around and use it against political reform efforts ("My opponents say billionaires handing me big bags of money undermines public confidence - but to claim that influences my vote is destructive cynicism, so they are the ones who are the real corrupters of public trust!")
Can you clarify what you're asking as to "Why?"? The topic of how likely is Trump to literally plunge the US into civil war, has been on my mind lately. I don't think it's real high - but I also don't think it's insignificant either. Now, if I have an "elite-tier forecasting department", I'd ask for the estimate here. Obviously it can't be straight "yes" or "no", it's some percentage (there's obviously a significant chance Trump will be unable to serve out his term). Now what?
I think it's worse than you are letting on. The people who predicted COVID were roundly ignored for many years, and when it turned out that they had been right all along, they were accused of being the masterminds behind an evil plot.
Because to the conspiracy way of thinking, there is no way to know that something is likely to happen unless you make it happen yourself.
When computers first came out at the insurance statistics company my dad had worked for as a statistician/actuary for a decade, he was not phased nor jumping on the messianic or doom and gloom band wagons like his colleagues.
Dad said, "Keep your pencils sharp, people."
Back then, the new "business machines" called computers, took up the volume in a space the size of my current home's ground floor, made loud noises, and required a small army of data people, key-punch operators, even air conditioning and refrigeration techs, to name but a few.
My dear old dad took it in stride. He was not in awe of it all like his younger newbie co-workers. Nor was he lamenting the inevitable employment changes, job losses, re-structuring. Dad had been to war and survived. This was nothing.
He'd tell the young whippersnappers that the computer was only as good as the humans who made it, managed it, programmed and coded it, and asked the important questions for getting the sought after information, and then used it. In other words, keep your pencils sharp, people. ✏️ ❤️🇨🇦
This is a bit overboard I think. Applying machine learning to meta analysis and forecasting has been refined to the point of being pretty routine these days. You should look at the analogs of these techniques in sports gambling.
Wondering if anyone can point me to recent studies where AI was given only the information available at the time and asked to predict a known event in the future.
For example, whether an AI program using the available literature about anti-biotics from around 1900 could then accurately predict when the discovery penicillin would happen and how long it would take before it could be mass produced.
You could try what I tried: asking an AI agent (I used XGrok) to simulate what it would predict about penicillin's discovery and mass production assuming its knowledge database contained only knowledge available around 1900.
Its response began this way:
"Based on the simulated constraint of my training data being limited solely to literature on antimicrobials and related bacteriological concepts available around the year 1900, I must approach this prediction as if I have no knowledge of subsequent historical developments. This means drawing inferences from the state of science at that time, including the germ theory of disease (established by Pasteur and Koch in the 1870s–1880s), early work on antiseptics (e.g., Lister's use of carbolic acid for surgical disinfection), and emerging ideas in chemotherapy, such as Paul Ehrlich's ongoing research into selective toxic agents that could target pathogens without harming the host (his "magic bullet" concept, which was in its formative stages around 1900).At this point in history (circa 1900), there is no formalized concept of "antibiotics" as substances produced by microorganisms to inhibit others— that terminology and framework simply do not exist in the literature. Instead, the focus is on chemical antiseptics, dyes, and arsenic-based compounds for treating infections."
and it ended this way:
"In summary, from a 1900 perspective, the discovery of a penicillin-like antibiotic seems imminent within 10–20 years due to burgeoning microbiology, but mass production would lag by another 15–25 years owing to technological and infrastructural immaturity. This is, of course, a speculative extrapolation—actual progress would depend on unforeseen genius, funding, and perhaps lucky laboratory accidents."
But, is it realistic for me to ask you to predict when some innovation you haven't even conceived of will be discovered or invented? Would it be any easier to ask you a simpler question: what innovations will occur in the future, let alone when in the future?
Given that AI draws on existing knowledge, it stems to reason that, as it gets "trained" better, it will exceed the ability of even superforecasters to predict the future. I don't find this surprising and am not dizzy nor hyperventilating.
Dan, you chide the Atlantic article for not mentioning Philip Tetlock but you didn't mention the article's author!
The proposed end point of December 2026, as outlined by the graph, seems just another forecast. But the data points for the past year look flat, not changing. Strictly a visual impression.
It’s not an “end point.” It is the point where the lines cross. And yes, as I wrote, it is an extrapolation of the current trajectories, and thus would over- or under-shoot if those trajectories change for any reason.
The interesting shift isn’t just that answers get cheap, it’s that question-selection becomes the real bottleneck. Once everyone has elite forecasts, advantage moves upstream to who gets to decide which futures are even queried. That’s less about curiosity and more about institutional position, incentives, and legitimacy. In that sense, forecasting abundance doesn’t democratize foresight so much as it relocates power to agenda-setting.
I wouldn’t dismiss the importance of differentials of influence — Musk’s queries will reverberate vastly more than mine — but surely we should not also dismiss that literally anyone can ask literally anything. This is not an era of press barons.
I was delighted to see that you highlighted the problem of "reflexivity" (forecasts themselves contribute to what winds up happening) and that you mentioned that it's a common phenomenon. But then I was surprised by the characterization of it as an important but not critical caveat in what followed. If everyone has powerful forecasting tools, wouldn't reflexivity -- already underestimated -- have an even greater impact on our affairs?
Depends which end of the telescope you look in, I suppose. But I take your point: If we hive off reflexivity and look at it in light of LLMs, it sure seems like its prevalence will soar. And yeah, I can totally imagine that will create hairy new dynamics, or at least a greatly accelerated pace of such dynamics, particularly if the problem is not widely recognized (until something blows up good, natural). On its own, that could get scary, fast.
The idea we don't look at AI and see the limitations of prediction, particularly in terms of chaos.
Brain dead.
And the idea we don't look at it and see the bottleneck of symbols and language.
Terrifying.
These pieces all miss the point. Life does not survive through prediction, that's a model (models are always wrong). Life survives by tinkering and selection.
Show me that in AI, and it can't. It's not in there.
I'm pretty sure there are several long paragraphs about the severe limitations of prediction in the essay, including one making exactly the point you made.
And while I'm a big fan of the "tinkering and selection" approach, I think you overstate. A dragonfly predicts the trajectory of a mosquito to intercept it, which is rather important to the dragonfly's survival. Multiply that by millions (billions!) of similar illustrations and I'm pretty sure prediction plays a role in survival. (Because you left out the end of George E.P. Box's famous dictum: All models are wrong *but some are useful.*)
Dragonflies do not predict trajectories, we know this in neuroscience and coordination dynamics. They meet optic flows. The same principle is how outfielder's can rarely predict where a ball will fall when standing, yet in motion can catch a high percentage in motion. (All models are wrong, period—we're at this point now).
You're using what I describe as UFOlogy as science, when the sciences have a much deeper approach that refutes AI.
Notice you ignored the limitations of symbols and words as a bottleneck?
Look, I'm always open to the possibility that my layman's grasp is totally inadequate to the scientifically informed thoughts of correspondents. But if you want to overturn the fundamentals of my model of the world, you'll need to work at explaining yourself a little more. You can't just toss off a few phrases and declarations -- "all models are wrong, period" -- and expect the other person to experience a miraculous conversion.
It's predictive and reactive control. Even the intro admits this. That's the essence of optic flow. Prediction is only a model of a part of an entire picture. I'd read that paper carefully.
Criticality emerges only from reactive control. That is survival.
There are quite a few approaches that exceed the cog-sci idea that prediction is the overarching model.
Control Theory
Coordination Dynamics
Enactive
Embodied
Ecological
Chaos
And yes, you are at the threshold where we ditch symbols, words, causal statements and models. AI is demanding we do this, or succumb to its arbitrary, causal, predictive models (ie markets). That is survival, to exceed those limitations. Or extinction. What you are describing in your piece it the bottleneck of extinction, non-trivially, explicitly. You should grasp that.
Good question. Answer: AI is not conscious and self-aware. It does not perceive reality. Hence it does not self-initiate an inquiry. It only responds to our prompts.
Ehhh you're mixing up a couple of concepts here Dan. Consciousness isn't necessary for agency. Current models can be scaffolded to run continuously and ask questions. See AI Village and Moltbook. There's incredible interest in building AI agents, so it'll happen. Forecasting agents won't be that hard to build, and they'll be able to create a massive amount of questions (though some humans will be able to ask some questions better than AI. In fact, it will be very valuable to get those good humans to collect their questions somewhere public for the AIs to forecast!)
If a fully automated factory runs without human intervention, does it have agency? No. It's merely executing the tasks that have been designed for it by humans, and agency involves choosing one's own course of action.
Similarly, an automated system that asks and answers "what's the weather this morning?" doesn't have agency. It's merely responding to a question it was told to ask.
An AI that was, let's say, busily generating paper clips because it had been told to optimize paper clip output, wouldn't ask, "why is the sky blue?" because that has no possible relevance to paper clip optimization. It *may* ask lots of questions relevant to paper clip optimization, but not that.
And you're not truly free to explore without that self-initiation.
That said, maybe I'm wrong about what AI can do (or will soon be able to do). And things would get considerably greyer, fast, if you gave an AI an incredibly wide remit and the instructions to have at it.
This raises such mind-bending questions about what it means to keep a human in the loop or, better, what it means to keep humans in control. But such questions probably won't matter to what happens thanks to our billionaire tech overlords and their headlong rush, because, of course, their thoughts are the only thoughts that matter.
What if the prompt is "be a great forecaster and ask, research, and forecast questions relevant for the world. You have enough compute credits for continuous use. Keep going."? I'm confused why you think current AI can't already be pretty productive with this, let alone near-term AI? You can call it agency or not (I think it's a spectrum and current AI have low but not zero agency), but AI can ask good questions
I told Gemini to ask questions. It came back with the functional equivalent of "about what?" Which is a question, granted, but you have to admit that's pretty funny.
Look, it makes sense to me on my current understanding -- I'll be the first to acknowledge how little I know about what's under AI's hood -- that you can set parameters and goals and it will ask questions accordingly. It would seem AI is doing that in the forecasting tournaments, and doing it well. But it's the upstream act I'm getting at: Who initiates? People are capable of looking at the world and suddenly asking, "why is the sky blue?" apropos of nothing. Is AI? I don't think so.
As I said, I get that, functionally, giving an AI parameters and goals and telling it to do its thing will generate lots of questions and this is functionally little different than lots of what people do. But it is that upstream act -- the original prompt -- that is the big limitation here, the thing we do that AI doesn't, so far as a I know.
Incidentally, what's shaping my thinking here (and why I find this so interesting) is that I'm writing a book about the history of technology and the fallacy of technological determinism. It's too complicated to go into but the ideas are really interwoven here. And AI certainly challenges them at a really fundamental level. Strange days.
I don't think that asking Gemini is a good test because of the lack of scaffolding. And I'm not sure if you're claiming that near-term AI also won't have agency?
Eh, I'm quite skeptical of the conclusions here. Not that AI's can get to be as good or better as top human superforecasters - that I'll take as given. More like, that this is actually all that helpful.
I think the flaw is right here: "Everyone having an elite-tier forecasting department on call 24/7 is transformative stuff."
No. The bottleneck is almost always not the marginal accuracy of forecasts. That is, people don't make anywhere near optimal use of what they have now, so it doesn't help to make that particular factor somewhat better.
Consider Covid. It's not a secret that there was a probability of a pandemic over the years. Some people repeated warned of such an event, and had governments make some preparations. They were mocked and attacked as wasting money, since low-probability events *usually* don't happen. And when they were proven right, nobody cared. A more accurate points probability will not change this at all.
Basically, having highly refined probabilities usually just doesn't matter.
Real example: What's the probability that the US goes into civil war under Trump? Now tell me what I do differently based on whether it's 2.78% or 3.14%
I’ll grant you lots of the above.
But I think you’re missing something. Do you have any idea how bad most people are at estimating even basic facts, like “what percentage of the population is Muslim”? Now imagine how off-base they are likely to be with something like “will there be civil war?” By introducing a super-low cost way to ask questions — no money, almost no time almost no effort — AI effectively invites people to start asking lots more questions, like “what percentage of the population is Muslim,” rather than operating on the basis of assumptions. Be surprised a few times when you ask questions and you may realize the value in this. Or even start to enjoy it.
Or go the next step: You ask AI “will there be a civil war?” You get a number. It startles you, one way or another. What’s the natural next question? “How did you make that forecast? What are you considering?” Suddenly, this person is looking at the various factors that may be relevant to the outcome. Which may prompt more questions. Now what’s happening? They’re exploring the whole problem space. And learning like crazy. That’s not nothing.
I speak from personal experience here: I’ve spent my life trying hard to be the curious sort of person who asks questions, but the sheer ease of asking questions with AI has super-charged my own question-asking. My daily total is going through the roof. And I encounter surprises daily.
A lot of this has to do with one’s model of a human, I think. How common is curiosity? Can it be cultivated? How easily? If it were made free and easy to learn, would many people want to learn more? I tend to be pretty optimistic about these questions. I think AI will be a test of my model.
Perfect question. "How common is curiosity?" Not that common, I fear. I find many social conversations peter out just as the topic gets interesting. Some curious person asks a great question and people drift off. Intensity in human interactions seems to scare a lot of people... Anyway, I agree that being super- charged curious is more fun now!
Curiosity isn't a fixed quantity or trait, in one person or one billion. It can be promoted and cultivated. Or stamped out. I expect and hope that for a substantial portion of the population -- perhaps a minority, but still substantial -- AI's lowering of the cost of answers (in terms of time, effort, and money) will promote and cultivate it. I can attest to that myself: I am asking far more questions these days, learning more, and discovering happy surprises that take me in unexpected directions routinely. It's pretty amazing.
With great sorrow, I think I do have an idea how bad people are at basic facts. It was the death of my attempts at tech policy. But in specific, currently I often inveigh against that we have an anti-vax lunatic as Secretary Of Health and now MEASLES IS BACK! This is incontrovertible proof that accurate knowledge is not a limiting factor. It's absurd to think the actual percentage of the population being Muslim matters in a world of "In Springfield, they're eating the dogs. The people that came in. They're eating the cats.".
Note, you didn't answer my question about US civil war - what do I DO once I have the presumably-best-forecast number?
With respect, that sort of cynicism is not merely descriptive (and wrong in description). It is destructive. You are not simply making a forecast. You are contributing to a self-fulfilling prophecy. You worry that everyone but you is impervious to facts and reason? I worry that cynicism like yours will produce a downward spiral making the world look more like your belief that it would have otherwise.
And if you think I didn't answer your question, I suggest you read a little more carefully. (I would also note that you have deliberately chosen a question where the particular value attached isn't likely to be useful. But there is a vast array of questions where the opposite is likely to be true. You know that. Please acknowledge that reality is complex and don't go for whatever is most likely to score points in your contrarian (as always) argument.) In this case, yes, the particular number isn't likely to be useful for most people. But the asking and answering of the question can be of enormous value if it leads someone to ask "what goes into this forecast? What matters to the outcome? How do we judge those factors?" That's exploratory thinking. It's likely to lead, at a minimum, to learning. And in some situations, that learning may itself be useful. At worst, it's learning. And that's good.
As Dwight Eisenhower wrote: "I have found that in preparing for battle, plans are useless, but planning is indispensable." If you understand the paradox of that wise statement, you understand why you are wrong.
Strawman: "You worry that everyone but you is impervious to facts and reason?"
Where did I *ever* write anything like *EVERYONE BUT [ME]*? I'm hardly the only person who has gotten vaccinated. But again, that MEASLES IS BACK - that a disease which had been virtually wiped-out in the US, has been surging, is irrefutuable proof that way too many people are indeed impervious to facts and reason. We aren't talking about complicated politics. We're talking about decades and decades where children didn't die from getting a disease, and now they are dying in part because, once more, we have an anti-vax lunatic as Secretary Of Health. It's about as blunt and clear demonstration possible of the limits of facts and reason.
That's "global". For "local", I have earned my cynicism by much direct experience.
You've moved the goalposts greatly with "if it leads someone ...". I don't claim there is not a single person in a single context where good forecasting could help them. Rather, that it is a very minor and specialized application, and will do nearly nothing in terms of society overall. For the avowedly cynical reason that being right very rarely matters: "Worldly wisdom teaches that it is better for reputation to fail conventionally than to succeed unconventionally." - John Maynard Keynes
[Anti-strawman: "very rarely" != "never, none, there is not a single case"]
OK, Seth, not "everyone but you." You got me! Point scored. Well done, you.
An aside: Seth, you're a smart guy and you contribute here, and I appreciate that, but you are relentlessly oppositional. I don't think I've ever seen you agree with even some some portion of a comment, or modify modestly in light of, etc. If I may: That subtracts from the value of what you do, particularly the value to you. For example, here, you just went to town on me for what suited your opposition but ignored my central point. That's not helpful to you, me, or anyone.
On the substance: As I explained to someone else (I can't keep comments on these threads straight) much depends on our model of people. In my model, people are capable of exploring and learning, and capable of enjoying that process, and if the cost of exploring and learning falls -- in terms of time, effort, and money - they will do more of it. Not all. But in substantial numbers. And even if that's only a minority of the population, that absolutely has the potential to do good things at an aggregate level.
You seemingly have a much bleaker model of people. Maybe you're right. But I worry that cynicism is not only descriptive but destructive. We see that in politics: The sort of cynicism that leads people to say things like "all politicians are liars" actually contributes to the breakdowns that make it more likely that all politicians will become liars.
Regarding "I don't think I've ever seen you agree with even some portion of a comment", I'll vigorously agree here with the portion of your comment that "You seemingly have a much bleaker model of people.". That's completely true 1/2 :-).
I do have very strong views on certain topics, and I'll defend them. I'm not contrarian for contrarian's sake. But if I think an argument is wrong, indeed I don't make accommodating statements, and I know that's a cultural difference. I'll never be much good at an agreeable communication style, it's beyond me, regrets. It's a flaw I freely admit.
I didn't ignore your central point, I understood it. I disagree with it, about amount. My view is that such effects are pretty small overall across all society (*not* absolute zero, but e.g. easily swamped by the devastation the Trump administration has done to US science).
We can't will the world into what we want it to be. If politicians have overwhelming incentives to be liars, deciding we won't say this because it's bad, won't change those overwhelming incentives. In fact, I'd contend making it socially disfavored to say that, provides a very strong weapon for the liars to use, as e.g. they can then turn around and use it against political reform efforts ("My opponents say billionaires handing me big bags of money undermines public confidence - but to claim that influences my vote is destructive cynicism, so they are the ones who are the real corrupters of public trust!")
Maybe I'm being relentlessly oppositional again.
Why?
Can you clarify what you're asking as to "Why?"? The topic of how likely is Trump to literally plunge the US into civil war, has been on my mind lately. I don't think it's real high - but I also don't think it's insignificant either. Now, if I have an "elite-tier forecasting department", I'd ask for the estimate here. Obviously it can't be straight "yes" or "no", it's some percentage (there's obviously a significant chance Trump will be unable to serve out his term). Now what?
I think it's worse than you are letting on. The people who predicted COVID were roundly ignored for many years, and when it turned out that they had been right all along, they were accused of being the masterminds behind an evil plot.
Because to the conspiracy way of thinking, there is no way to know that something is likely to happen unless you make it happen yourself.
When computers first came out at the insurance statistics company my dad had worked for as a statistician/actuary for a decade, he was not phased nor jumping on the messianic or doom and gloom band wagons like his colleagues.
Dad said, "Keep your pencils sharp, people."
Back then, the new "business machines" called computers, took up the volume in a space the size of my current home's ground floor, made loud noises, and required a small army of data people, key-punch operators, even air conditioning and refrigeration techs, to name but a few.
My dear old dad took it in stride. He was not in awe of it all like his younger newbie co-workers. Nor was he lamenting the inevitable employment changes, job losses, re-structuring. Dad had been to war and survived. This was nothing.
He'd tell the young whippersnappers that the computer was only as good as the humans who made it, managed it, programmed and coded it, and asked the important questions for getting the sought after information, and then used it. In other words, keep your pencils sharp, people. ✏️ ❤️🇨🇦
This is a bit overboard I think. Applying machine learning to meta analysis and forecasting has been refined to the point of being pretty routine these days. You should look at the analogs of these techniques in sports gambling.
Sorry, how does that translate into "a bit overboard"?
Thought provoking as always, Dan. Thank you.
Wondering if anyone can point me to recent studies where AI was given only the information available at the time and asked to predict a known event in the future.
For example, whether an AI program using the available literature about anti-biotics from around 1900 could then accurately predict when the discovery penicillin would happen and how long it would take before it could be mass produced.
You could try what I tried: asking an AI agent (I used XGrok) to simulate what it would predict about penicillin's discovery and mass production assuming its knowledge database contained only knowledge available around 1900.
Its response began this way:
"Based on the simulated constraint of my training data being limited solely to literature on antimicrobials and related bacteriological concepts available around the year 1900, I must approach this prediction as if I have no knowledge of subsequent historical developments. This means drawing inferences from the state of science at that time, including the germ theory of disease (established by Pasteur and Koch in the 1870s–1880s), early work on antiseptics (e.g., Lister's use of carbolic acid for surgical disinfection), and emerging ideas in chemotherapy, such as Paul Ehrlich's ongoing research into selective toxic agents that could target pathogens without harming the host (his "magic bullet" concept, which was in its formative stages around 1900).At this point in history (circa 1900), there is no formalized concept of "antibiotics" as substances produced by microorganisms to inhibit others— that terminology and framework simply do not exist in the literature. Instead, the focus is on chemical antiseptics, dyes, and arsenic-based compounds for treating infections."
and it ended this way:
"In summary, from a 1900 perspective, the discovery of a penicillin-like antibiotic seems imminent within 10–20 years due to burgeoning microbiology, but mass production would lag by another 15–25 years owing to technological and infrastructural immaturity. This is, of course, a speculative extrapolation—actual progress would depend on unforeseen genius, funding, and perhaps lucky laboratory accidents."
But, is it realistic for me to ask you to predict when some innovation you haven't even conceived of will be discovered or invented? Would it be any easier to ask you a simpler question: what innovations will occur in the future, let alone when in the future?
Interesting-thank you. I assume the final paragraph is your comment rather than XGrok's?
Given that AI draws on existing knowledge, it stems to reason that, as it gets "trained" better, it will exceed the ability of even superforecasters to predict the future. I don't find this surprising and am not dizzy nor hyperventilating.
Dan, you chide the Atlantic article for not mentioning Philip Tetlock but you didn't mention the article's author!
The proposed end point of December 2026, as outlined by the graph, seems just another forecast. But the data points for the past year look flat, not changing. Strictly a visual impression.
It’s not an “end point.” It is the point where the lines cross. And yes, as I wrote, it is an extrapolation of the current trajectories, and thus would over- or under-shoot if those trajectories change for any reason.
The interesting shift isn’t just that answers get cheap, it’s that question-selection becomes the real bottleneck. Once everyone has elite forecasts, advantage moves upstream to who gets to decide which futures are even queried. That’s less about curiosity and more about institutional position, incentives, and legitimacy. In that sense, forecasting abundance doesn’t democratize foresight so much as it relocates power to agenda-setting.
I wouldn’t dismiss the importance of differentials of influence — Musk’s queries will reverberate vastly more than mine — but surely we should not also dismiss that literally anyone can ask literally anything. This is not an era of press barons.
I was delighted to see that you highlighted the problem of "reflexivity" (forecasts themselves contribute to what winds up happening) and that you mentioned that it's a common phenomenon. But then I was surprised by the characterization of it as an important but not critical caveat in what followed. If everyone has powerful forecasting tools, wouldn't reflexivity -- already underestimated -- have an even greater impact on our affairs?
Depends which end of the telescope you look in, I suppose. But I take your point: If we hive off reflexivity and look at it in light of LLMs, it sure seems like its prevalence will soar. And yeah, I can totally imagine that will create hairy new dynamics, or at least a greatly accelerated pace of such dynamics, particularly if the problem is not widely recognized (until something blows up good, natural). On its own, that could get scary, fast.
Organizing Ignorance For Discovery is an old McLuhan approach !
I hope it belongs to an actual four-year-old, one with a PostgreSQL database and a trading account.
The idea we don't look at AI and see the limitations of prediction, particularly in terms of chaos.
Brain dead.
And the idea we don't look at it and see the bottleneck of symbols and language.
Terrifying.
These pieces all miss the point. Life does not survive through prediction, that's a model (models are always wrong). Life survives by tinkering and selection.
Show me that in AI, and it can't. It's not in there.
I'm pretty sure there are several long paragraphs about the severe limitations of prediction in the essay, including one making exactly the point you made.
And while I'm a big fan of the "tinkering and selection" approach, I think you overstate. A dragonfly predicts the trajectory of a mosquito to intercept it, which is rather important to the dragonfly's survival. Multiply that by millions (billions!) of similar illustrations and I'm pretty sure prediction plays a role in survival. (Because you left out the end of George E.P. Box's famous dictum: All models are wrong *but some are useful.*)
The limitations you mentioned are not inquisitive.
We're talking about an overarching idea about consciousness, not prediction markets.
https://pubmed.ncbi.nlm.nih.gov/38579270/
Dragonflies do not predict trajectories, we know this in neuroscience and coordination dynamics. They meet optic flows. The same principle is how outfielder's can rarely predict where a ball will fall when standing, yet in motion can catch a high percentage in motion. (All models are wrong, period—we're at this point now).
You're using what I describe as UFOlogy as science, when the sciences have a much deeper approach that refutes AI.
Notice you ignored the limitations of symbols and words as a bottleneck?
Well, alert the scientists that they're wrong: https://scienceeducation.fas.harvard.edu/event/predictive-control-internal-models-and-neural-circuitry-underlying-dragonfly
And "all models are wrong, period." OK, then.
Look, I'm always open to the possibility that my layman's grasp is totally inadequate to the scientifically informed thoughts of correspondents. But if you want to overturn the fundamentals of my model of the world, you'll need to work at explaining yourself a little more. You can't just toss off a few phrases and declarations -- "all models are wrong, period" -- and expect the other person to experience a miraculous conversion.
It's predictive and reactive control. Even the intro admits this. That's the essence of optic flow. Prediction is only a model of a part of an entire picture. I'd read that paper carefully.
Criticality emerges only from reactive control. That is survival.
There are quite a few approaches that exceed the cog-sci idea that prediction is the overarching model.
Control Theory
Coordination Dynamics
Enactive
Embodied
Ecological
Chaos
And yes, you are at the threshold where we ditch symbols, words, causal statements and models. AI is demanding we do this, or succumb to its arbitrary, causal, predictive models (ie markets). That is survival, to exceed those limitations. Or extinction. What you are describing in your piece it the bottleneck of extinction, non-trivially, explicitly. You should grasp that.
https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer
This is going to get meta but...
I asked Gemini to explain your comments. It turned what was bewildering into something quite fascinating.
May I make a suggestion? Put a little effort into how to explain your thoughts to a layperson. Or ask AI to translate for you.
I don't train models and don't require translation. Thankfully my job let's me communicate ideas at this level.
Too bad you didn't do the work yourself, you might have had a real insight or a eureka, instead you exported it to an extinction machine.
I was actually thinking about this. But what if AI can provide the questions too?
Fascinating and thought-provoking. We do indeed live in interesting times.
Trump thinks like one
What is stopping an advanced AI asking the right questions too?
Good question. Answer: AI is not conscious and self-aware. It does not perceive reality. Hence it does not self-initiate an inquiry. It only responds to our prompts.
Ehhh you're mixing up a couple of concepts here Dan. Consciousness isn't necessary for agency. Current models can be scaffolded to run continuously and ask questions. See AI Village and Moltbook. There's incredible interest in building AI agents, so it'll happen. Forecasting agents won't be that hard to build, and they'll be able to create a massive amount of questions (though some humans will be able to ask some questions better than AI. In fact, it will be very valuable to get those good humans to collect their questions somewhere public for the AIs to forecast!)
Here's my thinking:
If a fully automated factory runs without human intervention, does it have agency? No. It's merely executing the tasks that have been designed for it by humans, and agency involves choosing one's own course of action.
Similarly, an automated system that asks and answers "what's the weather this morning?" doesn't have agency. It's merely responding to a question it was told to ask.
An AI that was, let's say, busily generating paper clips because it had been told to optimize paper clip output, wouldn't ask, "why is the sky blue?" because that has no possible relevance to paper clip optimization. It *may* ask lots of questions relevant to paper clip optimization, but not that.
And you're not truly free to explore without that self-initiation.
That said, maybe I'm wrong about what AI can do (or will soon be able to do). And things would get considerably greyer, fast, if you gave an AI an incredibly wide remit and the instructions to have at it.
This raises such mind-bending questions about what it means to keep a human in the loop or, better, what it means to keep humans in control. But such questions probably won't matter to what happens thanks to our billionaire tech overlords and their headlong rush, because, of course, their thoughts are the only thoughts that matter.
What if the prompt is "be a great forecaster and ask, research, and forecast questions relevant for the world. You have enough compute credits for continuous use. Keep going."? I'm confused why you think current AI can't already be pretty productive with this, let alone near-term AI? You can call it agency or not (I think it's a spectrum and current AI have low but not zero agency), but AI can ask good questions
I told Gemini to ask questions. It came back with the functional equivalent of "about what?" Which is a question, granted, but you have to admit that's pretty funny.
Look, it makes sense to me on my current understanding -- I'll be the first to acknowledge how little I know about what's under AI's hood -- that you can set parameters and goals and it will ask questions accordingly. It would seem AI is doing that in the forecasting tournaments, and doing it well. But it's the upstream act I'm getting at: Who initiates? People are capable of looking at the world and suddenly asking, "why is the sky blue?" apropos of nothing. Is AI? I don't think so.
As I said, I get that, functionally, giving an AI parameters and goals and telling it to do its thing will generate lots of questions and this is functionally little different than lots of what people do. But it is that upstream act -- the original prompt -- that is the big limitation here, the thing we do that AI doesn't, so far as a I know.
Incidentally, what's shaping my thinking here (and why I find this so interesting) is that I'm writing a book about the history of technology and the fallacy of technological determinism. It's too complicated to go into but the ideas are really interwoven here. And AI certainly challenges them at a really fundamental level. Strange days.
You'll probably like this on the determinism btw! Matthijs is a strong thinker
https://open.substack.com/pub/criticalmaas/p/the-future-of-ai-is-not-yet-written?utm_source=share&utm_medium=android&r=6rc6a
I don't think that asking Gemini is a good test because of the lack of scaffolding. And I'm not sure if you're claiming that near-term AI also won't have agency?
I'm definitely dizzy. What an interesting article. And great comments. Thank you!