Concise Summary简洁概述
Yudkowsky argues that becoming a true rationalist requires breaking a deep emotional trust — first in family and community, then, crucially, in Science itself. Science, born before probability theory and cognitive-bias research, asks too little: it blesses good intentions without demanding genuine rigor. Bayesian reasoning is harder and offers no safety net either. The uncomfortable truth is that there is no trustworthy procedural defense, no checklist whose completion makes you safe. Real growth as a rationalist only begins after you have followed all the rules and still failed badly — and kept going anyway.
Yudkowsky 认为，成为真正的理性主义者需要打破一种深层的情感信任——先是对家人与社群，然后最关键的，是对科学本身。科学诞生于概率论和认知偏差研究出现之前，它对你的要求太低：它轻易地为好意点赞，却不要求真正的严格。贝叶斯推理更难，同样不提供安全网。令人不安的真相是：没有任何可信赖的程序性防线，没有任何一张清单，完成它就能让你安全。真正的理性主义成长，只有在你遵守了所有规则却依然惨败——并仍然继续前行——之后，才会开始。
Infographic信息图
Breaking the pack's hold
打破群体的束缚
Genuine rationality requires a deep emotional break with surrounding norms — lonely dissent that feels like wearing a clown suit, not cool rebellion.
真正的理性需要与周围规范做出深层情感上的决裂——那种孤独异见的感觉像是穿着小丑装去上学，而非酷炫的反叛。
Science demands too little
科学要求得太少
Science was born before probability theory and bias research; it blesses good intentions, accepts slowness, and cannot protect you from deep error.
科学诞生于概率论和偏差研究之前；它轻易为好意点赞，接受缓慢，无法保护你免于深层错误。
Bayescraft offers no safety either
贝叶斯之道也不提供安全
Bayesian reasoning is harder to apply, lacks textbooks and mentors, and leaves you perpetually unsure whether you've corrected for biases or overcorrected.
贝叶斯推理更难运用，缺乏教材和导师，让你始终无法确定自己是修正了偏差还是过度修正。
Failure, not words, breaks the spell
失败而非言辞才能破除魔咒
No argument can truly shatter complacency; only personally following all the rules and still failing badly makes the danger viscerally real.
没有任何论证能真正打碎自满；只有亲身遵守所有规则却依然惨败，才能让危险变得切肤真实。
The PS flaw is intentional
后记中的谬误是刻意的
Yudkowsky deliberately leaves a reasoning flaw in the essay — practicing what he preaches about no safe defense, not even in this very text.
Yudkowsky 刻意在文章里留下一处推理谬误——以此身体力行地示范：没有安全防线，就连这篇文章本身也不例外。
Detailed Summary详细概述
The Broken Trust That Makes Rationalists
Yudkowsky opens with a sociological observation: aspiring rationalists disproportionately report childhoods in which their trust in the sanity of surrounding people was forcibly broken — cults, clinically insane parents, or in his own case, an Orthodox Jewish upbringing. The connection he draws is psychological: until that core emotional trust breaks, you never truly start growing as a rationalist. You won't zig when others zag if the world still seems sane to you.
He calls it Lonely Dissent: true dissent doesn't feel like cool black clothes; it feels like wearing a clown suit. That's the discomfort required to say the unpopular true thing while everyone else performs individuality from a shoe commercial.
Eliezer~18~ and the Master Ideal
Yudkowsky then recounts his own experience with a second, subtler form of trust: trust in Science itself. Eliezer~18~ had already made the break from family and teachers. He had books — and he trusted them implicitly. He had internalized the ideal that the experimental method shall decide which hypothesis wins. He even dutifully acknowledged Science's social flaws: academia's slowness, misallocation, favoritism.
But this acknowledgment was a trap. Criticizing people who fail to live up to an ideal leaves the ideal itself intact. And Eliezer~18~ had a specific psychological block: an aversion to ideas that "resembled things idiots had said." Everyone who questioned the ideal of Science itself was selling snake oil or hiding from criticism. So Science remained the untouchable master idea.
The result: when he produced a stupid theory and made Novel Predictions for it, he felt safe — the way a child who has obeyed all the rules trusts their parent.
The Shattering
When Eliezer~23~ finally recognized how stupid the theory had been, the insight was not that Science had opposed him — Science had been perfectly fine with him wasting a decade, so long as he eventually admitted the results. Science demands too little:
- It was born before probability theory and cognitive-bias research.
- It only makes injunctions the average scientist can follow.
- It accepts slowness as a fact of life.
- It blesses good intentions without guaranteeing sound reasoning.
No Safe Defense — Including Bayescraft
The essay's central claim: there is no known procedure that makes your reasoning defensible. Not Science. Not Bayesianism either. Bayes is harder, younger, textbook-free, mentor-free, and forces you to confront that you don't know your own priors, can't apply the formal math, and may be overcorrecting for biases you've been warned about.
Paradoxically, Bayesian math is more frightening precisely because its difficulty is visible — you know you might be doing it wrong. Science's vagueness ("ideas should be tested by experiment") is comforting in a dangerous way.
What Actually Breaks the Trust
Yudkowsky is bracingly honest: his words alone will not suffice. The emotional belief that failure can't happen in real life if you follow the rules can only be broken by actually following the rules and failing badly. Words can convey the concept; only disaster produces the gut-level conviction.
The essay ends with a vision of readiness: only after your parents have failed you, your gods are dead, and your tools have shattered in your hand — and you still have something worth protecting that forces you to keep going — are you truly ready to take sole responsibility and forge a higher Art.
Post Scriptum: Yudkowsky notes he discovered an inexcusable flaw in this very essay and deliberately left it in — a final demonstration that no defense, not even this text, is safe.
造就理性主义者的信任破碎
Yudkowsky 以一个社会学观察开场：有志于理性主义的人，在成长经历中不成比例地报告了这样的童年——他们对周围人之理智的信任被强制打破：邪教、临床精神失常的父母，或如他自己的情况，正统犹太家庭的成长经历。他所揭示的联系是心理学层面的：在那种核心的情感信任被打破之前，你从未真正开始成长为一名理性主义者。如果世界在你眼中仍然理智有序，你就不会在别人走之字形时走之字形的反方向。
他称之为孤独异见：真正的异见并不像穿着酷炫的黑衣上学；它更像是穿着小丑装去上学。这正是说出那些不受欢迎的真话所需要承受的不适——彼时周围所有人都在依照运动鞋广告表演「做自己」。
叶利泽~18~与至高理想
随后，Yudkowsky 讲述了他自己与另一种更隐蔽的信任相遇的经历：对科学本身的信任。叶利泽~18~已经完成了与家人和老师的决裂。他有书——他对书怀有毫无保留的信任。他内化了这一理想：实验方法将裁定哪个假说胜出。 他甚至尽职尽责地承认了科学的社会性缺陷：学术界的迟缓、资源错配、任人唯亲。
但这种承认本身是一个陷阱。批评未能活出理想的人，理想本身安然无恙。而叶利泽~18~有一个特定的心理障碍：对「与蠢人所说之话相似的观念」怀有本能的厌恶。所有质疑科学这一理想本身的人，都是在兜售江湖骗术，或是在庇护自己最偏爱的某种愚蠢不受批评。于是科学始终是那个不可触碰的至高元观念。
结果是：当他提出一个愚蠢的理论并为之做出新奇预测时，他感到了安全——就像一个遵守了所有规则的孩子信任父母那样。
破碎
当叶利泽~23~最终认识到那个理论究竟有多愚蠢时，顿悟并非科学曾经反对过他——科学对他浪费十年毫无异议，只要他最终承认结果是错的就好。科学要求得太少：
- 它诞生于概率论和认知偏差研究出现之前。
- 它只制定普通科学家能够遵守的准则。
- 它把迟缓当作生活的既成事实而加以接受。
- 它为好意背书，却不保证推理的健全。
没有安全防线——贝叶斯之道也不例外
文章的核心主张：没有任何已知的程序能让你的推理无懈可击。 科学不行。贝叶斯主义也不行。贝叶斯更难、更年轻、没有教材、没有导师，而且迫使你直面这一事实：你不知道自己的先验是什么，无法运用正式的数学，而且可能在修正那些你曾被警告过的偏差时矫枉过正。
悖论之处在于，贝叶斯数学更令人恐惧，恰恰因为它的难度是显眼的——你知道自己可能在用错。而科学的模糊性（「观念应当接受实验检验」）是以一种危险的方式令人安慰的。
究竟是什么打破了信任
Yudkowsky 清醒而坦率：他的文字本身不够。「遵守规则就不会在现实生活中失败」这种情感信念，只能靠真正遵守规则后惨败来打破。文字能传递概念；只有灾难才能产生那种发自内脏的确信。
文章以一幅关于准备就绪的图景作结：只有在你的父母让你失望、你的神灵已死、你的工具在手中破碎之后——而且你仍然拥有某样值得保护的东西迫使你继续前行——你才真正准备好独自承担责任，去锻造一种更高的技艺。
后记： Yudkowsky 指出，他在这篇文章中发现了一处无可辩解的推理谬误，并刻意将其保留——这是最后的示范：没有任何防线是安全的，就连这篇文章本身也不例外。
FAQ常见问答
Why does becoming a rationalist require breaking emotional trust, not just learning logic?为什么成为理性主义者需要打破情感信任，而不只是学习逻辑？
Logic alone doesn't cost you anything socially. Real rationality sometimes requires conclusions that make you look strange to others — accepting cryonics, many-worlds, or betting on prediction markets when everyone around you ignores them. That move only becomes possible once you no longer need the surrounding community's sanity as an anchor. The break is emotional first, intellectual second.
单纯的逻辑在社会层面不需要你付出任何代价。真正的理性有时会要求你得出一些让你在旁人眼中显得怪异的结论——比如接受人体冷冻、多世界诠释，或者在所有人都无视预测市场时去那里下注。这个动作只有在你不再需要把周围社群的理智当作锚点之后，才会变得可能。这个决裂首先是情感层面的，其次才是智识层面的。
What exactly was wrong with Eliezer~18~'s trust in Science?叶利泽~18~对科学的信任究竟错在哪里？
He treated Science as a meta-ideal immune from the same scrutiny applied to other ideas — partly because anyone questioning Science seemed to be on the "Dark Side." The result was a false sense of safety: if he made a Novel Prediction and planned to test it, he felt he had done everything required. But Science does not demand you reason well — only that you eventually publish results. It blesses procedure, not insight.
他把科学当成一个元理想，将其豁免于施加在其他观念上的同等审视——部分原因是，任何质疑科学的人在他看来都站在「黑暗一边」。结果是一种虚假的安全感：只要他做出了新奇预测并计划加以检验，他就觉得自己已经尽到了所有义务。但科学不要求你推理得好——只要求你最终发表结果。它为程序背书，而非为洞察力背书。
Is Yudkowsky saying we shouldn't use the scientific method?Yudkowsky 是在说我们不该使用科学方法吗？
No. He's saying that following the scientific method is a necessary but not sufficient condition for good reasoning. Science sets a floor, not a ceiling. It's a baseline that prevents some errors but permits many others — and it was designed for an average scientist in a slow-moving community, not for someone trying to reason optimally under uncertainty.
不是。他的意思是，遵循科学方法是良好推理的必要条件，但不是充分条件。科学设定了一个下限，而非上限。它是一条基准线，能防止某些错误，但允许许多其他错误的发生——而且它是为在缓慢移动的共同体中工作的普通科学家而设计的，而非为试图在不确定条件下进行最优推理的人而设计的。
Why does Yudkowsky say Bayesian reasoning doesn't save you either?为什么 Yudkowsky 说贝叶斯推理也救不了你？
Bayesian reasoning is the more rigorous framework, but it's younger, lacks practical guides and mentors, can't be fully applied formally in most real cases, and leaves you perpetually unsure whether you've identified your priors correctly or corrected for biases appropriately. Crucially, its difficulty is visible — which is actually better, because you know you might be wrong, rather than resting in false comfort.
贝叶斯推理是更严格的框架，但它更年轻，缺乏实用指南和导师，在大多数真实情况下无法完全形式化地加以运用，而且让你始终无法确定自己是否正确识别了先验，或是否适当地修正了偏差。关键是，它的难度是显眼的——这其实更好，因为你知道自己可能出错，而不是停留在虚假的安慰中。
What is the significance of the deliberate flaw in the Post Scriptum?后记中那处刻意留下的谬误有何意义？
It is a performative demonstration of the essay's thesis. By leaving a known error in the text, Yudkowsky shows — rather than just argues — that no author, no method, no set of good intentions produces flawless reasoning. If you read the essay as guaranteeing safety, the PS corrects you. It also warns against the opposite failure: hunting too hard for flaws and finding fake ones to reassure yourself of your own critical prowess.
这是对文章主旨的一次表演性示范。通过在文中保留一处已知的错误，Yudkowsky 展示——而非仅仅论证——没有任何作者、任何方法、任何好意能产生无懈可击的推理。如果你把这篇文章当成安全保障来读，后记会纠正你。它同时也警告了反向的失败：过于用力地寻找谬误，并找到并不真实的谬误来安慰自己有多批判性。
What does "something to protect" mean at the end of the essay?文章末尾「值得保护的东西」是什么意思？
It's a reference to another essay (Something to Protect) and means a concrete goal — an outcome in the real world you genuinely care about — that prevents you from retreating into the comfortable resignation of "rationality has limits." The motivation must be strong enough that you cannot quit even after every defense has failed you.
这是对另一篇文章（值得保护的东西）的引用，指的是一个具体的目标——一个你真正在乎的现实世界中的结果——它能阻止你退守到「理性有其局限」这种舒适的放弃之中。这个动力必须足够强大，让你在每一道防线都使你失望之后，仍然无法退出。
In-depth Analysis · Pros & Cons深入解读 · 优缺点
This essay occupies a pivotal position in the Sequences: it dismantles the reader's last comfortable refuge — Science — and insists that no procedural ruleset provides genuine epistemic safety. Written with unusual personal candor, it draws on Yudkowsky's own intellectual history to argue that the crucial break is emotional before it is cognitive.
这篇文章在「序列」中占据一个枢纽位置：它拆除了读者最后一个舒适的避难所——科学——并坚持认为没有任何程序性规则集能提供真正的认识论安全。这篇文章以罕见的个人坦诚写就，借助 Yudkowsky 自身的智识历史来论证：关键的决裂在认知层面到来之前，首先是情感层面的。
- Unflinching honesty about the author's own failure对作者自身失败的毫不掩饰的诚实Using his own decade wasted on a stupid theory — rather than a generic example — gives the argument uncommon credibility and models the very vulnerability it preaches.以自己在愚蠢理论上浪费的十年为例——而非泛泛而谈的例子——赋予了这一论证罕见的可信度，并以身作则地示范了文章所宣扬的那种脆弱坦诚。
- The performative PS flaw表演性的后记谬误Deliberately embedding a known error is a rhetorically brilliant move: it converts an abstract claim ('no defense is safe') into a live demonstration within the text itself.刻意嵌入一处已知错误是一个修辞上的精妙举动：它把一个抽象主张（「没有任何防线是安全的」）转化为文本内部的一次实时示范。
- Honest about the limits of words对文字局限性的诚实Explicitly warning that the essay's argument cannot itself break the reader's trust — only personal disaster can — shows unusual epistemic integrity and sets appropriate expectations.明确警告文章的论证本身无法打破读者的信任——只有亲身经历的灾难才能——展示了罕见的认识论诚实，并设定了恰当的期望。
- The loneliness of genuine dissent is vividly rendered对真正异见之孤独的生动描绘'Wearing a clown suit' rather than 'cool black clothes' captures something psychologically real about what it costs to hold and voice genuinely unpopular views.「穿着小丑装」而非「酷炫的黑衣」捕捉到了一种心理上真实的东西——坚持并表达真正不受欢迎的观点需要付出的代价。
- The childhood-trauma theory of rationalists is speculative and unexamined关于理性主义者童年创伤的理论是推测性的、未经检验的The opening claim that the best rationalists have suffered broken trust in childhood rests entirely on self-selected anecdotes from Yudkowsky's social circle. No attempt is made to ask how many traumatized people failed to become good reasoners, or how many excellent rationalists had stable, trusting childhoods.开篇关于最优秀的理性主义者都经历过童年中信任破碎的说法，完全建立在来自 Yudkowsky 社交圈的自我选择轶事上。没有任何尝试去追问，有多少经历了信任破碎的人未能成为优秀的推理者，或有多少优秀的理性主义者拥有稳定而信任充足的童年。
- Conflates 'Science as institution' with 'the ideal of empirical testing'混淆了「作为制度的科学」与「实证检验的理想」The critique that Science is too lenient applies to the scientific community and its published norms. But the core epistemic norm — update beliefs in proportion to evidence — is not obviously flawed in the same way. Yudkowsky slides between these two targets without always distinguishing them.关于科学过于宽松的批评，适用于科学共同体及其发表的规范。但核心的认识论规范——按比例于证据更新信念——并不以同样的方式明显有缺陷。Yudkowsky 在这两个目标之间滑动，并不总是加以区分。
- The proposed alternative is underdeveloped所提出的替代方案未经充分发展The essay is far more articulate about what fails than about what should replace it. 'Forge a higher Art' and 'take sole responsibility' are stirring phrases but thin prescriptions. Readers left without a procedural foothold may slide from healthy skepticism into paralysis.文章对什么会失败的阐述，远比对什么应该取而代之的阐述更为清晰。「锻造更高的技艺」和「独自承担责任」是激动人心的短语，但作为处方却薄弱。没有程序性立足点的读者可能会从健康的怀疑主义滑向瘫痪。
- The argument against Bayes proves too much对贝叶斯的批评证明过度The argument that Bayes is hard to apply and leaves you unsure of your priors is true, but the same logic could dismiss any tool — including careful thinking itself. If perpetual uncertainty disqualifies Bayes as a defense, it is unclear what would constitute a legitimate epistemic tool at all.贝叶斯难以运用、让你对自己的先验不确定的论证是真实的，但同样的逻辑可以驳斥任何工具——包括谨慎的思考本身。如果永久的不确定性使贝叶斯失去资格，那么什么样的东西才能构成一个合法的认识论工具，这一点并不清楚。
A powerful and honest dismantling of epistemic complacency, strongest in its refusal to let Science serve as a comfortable authority. Its chief limitation is asymmetry: the essay's destructive clarity outpaces its constructive guidance, leaving motivated readers with broken tools and no clear next step beyond 'fail harder and keep going.' Best read as a necessary clearing — the ground that must be leveled before the Sequences' more constructive work can be built.
一篇对认识论自满的有力而诚实的拆解，最强之处在于拒绝让科学充当舒适的权威。它的主要局限在于不对称：文章的破坏性清晰度超越了其建设性的指引，让有动力的读者面对破碎的工具，却没有超越「更彻底地失败然后继续前行」的清晰下一步。最好把它当作一片必要的清理——在「序列」更具建设性的工作得以建立之前，必须夷平的地基。
Original Text原文
I don't ask my friends about their childhoods—I lack social curiosity—and so I don't know how much of a trend this really is:
Of the people I know who are reaching upward as rationalists, who volunteer information about their childhoods, there is a surprising tendency to hear things like: "My family joined a cult and I had to break out," or "One of my parents was clinically insane and I had to learn to filter out reality from their madness."
My own experience with growing up in an Orthodox Jewish family seems tame by comparison... but it accomplished the same outcome: It broke my core emotional trust in the sanity of the people around me.
Until this core emotional trust is broken, you don't start growing as a rationalist. I have trouble putting into words why this is so. Maybe any unusual skills you acquire—anything that makes you unusually rational—requires you to zig when other people zag. Maybe that's just too scary, if the world still seems like a sane place unto you.
Or maybe you don't bother putting in the hard work to be extra bonus sane, if normality doesn't scare the hell out of you.
I know that many aspiring rationalists seem to run into roadblocks around things like cryonics or many-worlds. Not that they don't see the logic; they see the logic and wonder, "Can this really be true, when it seems so obvious now, and yet none of the people around me believe it?"
Yes. Welcome to the Earth where ethanol is made from corn and environmentalists oppose nuclear power. I'm sorry.
(See also: Cultish Countercultishness. If you end up in the frame of mind of nervously seeking reassurance, this is never a good thing—even if it's because you're about to believe something that sounds logical but could cause other people to look at you funny.)
People who've had their trust broken in the sanity of the people around them, seem to be able to evaluate strange ideas on their merits, without feeling nervous about their strangeness. The glue that binds them to their current place has dissolved, and they can walk in some direction, hopefully forward.
Lonely dissent, I called it. True dissent doesn't feel like going to school wearing black; it feels like going to school wearing a clown suit.
That's what it takes to be the lone voice who says, "If you really think you know who's going to win the election, why aren't you picking up the free money on the Intrade prediction market?" while all the people around you are thinking, "It is good to be an individual and form your own opinions, the shoe commercials told me so."
Maybe in some other world, some alternate Everett branch with a saner human population, things would be different... but in this world, I've never seen anyone begin to grow as a rationalist until they make a deep emotional break with the wisdom of their pack.
Maybe in another world, things would be different. And maybe not. I'm not sure that human beings realistically can trust and think at the same time.
Once upon a time, there was something I trusted.
Eliezer~18~ trusted Science.
Eliezer~18~ dutifully acknowledged that the social process of science was flawed. Eliezer~18~ dutifully acknowledged that academia was slow, and misallocated resources, and played favorites, and mistreated its precious heretics.
That's the convenient thing about acknowledging flaws in people who failed to live up to your ideal; you don't have to question the ideal itself.
But who could possibly be foolish enough to question, "The experimental method shall decide which hypothesis wins"?
Part of what fooled Eliezer~18~ was a general problem he had, with an aversion to ideas that resembled things idiots had said. Eliezer~18~ had seen plenty of people questioning the ideals of Science Itself, and without exception they were all on the Dark Side. People who questioned the ideal of Science were invariably trying to sell you snake oil, or trying to safeguard their favorite form of stupidity from criticism, or trying to disguise their personal resignation as a Deeply Wise acceptance of futility.
If there'd been any other ideal that was a few centuries old, the young Eliezer would have looked at it and said, "I wonder if this is really right, and whether there's a way to do better." But not the ideal of Science. Science was the master idea, the idea that let you change ideas. You could question it, but you were meant to question it and then accept it, not actually say, "Wait! This is wrong!"
Thus, when once upon a time I came up with a stupid idea, I thought I was behaving virtuously if I made sure there was a Novel Prediction, and professed that I wished to test my idea experimentally. I thought I had done everything I was obliged to do.
So I thought I was safe—not safe from any particular external threat, but safe on some deeper level, like a child who trusts their parent and has obeyed all the parent's rules.
I'd long since been broken of trust in the sanity of my family or my teachers at school. And the other children weren't intelligent enough to compete with the conversations I could have with books. But I trusted the books, you see. I trusted that if I did what Richard Feynman told me to do, I would be safe. I never thought those words aloud, but it was how I felt.
When Eliezer~23~ realized exactly how stupid the stupid theory had been—and that Traditional Rationality had not saved him from it—and that Science would have been perfectly okay with his wasting ten years testing the stupid idea, so long as afterward he admitted it was wrong...
...well, I'm not going to say it was a huge emotional convulsion. I don't really go in for that kind of drama. It simply became obvious that I'd been stupid.
That's the trust I'm trying to break in you. You are not safe. Ever.
Not even Science can save you. The ideals of Science were born centuries ago, in a time when no one knew anything about probability theory or cognitive biases. Science demands too little of you, it blesses your good intentions too easily, it is not strict enough, it only makes those injunctions that an average scientist can follow, it accepts slowness as a fact of life.
So don't think that if you only follow the rules of Science, that makes your reasoning defensible.
There is no known procedure you can follow that makes your reasoning defensible.
There is no known set of injunctions which you can satisfy, and know that you will not have been a fool.
There is no known morality-of-reasoning that you can do your best to obey, and know that you are thereby shielded from criticism.
No, not even if you turn to Bayescraft. It's much harder to use and you'll never be sure that you're doing it right.
The discipline of Bayescraft is younger by far than the discipline of Science. You will find no textbooks, no elderly mentors, no histories written of success and failure, no hard-and-fast rules laid down. You will have to study cognitive biases, and probability theory, and evolutionary psychology, and social psychology, and other cognitive sciences, and Artificial Intelligence—and think through for yourself how to apply all this knowledge to the case of correcting yourself, since that isn't yet in the textbooks.
You don't know what your own mind is really doing. They find a new cognitive bias every week and you're never sure if you've corrected for it, or overcorrected.
The formal math is impossible to apply. It doesn't break down as easily as John Q. Unbeliever thinks, but you're never really sure where the foundations come from. You don't know why the universe is simple enough to understand, or why any prior works for it. You don't know what your own priors are, let alone if they're any good.
One of the problems with Science is that it's too vague to really scare you. "Ideas should be tested by experiment." How can you go wrong with that?
On the other hand, if you have some math of probability theory laid out in front of you, and worse, you know you can't actually use it, then it becomes clear that you are trying to do something difficult, and that you might well be doing it wrong.
So you cannot trust.
And all this that I have said, will not be sufficient to break your trust. That won't happen until you get into your first real disaster from following The Rules, not from breaking them.
Eliezer~18~ already had the notion that you were allowed to question Science. Why, of course the scientific method was not itself immune to questioning! For are we not all good rationalists? Are we not allowed to question everything?
It was the notion that you could actually in real life follow Science and fail miserably, that Eliezer~18~ didn't really, emotionally believe was possible.
Oh, of course he said it was possible. Eliezer~18~ dutifully acknowledged the possibility of error, saying, "I could be wrong, but..."
But he didn't think failure could happen in, you know, real life. You were supposed to look for flaws, not actually find them.
And this emotional difference is a terribly difficult thing to accomplish in words, and I fear there's no way I can really warn you.
Your trust will not break, until you apply all that you have learned here and from other books, and take it as far as you can go, and find that this too fails you—that you have still been a fool, and no one warned you against it—that all the most important parts were left out of the guidance you received—that some of the most precious ideals you followed, steered you in the wrong direction—
—and if you still have something to protect, so that you must keep going, and cannot resign and wisely acknowledge the limitations of rationality—
—then you will be ready to start your journey as a rationalist. To take sole responsibility, to live without any trustworthy defenses, and to forge a higher Art than the one you were once taught.
No one begins to truly search for the Way until their parents have failed them, their gods are dead, and their tools have shattered in their hand.
Post Scriptum: On reviewing a draft of this essay, I discovered a fairly inexcusable flaw in reasoning, which actually affects one of the conclusions drawn. I am leaving it in. Just in case you thought that taking my advice made you safe; or that you were supposed to look for flaws, but not find any.
And of course, if you look too hard for a flaw, and find a flaw that is not a real flaw, and cling to it to reassure yourself of how critical you are, you will only be worse off than before...
It is living with uncertainty—knowing on a gut level that there are flaws, they are serious and you have not found them—that is the difficult thing.
我不问朋友们的童年——我缺乏社交好奇心——所以我不知道这究竟是多大范围的规律：
在我认识的那些向上攀爬的理性主义者中，凡是主动谈及自己童年的，有一个令人惊讶的倾向，他们说的话往往是这样的：「我家加入了一个邪教，我不得不从中逃脱」，或者「我的一个父母有临床意义上的精神失常，我不得不学会从他们的疯狂中筛出现实。」
相比之下，我自己在正统犹太家庭中长大的经历显得温和许多……但它达成了同样的结果：它打碎了我对周围人之理智的核心情感信任。
在这种核心的情感信任被打破之前，你不会开始作为理性主义者成长。我难以用言语说清这是为什么。也许任何不寻常的技能你所习得的——任何让你异乎寻常地理性的东西——都要求你在别人走之字形时走之字形的反方向。也许这太可怕了，如果世界在你看来仍然是一个理智的地方的话。
或者也许是，如果寻常的状态不让你感到恐惧，你根本不会费心去做那些额外的工作，去变得格外地正常以外的清醒。
我知道许多有志于理性主义的人，似乎在人体冷冻或多世界诠释这样的问题上遭遇了障碍。并非他们看不懂逻辑；他们看懂了逻辑，然后困惑：「这真的可能是真的吗？现在看来如此显而易见，然而我周围没有人相信它？」
是的。欢迎来到乙醇从玉米中提取、而环保主义者反对核能的地球。很抱歉。
（另见：邪教式的反邪教主义。如果你陷入了一种焦虑地寻求宽慰的心态，这从来都不是好兆头——哪怕这是因为你正准备相信某件听起来合乎逻辑、但可能让别人用奇怪眼神看你的事情。）
那些对周围人之理智的信任已经破碎的人，似乎能够凭借陌生想法自身的是非曲直来评估它们，而不会因其陌生而感到不安。将他们绑定在当前位置的胶水已经溶解，他们可以朝某个方向走去，但愿是向前。
孤独异见，我这样称呼它。真正的异见并不像穿着黑衣去上学；它更像是穿着小丑装去上学。
这正是成为那个孤独的声音所需要的——那个说出「如果你真的以为自己知道谁会赢得选举，你为什么不去 Intrade 预测市场上拿走那些免费的钱？」的声音——而此时你周围所有人都在想：「做一个有主见、形成自己看法的人是好事，运动鞋广告这样告诉我的。」
也许在某个别的世界，某个拥有更理智人类种群的 Everett 分支中，情况会有所不同……但在这个世界，我从未见过任何人在真正开始作为理性主义者成长之前，不先与群体的智慧做出深层的情感决裂。
也许在另一个世界，情况会不同。也许不会。我不确定人类在现实中能否同时做到信任与思考。
曾经，有一样东西是我所信任的。
叶利泽~18~信任科学。
叶利泽~18~尽职尽责地承认科学的社会过程是有缺陷的。叶利泽~18~尽职尽责地承认学术界是迟缓的，资源分配失当，搞小圈子，苛待它宝贵的异端。
承认未能活出你的理想的人身上的缺陷，有一个方便之处：你不必质疑理想本身。
但究竟谁会愚蠢到去质疑「实验方法将裁定哪个假说胜出」这一点呢？
部分地愚弄了叶利泽~18~的，是他有的一个普遍问题，对与蠢人所说之话相似的观念怀有厌恶。叶利泽~18~见过许多质疑科学本身之理想的人，无一例外都站在黑暗一边。质疑科学之理想的人，无一不是在向你兜售江湖骗术，或是试图庇护他们最偏爱的某种愚蠢不受批评，或是试图将他们个人的放弃伪装成对徒劳无功的「深邃智慧」式接受。
如果存在任何其他已经有几百年历史的理想，年轻的叶利泽会审视它然后说：「我不知道这是否真的正确，是否有更好的方式。」但科学这个理想不行。科学是那个元观念，那个让你得以改变其他观念的观念。你可以质疑它，但你理应是质疑然后接受它，而不是真的说「等等！这是错的！」
因此，当我曾经一度提出一个愚蠢的想法时，我以为，只要我确保自己做出了新奇预测，并且宣称我希望以实验来检验我的想法，我就是在表现得有德行。我以为我已经尽到了所有应尽的义务。
于是我以为自己是安全的——并非安全于某种特定的外部威胁，而是在某个更深的层次上安全，就像一个信任父母、并遵守了父母所有规则的孩子。
我早已对家人或学校老师的理智失去了信任。而其他孩子又不够聪明，无法与我和书本进行的对话相竞争。但我信任那些书，你知道的。我信任，只要我按照理查德·费曼告诉我的去做，我就会安全。我从未把这些话大声说出来，但这是我的感受。
当叶利泽~23~意识到那个愚蠢的理论究竟有多愚蠢——而且传统理性并没有将他从中拯救出来——而且科学对他浪费十年去检验那个愚蠢的想法完全没有意见，只要他事后承认它是错的……
……好吧，我不打算说这是一场巨大的情感震荡。我确实不怎么爱好那种戏剧。它只是变得显而易见：我一直是个蠢货。
这就是我试图在你身上打破的信任。你并不安全。永远不会。
就连科学也救不了你。科学的理想诞生于几个世纪之前，那时没有人对概率论或认知偏差有任何了解。科学对你的要求太少，它过于轻易地为你的好意点赞，它还不够严格，它只制定普通科学家能够遵守的准则，它把迟缓当作生活的既成事实接受。
所以不要以为，只要你遵守科学的规则，你的推理就是无懈可击的。
没有任何你能遵守的已知程序，能让你的推理无懈可击。
没有任何你能满足的已知准则集合，能让你知道自己没有成为蠢货。
没有任何你能尽力遵守的推理道德准则，能让你知道自己因此受到了批评的庇护。
不，就算你转向贝叶斯之道也不行。它更难使用，你永远无法确定自己是否用对了。
贝叶斯之道的这门学科，比科学这门学科年轻得多。你找不到教材，找不到年长的导师，找不到记录成败的历史，找不到铁板钉钉的硬性规则。你将不得不研究认知偏差、概率论、进化心理学、社会心理学，以及其他认知科学，还有人工智能——并且自行想清楚，如何将所有这些知识运用到纠正你自己的情况上，因为这在教材中尚未存在。
你不知道自己的心智究竟在做什么。他们每周都会发现一种新的认知偏差，你永远不确定自己是否已经修正了它，或者是过度修正了。
正式的数学是无法运用的。它不像约翰·Q·不信者所以为的那样容易崩溃，但你永远不真正确定基础从何而来。你不知道为什么宇宙简单到足以被理解，也不知道为什么任何先验对它有效。你甚至不知道自己的先验是什么，更不用说它们是否任何程度上是好的。
科学的一个问题是，它太模糊了，以至于无法真正让你感到恐惧。「观念应当接受实验检验。」你怎么可能在这上面出错呢？
另一方面，如果你面前摆着某些概率论的数学，而且更糟的是，你知道自己实际上无法运用它，那么就变得清晰了：你正在试图做一件困难的事，而你很可能在做错它。
所以你无法信任。
而我所说的这一切，不足以打破你的信任。那不会发生，直到你因为遵守规则而陷入你的第一次真正的灾难——而不是因为打破规则。
叶利泽~18~早就有一个概念，认为你被允许质疑科学。当然，科学方法本身也不能免于质疑！因为我们不都是优秀的理性主义者吗？我们不是被允许质疑一切吗？
而叶利泽~18~在情感上并不真正相信这种可能性的，是：在现实生活中，一个人真的可以遵循科学然后惨败。
哦，当然他说这是可能的。叶利泽~18~尽职尽责地承认了出错的可能性，说：「我可能是错的，但……」
但他并不认为失败真的会发生在，你知道的，现实生活中。你应该寻找缺陷，而不是真的找到它们。
而这种情感上的差异，是一件极其难以用文字来完成的事情，我担心我没有办法真正警告你。
你的信任不会破碎，直到你把你在这里和其他书中所学到的一切都运用起来，并且走到你所能走到的最远处，然后发现这也让你失望了——你依然是个蠢货，没有人警告过你——所有最重要的部分都被遗漏在你所接受的指引之外——你所遵循的一些最珍贵的理想，把你引向了错误的方向——
——而如果你仍然有值得保护的东西，以至于你必须继续前行，而无法退而求其次地明智地承认理性的局限——
——那么你将准备好开始你作为理性主义者的旅程。独自承担全部责任，生活在没有任何可信赖的防线之中，并且锻造出一种比你曾经被传授的更高的技艺。
没有人会真正开始寻找那条道路，直到他们的父母让他们失望，他们的神灵已死，而他们的工具在他们手中破碎。
后记： 在审阅这篇文章的草稿时，我发现了一处颇难辩解的推理缺陷，它实际上影响了所得出的一个结论。我将其保留在文中。以防你以为遵从我的建议让你安全了；或者以为你应该寻找缺陷，但不会真的找到任何。
当然，如果你过于用力地寻找缺陷，找到了一个并非真正缺陷的缺陷，并且紧抓它不放以安慰自己有多批判性，你只会比以前更糟糕……
带着不确定性生活——在内脏层面知道那些缺陷存在、它们是严重的而你还没找到它们——这才是困难之所在。