李绳武谈为何及如何Fail Fast

除了显赫家世（李光耀之孙、李显扬之子）外，李绳武在经济学界也绝对是明星，入选REStud Tour，现任哈佛大学经济系助理教授。

最近我读到他关于“fail fast”的一些话，觉得很受启发（他的论文我反正是读不懂的，就只能对这种学术边角料感兴趣了），翻译一下，分享给大家。

关于在研究中学会 "快速失败"的看法

李绳武

当你有一个新想法，而这个项目只有在P的情况下才有价值，那么你需要先检查P，而不是等到最后。

举例而言，1你只能就某种特定数据建立因果关系。

2你的理论只有在某一猜想成立时才有意义。

3只有当你能排除某种扰动时，你的实验才具有说服力。

当你可以很低的成本检查相关条件时，这就是你必须做的第一件事，而不是最后才做。

为什么人们会犯等到最后这种错误？许多研究生有这样的心态：想法是稀缺的，时间是充裕的。"拥有一个活跃的研究项目 "是令人放心的，并给出了（这个项目的）结构（structure）。丢弃一个项目感觉像是一种挫败，所以拖延检查必要条件就很诱惑人。

相比之下，经验老到的研究人员的心态是这样的：想法很多，时间很少。大多数项目都会失败，而迅速发现这一点可以让你把时间用在刀刃上。

因此，人们应该优先考虑那些能够迅速发现“一个项目值不值得”的任务，即使这些任务对完成项目没有直接效果。

这类任务包括：

1运行在线试点实验，看看数据是否过于嘈杂而无法检测出合理的效应范围。

2编写代码来试图生成关键猜想的反例。

3搜索那些“如果合理就会使项目失败”的扰动因素。

在项目初期的讨论中，我们很容易把注意力集中在项目的优点上(这是你的想法，你很兴奋！）。但你的时间很宝贵，大多数项目都会失败，你应该优先检查必要条件。

进一步的猜想：这就是为什么学生会觉得教师在Brown Bag午餐会上 "太挑剔"，而教师则觉得他们"在帮助"的部分原因。他们对成功率和操作顺序有不同的理解。

我不是在谈论因为项目没有显著效应而放弃项目。我说的是放弃项目是因为它们不连贯/识别性差/没有信息量。

Thoughts on learning to "fail fast" in research

@ShengwuLi

When you have a new idea, and the project is worthwhile only if P, then you need to check P first, not last. 

Examples: 1. You can only establish causality with a certain kind of data. 

2. Your theory is only interesting if a certain conjecture holds.

3.Your experiment is only convincing if you can rule out a certain confound.

When the relevant condition can be cheaply checked, this has to be the first thing you do, not the last.

Why do people make this error? Many grad students have the mindset that ideas are scarce and time is plentiful. "Having an active project" is reassuring and gives structure. Discarding a project feels like a setback, so it's tempting to put off checking necessary conditions.

By contrast, experienced researchers have the mindset that ideas are plentiful and time is scarce. Most projects fail, and discovering this quickly lets you put your time to better use.

Hence, one should prioritize tasks that can quickly reveal that a project is not worthwhile, even if those tasks are not directly productive to completing the project!

Examples of such tasks:

1. Running online pilot experiments to see if the data is too noisy to detect reasonable effect sizes.

2. Writing code that tries to generate counterexamples to key conjectures.

3. Searching for confounds that, if plausible, would torpedo the project.

In early-stage discussions, it's tempting to focus on the upsides of the project. (it's your idea, you're excited about it!) but your time is valuable, most projects fail, and you should check the necessary conditions first.

Further conjecture: this is partly why students can feel like faculty at brown bag lunches are "too critical", while faculty feel like they're "being helpful". They have different understandings of success rates and order of operations.

I’m not talking about dropping projects because they have null results. I’m talking about dropping projects because they’re incoherent / poorly identified / uninformative.

原文：https://twitter.com/ShengwuLi/status/1385614637970886658

阅读更多：