Rationality, heuristics, and cost of computation
When viewed from the perspective of computer science, it’s natural to ask “How are people so smart?”: human beings still set the standard we aspire to in many areas of machine learning and artificial intelligence research, from high-level perception to language to causal reasoning. But when viewed from the perspective of psychology, we might ask “Are people so smart?”: the extensive literature on human reasoning and decision-making highlights systematic ways in which people seem to deviate from rationality. I will argue that these two different perspectives can be reconciled by using a more nuanced characterization of rationality, taking into account the consequences of the cost of computation. Using this approach, several standard “irrational” heuristics people use can be shown to be sensible compromises between error and computational cost. The resulting framework gives us a way to characterize what makes a good heuristic, a systematic approach to developing models of human reasoning, and tools for translating the algorithms that people use (and the ways that they find effective algorithms) into methods that are potentially equally useful for machine learning and artificial intelligence.