From 2ce26c057644faa0637e09afa51516b1dc39a9e6 Mon Sep 17 00:00:00 2001 From: Souradeep Nanda Date: Wed, 12 Jun 2024 10:25:47 -0500 Subject: [PATCH 1/3] The Bitter Lesson --- README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.md b/README.md index 1325839..1a24b19 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,7 @@ Like this project? Please considering [sponsoring me](https://github.com/sponsor * [The Two Pizza Rule](#the-two-pizza-rule) * [Wadler's Law](#wadlers-law) * [Wheaton's Law](#wheatons-law) + * [The Bitter Lesson](#the-bitter-lesson) * [Principles](#principles) * [All Models Are Wrong (George Box's Law)](#all-models-are-wrong-george-boxs-law) * [Chesterton's Fence](#chestertons-fence) @@ -714,6 +715,17 @@ See also: Coined by Wil Wheaton (Star Trek: The Next Generation, The Big Bang Theory), this simple, concise, and powerful law aims for an increase in harmony and respect within a professional organization. It can be applied when speaking with coworkers, performing code reviews, countering other points of view, critiquing, and in general, most professional interactions humans have with each other. +### The Bitter Lesson + +[PDF](https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf) +[HTML](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) + +> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. +> +> Rich Sutton (2019) + +Scale, in terms of both data and computational power, has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves. + ## Principles Principles are generally more likely to be guidelines relating to design. From a97981e73580730341055528f78d404d45c8ff08 Mon Sep 17 00:00:00 2001 From: Souradeep Nanda Date: Wed, 12 Jun 2024 10:29:07 -0500 Subject: [PATCH 2/3] Add twitter handle of author --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1a24b19..f81a497 100644 --- a/README.md +++ b/README.md @@ -724,7 +724,7 @@ Coined by Wil Wheaton (Star Trek: The Next Generation, The Big Bang Theory), thi > > Rich Sutton (2019) -Scale, in terms of both data and computational power, has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves. +The bitter lesson by [Rich Sutton](https://x.com/richardssutton) says that scale, in terms of both data and computational power, has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves. ## Principles From 8e99eb1643a46d6ec2daec357e01bb80f2ef13b0 Mon Sep 17 00:00:00 2001 From: Dave Kerr Date: Fri, 28 Mar 2025 10:29:47 +0000 Subject: [PATCH 3/3] docs: cleanup 'the bitter lesson' --- .github/website/src/index.html.jinja | 1 - .github/website/src/script.js | 3 +++ .github/website/src/styles.css | 7 +++++++ README.md | 30 +++++++++++++++------------- 4 files changed, 26 insertions(+), 15 deletions(-) diff --git a/.github/website/src/index.html.jinja b/.github/website/src/index.html.jinja index 6dc6117..6d646ed 100644 --- a/.github/website/src/index.html.jinja +++ b/.github/website/src/index.html.jinja @@ -61,7 +61,6 @@ -
↑ Back to Top
{% endfor %} diff --git a/.github/website/src/script.js b/.github/website/src/script.js index 0dd5292..67c9fd5 100644 --- a/.github/website/src/script.js +++ b/.github/website/src/script.js @@ -15,4 +15,7 @@ $(document).ready(function() { // Append to the heading $heading.append($anchor); }); + + // Bootstrap requires that blockquote elements have the 'blockquote' class. + $('blockquote').addClass('blockquote').addClass('.quote'); }); diff --git a/.github/website/src/styles.css b/.github/website/src/styles.css index af2a022..1500c6a 100644 --- a/.github/website/src/styles.css +++ b/.github/website/src/styles.css @@ -11,17 +11,24 @@ body { .container { max-width: 800px; } + header { text-align: center; margin-bottom: 2rem; padding: 2rem 0; border-bottom: 1px solid #e5e5e5; } + h1, h2, h3, h4, h5, h6 { font-family: 'Libre Baskerville', serif; /* Avoid scrolling under the sticky header. */ scroll-margin-top: 80px; } + +blockquote { + font-style: italic; +} + .law-section { margin-bottom: 2rem; padding: 1.5rem; diff --git a/README.md b/README.md index 543c551..f44ea09 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,10 @@ --- - 📖 Check out my new book [Effective Shell](https://effective-shell) on [Amazon](https://amzn.to/4ho0F91) -- 🌍 Check out the website [hacker-laws.com](https://hacker-laws.com) +- 🌍 Try [hacker-laws.com](https://hacker-laws.com) +- 🧠 Experiment with my new project [Terminal AI](https://github.com/dwmkerr/terminal-ai) - ☕️ Like this project? Consider [buying me a coffee with a one-off donation](https://github.com/sponsors/dwmkerr?frequency=one-time) -- 🧠 Check out my new project [Terminal AI](https://github.com/dwmkerr/terminal-ai) -- 🎧 Try the podcast [The Changelog - Laws for Hackers to Live By](https://changelog.com/podcast/403) +- 🎧 Listen to the podcast [The Changelog - Laws for Hackers to Live By](https://changelog.com/podcast/403) - 📖 Download the [PDF eBook](https://github.com/dwmkerr/hacker-laws/releases/latest/download/hacker-laws.pd) --- @@ -47,6 +47,7 @@ - [Premature Optimization Effect](#premature-optimization-effect) - [Putt's Law](#putts-law) - [Reed's Law](#reeds-law) + - [The Bitter Lesson](#the-bitter-lesson) - [The Ringelmann Effect](#the-ringelmann-effect) - [The Law of Conservation of Complexity (Tesler's Law)](#the-law-of-conservation-of-complexity-teslers-law) - [The Law of Demeter](#the-law-of-demeter) @@ -598,6 +599,18 @@ See also: - [Metcalfe's Law](#metcalfes-law) - [Dunbar's Number](#dunbars-number) +### The Bitter Lesson + +[The Bitter Lesson by Richard S. Sutton](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) + +> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. +> +> Richard S. Sutton (2019) + +The "Bitter Lesson", stated by [Rich S. Sutton](https://en.wikipedia.org/wiki/Richard_S._Sutton), says that scale (in terms of both data and computational power) has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves. + +He goes on to suggest that this indicates we should stop trying to build simplified (or even complex) models of the mind as history has shown that these have always in the long term been failures compared to (as an example) scaling the capacity of neural networks and applying existing methods such as convolution. + ### The Ringelmann Effect [The Ringelmann effect on Wikipedia](https://en.wikipedia.org/wiki/Ringelmann_effect) @@ -775,17 +788,6 @@ See also: Coined by Wil Wheaton (Star Trek: The Next Generation, The Big Bang Theory), this simple, concise, and powerful law aims for an increase in harmony and respect within a professional organization. It can be applied when speaking with coworkers, performing code reviews, countering other points of view, critiquing, and in general, most professional interactions humans have with each other. -### The Bitter Lesson - -[PDF](https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf) -[HTML](http://www.incompleteideas.net/IncIdeas/BitterLesson.html) - -> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. -> -> Rich Sutton (2019) - -The bitter lesson by [Rich Sutton](https://x.com/richardssutton) says that scale, in terms of both data and computational power, has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves. - ## Principles Principles are generally more likely to be guidelines relating to design.