This work is licensed under a Creative Commons License.
This tweet reminded me of an idea bouncing around my head.
@cdixon "Given enough eyeballs, all bugs are shallow" turns out to have hidden depth. A slogan, but not a meaningless one.
— Jay Rosen (@jayrosen_nyu) January 11, 2014
“With enough eyeballs, all bugs are shallow” is clever and wise. But also be misleading.
Given a piece of software, if you keep adding developers to it, eventually you’ll probably find more bugs, but why?
Because it doesn’t actually tell you who finds bugs, other than the mythical “eyeballs”.
*You’ll find a bug when someone does something new that causes a problem and is sufficiently motivated to track down the problem.*
Almost all software has existing bugs. They exist because no one has hit that case *and* been motivated to fix it.
In my time at Twitter I’ve worked on or observed many projects that make use of open source software (OSS). Invariably we push software into edge-cases that no one has experienced before. And those edge-cases invariably have bugs that we fix.
Did those bugs get fixed because there were more eyes looking at the code? Or was it because someone with larger scale and more rigorous success criteria used it? In other words, do bugs get fixed because someone with a motivations different than its creators, different than even its previous users, relied on the it?
When evaluating the quality of open source software, you should consider not just the quantity of users, but their diversity and/or similarity to your usage patterns.
When running a project, pay attention to your new users, your outliers and the oddballs. They’re often going to discover some interesting things about your software.
This work is licensed under a Creative Commons License.