-
@ gsovereignty
2024-02-17 13:54:35Before I tell you how to design perfect software, let's look at the two most common design processes in use today.
These don't work, yet are firmly applied by a majority of engineers and designers, especially in software where it's possible to construct byzantine complexity.
They are slow-motion tragedies but can be fun to watch, from a safe distance.
Shit Driven Development (SDD)
The most popular design process in large businesses is Shit Driven Development.
SDD feeds off the belief that ideas are scarce and we just need a good idea to make money.
The main output of SDD is expensive "ideations", concepts, design documents, and finally products that go straight into the garbage.
-
The Creative People come up with long lists of "we could do X and Y". Once the creative work of ideation has been done it's just a matter of implementation.
-
Managers and consultants pass these brilliant ideas to UX designers.
-
The designs get passed to engineers, who scratch their heads and wonder who came up with such stupid nonsense.
The designs do not factor in the practical costs of implementation. Minor whims create weeks of work. The project gets delayed and the managers bully the engineers into working harder.
Something resembling a working product makes it out the door, but no-one wants to use it.
After twelve months of intense marketing, the company shelves it and buys a competing product from a small start-up.
The main lessons of SDD
Ideas are cheap. No exceptions. There are no brilliant ideas.
The starting point for a good design process is to collect problems that confront people, and filter out anything that isn't valuable enough to solve.
Success depends on how good and cheap the solution is, how important the problem is, and how simple the solution is to actually use.
Complexity Driven Development (CDD)
Really good engineering teams can usually build good products. But the vast majority of products still end up being too complex and less successful than they might be. This is because specialist teams, even the best, usually follow Complexity Driven Development (CDD).
Management correctly identifies some interesting and difficult problem with economic value. In doing so they leapfrog any Shit Driven Developemt team.
The team loves building things and they get straigh to it, starting with prototypes and core layers. These work as designed and the team is encouraged and enthusiastic. They go off into intense design and architecture discussions, coming up with elegant schemas that look beautiful and solid.
Management comes back and challenges team with yet more difficult problems. We tend to equate value with cost, so the harder the problem, and more expensive to solve, the more the solution should be worth, in their minds.
The team, being engineers and thus loving to build stuff, build stuff. They build and build and build and end up with massive, perfectly designed complexity.
The products go to market, and the market scratches its head and asks, "seriously, is this the best you can do?" People do use the products, especially if they aren't spending their own money in climbing the learning curve.
Management gets positive feedback from its larger customers, who share the same idea that high cost (in training and use) means high value.
Meanwhile somewhere across the world, a small team is solving the same problem using Simplicity Driven Development, and a year later smashes the market to little pieces.
CDD is characterized by a team obsessively solving the wrong problems to the point of ridiculousness. CDD products tend to be large, ambitious, complex, and unpopular. A lot of open source software is the output of CDD processes. It is insanely hard for engineers to stop extending a design to cover more potential problems. They argue, "what if someone wants to do X?" but never ask themselves, "what is the real value of solving X?"
A good example of CDD in practice is Bluetooth, a complex, over-designed set of protocols that users hate. It continues to exist only because there are no alternatives. Bluetooth is perfectly secure, which is close to useless for a proximity protocol. At the same time it lacks a standard API for developers, meaning it's really costly to use Bluetooth in applications.
CDD is a form of large-scale "rabbit holing", in which designers and engineers cannot distance themselves from the technical details of their work. They add more and more features, utterly misreading the economics of their work.
The main lessons of CDD
-
Making stuff that you don't immediately have a need for is pointless. Doesn't matter how talented or brilliant you are, if you just sit down and make stuff, you are most likely wasting your time.
-
Not all problems are not equal. Some are simple, and some are complex. Ironically, solving the simpler problems often has more value to more people than solving the really hard ones.
-
It is crucial to have a "stop mechanism", a way to set short, hard deadlines that force people to make smaller, simpler answers to just the most crucial problems.
Reality Driven Development (RDD)
Reality Driven Development starts with a realization: we do not know what we have to make until after we start making it.
Coming up with ideas, or large-scale designs isn't just wasteful, it's directly toxic to creating truly accurate solutions. The really juicy problems are hidden away in crevices across the landscape. Any activity except active scouting creates a fog that hides those crevices. You need to keep mobile, pack light, and move fast.
RDD works as follows:
We collect a set of interesting problems (by looking at how people use technology or other products) and we line these up from simple to complex, looking for and identifying patterns of use.
We take the simplest, most dramatic problem and we solve this with a minimal plausible solution, or "patch". Each patch solves exactly a genuine and agreed problem in a brutally minimal fashion.
We apply one measure of quality to patches, namely "can this be done any simpler while still solving the stated problem?" We can measure complexity in terms of concepts and models that the user has to learn or guess in order to use the patch. The fewer, the better. A perfect patch solves a problem with zero learning required by the user.
Our product development consists of a patch that solves the problem "we need a proof of concept" and then evolves in an unbroken line to a mature series of products, through hundreds or thousands of patches piled on top of each other.
We do not do anything that is not a patch. We enforce this rule with formal processes that demands that every activity or task is tied to a genuine and agreed problem, explicitly enunciated and documented.
We build our projects into a supply chain where each project can provide problems to its "suppliers" and receive patches in return. This arrangement creates a "stop mechanism" - when people are impatiently waiting for a patch, we necessarily cut our work short.
Individuals are free to work on any projects, and provide patches at any place they feel it's worthwhile. No individuals "own" any project, except to enforce the formal processes. A single project can have many variations, each a collection of different, competing patches.
Projects export formal and documented interfaces so that upstream (client) projects are unaware of change happening in supplier projects. Thus multiple supplier projects can compete for client projects, in effect creating a free and competitive market.
We tie our supply chain to real users and we iterate the whole process in rapid cycles so that a problem received from a user can be analyzed, evaluated, and solved with a patch in a few hours.
At every moment from the very first patch, our product is shippable. This is essential, because a large proportion of patches will be wrong (10-30%) and only by giving the product to users can we know which patches have become problems and themselves need solving.
RDD is a form of "hill climbing algorithm" or a drunken stumble to greatness. It's a reliable way of finding optimal solutions to the most significant problems in an unknown landscape.
This was created as a collaboration between G and Pieter H.
-