Once upon a time in 2011, I serendipitously stumbled on to a question on a stack exchange site I had never been to, and would never visit again. The title was so click-baity I just had to click!
It was another programmer asking for help: Why am I not motivated in this excellent situation? As it happened, I had been casually studying this exact topic for the past few years and thought I knew the answer.
The personal productivity stack exchange has since gone away but I want this question to live on here for two reasons:
First, it gets to the heart of something that I think is counter intuitive (or at least counter narrative) about human motivation and so I think the information might be useful to others searching for their own answers to this question.
Second, since I’ve read the details of this question it has continued to haunt me. I think about this question all the time. Until David asked this I had never considered the business model proposed – and now it’s all I think about. It has lead me to a deep seeded belief that programmers should be getting royalties, and backwards from that, that programming is more like writing than engineering.
Question
Why am I not motivated in this excellent situation?
I am working as a freelance contractor. For a long time I have been paid by the hour. This has worked fine and my motivation has never been a problem. Now, I have gotten a deal where I get half oa ny increased profit that are due to my actions/ideas etc. This is an excellent deal, which would most likely raise my income ten-fold.
My problem, however, is that the deal caused me to completely lose my motivation. Meaning that I have literally not done any meaningful work for them for about three months. Why is that? How could such an excellent deal cause me to loose motivation? I am after understanding this to depth so please only answer if you have specific references.
David
Answer
As silly as it sounds, getting paid more money actually DECREASES performance for non-trivial tasks (see references). This problem has been studied a lot in behavioral economics, and psychology.
The problem is one of Extrinsic Motivation replacing Intrinsic Motivation.
Intrinsic motivation is your innate desire to do a good job. It’s what you feel when you’re working on something you want to be working on because you yourself want to see the project completed. Working on hobbies, or learning new non-work related skills are examples of intrinsic motivation.
Extrinsic motivation is when you receive something in exchange for your efforts. When you are paid to do some job, or when you receive a grade in school.
Intrinsic motivation is much more powerful, people who want to do a good job often produce much
better work than people who are merely getting paid to accomplish a task. The terrible thing is that our
brains are wired to replace intrinsic motivation with extrinsic motivation almost at the drop of a hat.
These people explain what’s happening far better than I can:
“But when you offer people money to do things that they wanted to do, anyway, they suffer from something called the Overjustification Effect. “I must be writing bug-free code because I like the money I get for it,” they think, and the extrinsic motivation displaces the intrinsic motivation. Since extrinsic motivation is a much weaker effect, the net result is that you’ve actually reduced their desire to do a good job. When you stop paying the bonus, or when they decide they don’t care that much about the money, they no longer think that they care about bug free code.”
I’ve tried to keep the formatting and wording of the original as accurate as possible. I never found out what happened to David; how this deal worked out in the end. He left a very touching comment at the time: “Thank you, I will use this information everyday for the rest of my life”. I wish I knew how to get in touch with him to tell him how his question has also changed my fundamental understanding of software engineering, and thus my life too.
Posted on
I can probably use this like Twitter and relieve myself of the last social media I am addicted to.
Honestly social media is really just centralized blogs with RSS and a share button.
This is by far the topic I get the most push back from engineers about. Even more than Software Engineering is Writing. They always want to go slower, more carefully. The idea of doing what they’re doing now but faster seems crazy. It is crazy. You can’t go faster doing what you’re doing now. Instead I want us to change what we’re doing to go faster.
Why should we be trying to go faster? In short, because we’re professionals.
Professionals vs Hobbyist
Activity
Professional
Hobbyist
Driving
200 MPH
65 MPH
Football QB decision to pass or run
3 sec
10 sec
Cleaning a 1br Apartment
4 hr
1 day
Baseball fastball
90 MPH
50 MPH
Programming – implementing a single screen of a prototype UI
1 month
2-3 days
An almost universal difference between a professional and a hobbyist in every job is speed. Professionals are faster than amateurs.
Most of the above data comes from the book “Wait: The Art and Science of Delay” by Frank Partnoy. The title is an interesting choice as the main argument I get out of the book is: Focus on speed. Speed allows you to wait longer to take action. Waiting longer allows you to gather more information, which increases the likelihood of a better outcome. It reminds me of the Special Forces phrase: “Slow is smooth and smooth is fast”.
The last line in the table comes from a friend at Google. They told me this is how long it takes them to make a single prototype screen for a UI. I nearly spit out my coffee. I chose this because the majority of programmers I talk with are Googlepheliacs who think everything they do is “the way” and should be emulated always, everywhere.
Speed of what?
If you want a metric, the one I’m currently using is: Working software, in the hands of customers, as quickly as possible with as low a defect rate as possible.
Another valid approach might be something like: minimizing median time to feedback.
But won’t quality suffer?
No, I don’t believe so.
It might at first, as people who are used to driving their car straight until they hit a guard rail realize they might actually have to learn how to drive. But on the whole I believe the quality of your programming skills increase faster with the quantity of finished products more than slow reasoned debate. If that’s true then the faster you build products, the faster your skills develop.
The following story illustrates the impact of speed on quality:
The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pounds of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an “A”. Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
“Ok, I’m not gonna click on any of that, but how would you propose we accomplish this?” I hear you say. Well, I have some thoughts…
Work on the problem in front of you
Stop trying to predict the future (“future proofing systems”) and just solve the problem you’re trying to solve with the minimal amount of code necessary. The current approach is hurting in the following ways:
We’re all bad at predicting the future, so what you write may never be necessary.
It takes more time to write the future proof version of the code.
Creating flexibility requires abstraction. This abstraction is implemented through indirection. Every layer of indirection adds cognitive load that is unnecessary and unrelated to the task (sometimes called incidental complexity).
Good abstractions are difficult to make correctly and require lots of examples to do well.
Repeat Yourself
A little bit of code duplication is better than maintaining 20+ abstract types.
I use the Rule of 3: I will copy and paste the same code twice. Once there is a third concrete need for that same code, I then feel like I have enough information that the likelihood of creating the appropriate abstraction is high.
Don’t Break Tools!
Two years ago I quit my job and decided to strike out on my own. Since I was programming for myself, I thought there’s no harm in using a programming language I love and had used to great success in production elsewhere, and so I wrote everything in F#. I truly believe that F# is the best language on the .NET platform and has the superior programming model for eliminating entire categories of bugs (no null, for example). I believed the strong type checker would help me move more quickly than in C#.
I was wrong, but not for the reason I originally thought. For the last 6 months I’ve moved back to C# for only one reason. Tooling. All of the tooling around C# really increases your productivity with the language despite some of the issues around null, verbosity around types, etc. in F# I had to write the simple database migrations myself or otherwise jump through hoops to have a single C# project linked to my F# project so that EF Core would do the magic. The same thing is true for writing mobile apps with Xamarin. The tooling for C# ‘just works’, while for F# there’s a lot of gotchas and edge cases that you need to figure out.
The time to go from idea to working code either in the browser or on my mobile device is shorter with C# because of the tooling aiding me in writing correct robust code.
So this has changed my language stack-rank. I’ve found it’s preferable to use the ones with the best tools. Modern IDEs, debuggers, profilers, static analysis tools all help you immensely to speed up the time it takes to write correct, robust code. If you’re not using them, then you don’t even know what you’re missing. I like VIM as much as the next person, but it doesn’t hold a candle to Visual Studio or IntelliJ in terms of productivity.
Yet in almost every large project I’ve worked on, no matter the language, someone has broken the tooling. Usually the culprit is the build. They’ve put things in non-standard directories and then write their own build script to re-assemble everything in the correct place for CI… but I wish they had just taken the time to get it working for everyday developers in the standard tool.
Testing and Other Quality Control
I’ve heard arguments from the TDD crowd that although testing requires more upfront work it actually makes you go faster. This is not born out by the evidence in the book “Making Software: What Really Works, and Why We Believe It” by Andy Oram, Greg Wilson which is as close as I’ve been able to find to someone using the scientific method to measure things about software engineering processes. You can quibble about the testing methodologies of the underlying papers, but it’s still peer reviewed and published data, which is better than any blog post, including this one.
Testing absolutely helps when a project is so large you can’t keep it all in your head, but it’s not an unalloyed good. Testing makes permanent. Maybe that’s what you want, but maybe it’s not. How can we tell the difference? A maturity model of software is useful here.
A Maturity Model of Software
I think it’s obvious on its face that not all software is created equal. Software that crashes is probably fine for a prototype of an Instagram clone – but not so much for avionics or automobiles. But the thing people seem to struggle with is the idea that different code within the same organization can be at different levels of maturity at the same time.
It doesn’t make much sense to enforce all of the quality gates that business critical software uses on prototypes or experiments. The first iteration of a new feature should be shipped to customers at prototype quality until such time it is proven valuable to customers. It is proven when people are actually using it, or better paying for it. Labels like Beta are helpful for signalling to customers that this is an experiment, allowing them to decide if they only want the stable stuff.
This minimizes mean time to feedback, and if you’ve built the wrong thing you didn’t waste 6 months making it rock solid just to have it fail in the marketplace. You’ve learned it was not viable for way cheaper. I’ve learned the hard way in my career that Field of Dreams development – “If you build it, they will come” – is NOT true.
The following three sections outline example properties of software at different maturity stages, from least mature to most mature.
Prototype/Experimental Software
Key properties include:
Customers are internal initially
Speed to customers/mean time to feedback is the most important metric
This is more important than code coverage metrics
Few automated tests or build guarantees
It takes time to write tests, but even MORE time to unwrite them when you realize you’ve made the wrong thing. Make sure you’ve built the right thing first, then add automated tests once that is proven.
Doesn’t need to follow the style guide yet if the guidelines propose onerous requirements for uses of frameworks, OOP design patterns, etc.
If it’s easier to access the database without the ORM, then do that
No onerous secrets management or storage
If it is a feature worth keeping then rotate the keys when that’s proven. Until then, check them into the repo. These keys have been designed to be easy to rotate, so stop worrying about making it so they never get leaked and instead focus on making them easy to change, should that happen
No encrypted network traffic
TLS and Secrets Management combine to make the day to day lives of developers a living nightmare. They aren’t necessary for a feature that hasn’t shipped to customers yet, so stop shooting yourselves in the foot requiring every developer to jump through 20 hoops before they can get a single line of code working.
Performance not an issue
Get it working correctly first in the simplest way possible. You might not actually need more, and if you do you’ll figure that out too.
Beta Software
This software is exactly the same as the prototype software upon first release, except that it is made available to a select few external customers. The software then incrementally matures towards critical software maturity as it is more and more proven out. Properties include:
Performance optimizations for hot paths
Begin adding automated tests for features that have market fit
Remove or rewrite features that no one is using based on data
Implement stricter security
Accessibility concerns are addressed in this phase
Improved logging and error handling
Critical Software
This is the software that makes you money, and when it goes down it affects revenue and presumably your bonus/stock valuation
Strong automated testing with CI/CD gates
Blue/Green deployment controls
Logging
Secrets management
TLS encryption on network communications
PII Encrypted at rest*
Good Security hygiene
Accessible
The key here is to apply this model not to the entire organization, but project by project or, even better, feature by feature.
This is not a new concept, although the above definitions are my own and are far from exhaustive. This idea is actually used by the the US Governement where they call it Capability Maturity Model (CMM). There are several version of CMM, and I am sure others have created many more.
If you have one you like, think I’ve left out your favorite property, or have any other thoughts let me know in the comments!
*You should always follow all local laws and regulations… maybe. I mean, Uber and AirBnB seem to have had their businesses impacted very little by flouting local laws but, what do I know IANAL
