How to break up big tech

Op-ed: Breaking up big tech

by Jim Wallace

Recently the Department of Justice announced they were going to start investigating the large tech companies for antitrust violations, potentially leading to breaking up big tech. It’s not surprising; large tech companies love to tell the story about how they are neutral platforms or common carriers and, thus, not responsible for the content others upload. This seems incredulous. How can one be a neutral platform and have a recommendation engine that chooses what subset of the data to show? These services could and should be separate: the platform that holds and distributes the data should serve 3rd parties that compete on the best way to display that data. This model mirrors how regulators decoupled power generation and transmission to protect consumers.

Almost all of the problems with social media, from the perspective of its users, come from the recommendation engines and algorithmic feeds that amp up controversy in the name of engagement. Those engines work for the advertisers — the real customers — not the users. It’s quite possible we don’t have the right technology or the right incentives to make a single technology service that works for everybody. Even if we do, it seems unlikely that a single company will get it right. In fact, as controversy after controversy makes the news, there seems to be ample evidence that none of them have gotten their algorithms right, based on the antisocial outcomes we are seeing.

In a decoupled model it wouldn’t be up to a single company to get the algorithms right. Instead data scientists at many companies could create competing algorithms, and users could then pick and choose the view they want. This model has proven effective in other markets. The Associated Press, for example, provides a stream of news stories, and news organizations then choose which ones to publish that are best for their respective audiences.

In this model a company like Facebook would be split into two companies. One company would collect and, for a reasonable fee, distribute posts in chronological order to any company. The second company could then display those posts however they feel is best for their audience.

Twitter is already closest to this model, because they license their data through the “fire hose”. 

The fire hose is a service where every time someone tweets, Twitter passes the tweet along unfiltered to the company that subscribed to the service. However, the terms of service and API updates prevent companies from using the fire hose in creating competing views of Twitter content. That’s something the DOJ could make illegal, just as they did with Microsoft in the early 2000s. They forced Microsoft to make its private APIs public so that 3rd parties (like Netscape) could compete on an even playing field.

If Facebook, Instagram and YouTube were forced to offer a service like the Twitter fire hose, one can imagine a whole slew of new innovative 3rd party services such as a ”life stream” that aggregates the updates from all the people you follow across all the platforms in one place. Parents could subscribe to a kid friendly version of YouTube, as a paid subscription, that is not trying to get you to spend more time on the site by hacking your dopamine system. Such specialized service could help parents struggling to setup some screen time boundaries for their children. Companies looking for competitive differentiation could even extend the platform to include things that users want but large tech seems deaf to, such as an edit button for tweets.

This model can generate plenty of revenue for both platform and providers. ConEd and the AP both use this model. Cable companies today make money hand over fist selling access to their pipes. 

This is not to say that large tech companies shouldn’t produce their own algorithms. Rather, they should not be the only companies allowed to produce them. We need competition to bring the best services to consumers.

To be sure, some may feel that social media, and the algorithms included, already work well based on some of the positive benefits we have seen. No one doubts the role it played in the Arab Spring. It is incredible when someone has a question about rockets and both Elon Musk and John Carmack respond! But neither that interaction nor the Arab Spring depended on an algorithm to facilitate them. 

We don’t have to throw out the baby with the bathwater. We can keep what’s great about these platforms while tempering the parts that induce anti-social behavior, through competing algorithms and user choice.


I decided to publish this op-ed here after Jack Dorsey wrote a tweet thread about opening up the platform. I thought this would be a good time to post. Looks like @Jack has been reading my unpublished work from this summer 😉

Cheesecake and Software Engineering

I think the people who liked the above tweet think I’m being funny by choosing a cake instead of a pie; that I’m suggesting that cake is superior to pie. But I’m actually being serious – Cheesecake is a pie not a cake.

When you make pumpkin pie, you make the crust, then the filling; you pour the filling into the crust and then bake it. When you make cheesecake you make the crust, and then the filling, then you pour the filling into the crust and bake it. With cake, you typically mix everything in a big bowl, pour it into a pan and bake it as is – no crust, no filling. You see, just because cheesecake is called cake doesn’t mean it actually is cake.

Words matter. They affect the way we think about things. When something is mislabeled, or we use an inappropriate metaphor, it can lead to lots of misconceptions down the road as people try to reason about it with the mental model in their head that the name invokes.

I think this has happened to Software Engineering. After 20 years in the industry, and having earned an engineering degree, I think I can now safely say that almost none of what we do is engineering. It’s writing, similar to any kind of writing that uses a team, like television writing. This mislabeling has caused a great deal of confusion for many, many years about why our industry doesn’t seem to work like any of the other engineering disciplines. But perhaps more importantly, the engineering label has limited our thinking about both the field itself and important things related to the field such as compensation models.

A personal hero, John Carmack, might agree with this. In 2012 during one of his infamous QuakeCon keynotes John Carmack said:

“… It’s nice to think of myself as a scientist/engineer sort dealing in these things that are abstract or provable or objective… and in reality, in computer science just about the only thing that’s really science is when you’re talking about algorithms and optimization is an engineering task …

… But 90% of the programmers are doing programming work to make things happen, and when I start really looking at whats going on in all of these, there really is no science and engineering and objectivity to most of these tasks …

… we like to think [that] we can be smart engineers about this, that there are objective ways to make good software. But as I’ve been looking at this more and more, it’s been striking to me how much that really isn’t the case. That aside from these things that we can measure, measure and reproduce — which is the essence of science, to be able to measure something, reproduce it, make an estimation and test that — and we get that on optimization and algorithms. But everything else we do really has nothing to do with that, and it’s about social interactions between programmers or even between yourself over time …”

https://youtu.be/wt-iVFxgFWk?t=1821

I think that if I’m honest with myself, I agree with everything he’s saying, and actually this talk is the one that made me really start thinking about this.

What would it mean to be a Software Writer rather than a Software Engineer?

Writing and Translation

I think looking at software as an act of both writing and translation would explain a few things. Like why as an industry we’re notoriously off when trying to estimate how long things take. Just like fiction authors are typically over deadline. I mean how long would it take you to write a chapter of a book? How about a good chapter?

That to me sure feels the same as: How long will it take to implement feature X? What if we want it to be a ‘good’ feature?

What does it mean to write “good” code? What does it mean to write good prose? I’ve often felt that much of what people say are good coding practices are pretty arbitrary or a matter of style or personal preference. I felt the same way in English class when we talked about the rules for good writing.

“All writing takes the same amount of time. If you’ve thought a lot about what you want to say, it flows easily.”

– Jessica Hibbard, Head of Content and Community, Luminary Labs

The quote above reminds me of an argument I often hear from a certain class of programmer. They want people to stop and think more before they write code. Typically I hear this from the functional programming types, but I’ve heard it in other contexts.

Seeing software as language and writing would probably make it more appealing to women. For decades the number of women studying computer science was growing faster than the number of men. But in 1984, something changed. That thing was to make computer science appear to be more mathematically based.

Compensation

The parallels between writing and programming don’t stop there. When you write a book, all of the effort is upfront and then once it’s complete you can sell an unlimited number of copies. Does that sound familiar?

That model also applies to TV shows and movies. One thing all of those industries has in common? The writers get royalties.

I think software writers should get royalties too. I have code in Visual Studio from 2010 that is still shipping today long after I’ve left Microsoft. I’ve written software for traders to manage the risks on large portfolios, more volume than any human could keep track of themselves and yet the traders received the huge bonuses and I got nothing. I wrote a good chunk of the order processing system for Jet.com, a system which processes millions of dollars in orders every day and I didn’t even get a choice about keeping my stock or not when it got sold to Walmart. All of that code is still running, producing value for those companies and BONUS they don’t have to pay me anymore since I don’t work there.

If software was seen as writing then, for those people who want to (Google employees?), they could join the writer’s guild rather than trying to unionize workplace by workplace.

What do you think? In what other ways is writing software more like writing other media? Let me know in the comments.

Update:

After a discussion over on reddit about this piece I’ve come up with even more ways in which software is like writing.

Update2:

Since people keep reading, I’ll keep writing or clarifying things as they come to me. Appreciate the comments!

More Similarities to writing

Software projects can be successful without good engineering, just like books can be successful without good writing.

Following all the rules and best practices in software do not guarantee success of a software project any more than following all the rules of grammar will make your book successful. In fact I’ve never seen any published paper that shows any correlation between the two. Would love to read it if someone has it.

New ideas implemented in software (Google page rank, Uber, etc.) influence the real world as much as books like The Communist Manifesto, or The Prince did. I’d be willing to wager Excel has fundamentally changed as many economies as The Communist Manifesto did.

Video game developers seek publishers that do exactly the same thing as publishers in the book industry do.

“Good” code from 1998 (when I started) looks very different from “Good” code in 2019. “Good” writing from 1886 looks very different from “good” writing in 2019.

Some authors are better writers than others at coming up with eloquent ways to say things, just like some programmers are just better at coming up with elegant solutions.

Code reviews sure look a lot like sending your writing to an editor.

Consequences if true

If the analogy to writing is closer than the analogy to engineering, then to become a better software writer/author one should look to the tools that writers use to become better writers rather than to other engineering disciplines.

https://twitter.com/ID_AA_Carmack/status/840632547075792896

Software being like writing would also explain (to me) why the many attempts I’ve seen to bring more rigorous, formal processes, like those in other engineering disciplines, has failed so consistently over the last 20 years. Software just doesn’t need to be an engineering discipline in order to be successful. Usually the explanation for this is that this is some new kind of engineering the world hasn’t seen before. Between the competing theories that SE is somehow a new engineering discipline that conveniently doesn’t seem to work like any other engineering discipline and the theory that it is not engineering at all. Occam’s Razor would dictate the simpler “not actually engineering” is the better theory until proven otherwise.