The Unbearable Lightness of Software

…is heaviness truly deplorable and lightness splendid?

— Milan Kundera, The Unbearable Lightness of Being

As long as I have been writing software (a long time), developers have rejected software systems on the basis of “heaviness.” In a way, it’s an odd metaphor for something so ephemeral as software. I think generally what people mean in this context is that a given system has a substantial cognitive surface area that one must contend with in order to be productive. “Heavy,” as in a “heavy lift” to become productive. The idea could also encompass operational aspects of keeping a “heavyweight” system running, such as memory requirements (a bigger deal in the olden days) or other external resource requirements. A more general adage holds wisdom for us here: keep it simple, stupid (KISS).

But something is being smuggled into the argument: that “heaviness” is necessarily a bad thing. Put another way: is “heaviness” the right quality to select for in any software context? Generally, I don’t think so. I tend to think more in terms of ergonomics, or what people commonly and more generally refer to as “the right tool for the job.”

Let’s say a client has an ETL pipeline that was originally implemented with a handful of Python scripts and has started to become a little unwieldy as new requirements have forced design changes. It’s tempting to think “I’ll just reorganize a few things here and improve the system design.” But you might reflect that this is exactly how Apache Airflow was born at Airbnb. Are you rebuilding Apache Airflow because selecting it would be “too heavy”? I’ve been in those meetings more than once. The better question is “Does this tool ergonomically address our use-cases?” Caveat: it might take a bit of digging and learning before one is able to make a decision.

In Kundera’s landmark work, The Unbearable Lightness of Being, characters grapple with two fundamentally opposing drives: to pursue freedom or to deepen meaning through responsible consistency. I must admit, the novel touched me because I too have felt inner polarity around these competing impulses. I have chosen to pursue a career as a consulting engineer rather than taking more predictable and stable options in no small part due to my deep love of personal freedom and endless curiosity to explore new horizons. But I have traded cozier options at large stable companies for the thrill of owning those horizons.

This tension of heaviness vs. lightness runs up and down the stack and pervades the business environment on many levels. Do we adopt trunk-based development or git flow? Which suits us best, extreme programming or more gated options just south of waterfall? Is the organizational model mostly flat or mostly hierarchical? But I think the axis of “light” vs. “heavy” is a byproduct of an upstream decision around what specific things we are trying to achieve and how well a system maps to these needs. Heaviness can be splendid and lightness deplorable in the right context. It depends.