As software developers I consider we have the opportunity to interact practically with a large audience and build a know-how on complex systems. Maybe more than most people actually do, and this knowledge can be used in your daily life…
What exactly is software development about ?
First, I must confess that it is often hard to explain what software and software development is to people without prior knowledge. You can find a lot of definitions like “a collection of instructions that tell a computer how to work”, “the art of creating a program which can perform a required task” or “the collective processes involved in creating programs”. I think we need something more “practical”, so I would say that software programs are tools used by people to exchange and manipulate meaningful information, i.e. concepts (I know this does not cover all types of software but mainly those used by people in their real-life). Usually, these concepts are bound to a specific domain like accounting (e.g. ERP), natural language communication (e.g. chat), etc. Software developers simply are people building these tools.
Manipulating meaningful information is actually what your brain always does. And this information is always a prescription for action, either by modifying your behavior, the behavior of others or by interacting with real-world objects. For instance, a software notifying alerts might transform a sensor input into a meaningful visual output (e.g. a red light), causing people to stop a machine or prevent others to perform some tasks when receiving the information. Almost every business firm or human organization can use a software program to perform various tasks, which can ease out the work of a person, ensure more safety, reduce the number of errors, reduce the time to perform a task and/or reduce the requirement of manpower.
Why is software development a unique profession ?
Software development has never been an exact science and projecting successful completion and execution of even small programs is extremely difficult. Although it is out-of-the-scope of this article to demonstrate this assumption here are some insights: requirements are gathered from ambiguous human language, algorithms are subject to the halting problem, testing is subject to combinatorial explosion, etc. You can have a look to https://itdashboard.gov/ if you’d like to see how it is difficult for software projects to meet deadline and budget.
More specifically, software involves complex systems (intrinsically or interacting with). A complex system is a system composed of many components which interact with each other in a way intrinsically difficult to model, leading to unforeseen reactions. Human groups or societies, formed by the interaction and mutual dependence of individuals, is a complex system. Others examples of complex systems are organisms, the human brain, economic organizations, etc.
Last but not least, software development has been an early adopter of the open-source model that encourages open collaboration. This refers to, on the one hand, peer production relying on self-organizing public communities of individuals. On the other hand, licensed in such a way as to allow to be used, modified and distributed freely.
In a sense, software development is probably the sole craft industry mixing both high technology and massive transnational collaboration.
What do software developers have learnt ?
1. Context is everything
This principle is maybe the most important because all others result from it in some way, I will try to explain it with a real example. In the past I worked on a satellite control center software, we replicated the running environment (OS, applications, etc.) and performed the so called factory acceptance in our premises. As far as we could see everything worked as expected. Then we followed with a growing astonishment the so called on-site acceptance at our customer premises. Indeed, things started to go wrong quickly, not that the software crashed but it behaved randomly sometimes, without raising any error. After a lot of investigations we noticed that some of the configuration values were a little different in memory than in the files. We finally discovered that the decimal place of the OS was set in a different way than in our test environment, causing the parser to wrongly read floating point values. Indeed, the Great Britain and the United States are two of the few places in the world that use a period to indicate the decimal place while many other countries use a comma instead and unfortunately the server of our customer. The explanation of our failure is simple: the context has changed. What worked fine in one context did not even work in a slightly different one.
So, the next time a politician try to persuade you a recipe should be applied because it has provided good results 50 years ago or in another country, ask yourself: are contexts similar (and by the way if similar contexts could even be found in complex systems like a human society)?
2. Planning in detail does not help meeting deadlines
The planning fallacy is one of the most common and consistently demonstrated cognitive biases. It has been proven that people underestimate the time it will take to complete a future task, despite knowledge that previous tasks have generally taken longer than planned. In short, the more tasks your project involves, the more opportunities there are for planning to go wrong. I don’t say you should not plan at all but if your schedule has too much detail, you’re also likely to end up completely overwhelmed in trying to maintain it, at peril to the project. In my humble opinion, the same applies if you try to control all details of the production.
As a consequence, in recent years, software project management traded exhaustive upfront planning (a.k.a. Waterfall approach) for getting started with small steps, gather feedback, learn, course correct then repeat (a.k.a. Agile approach). One of the four core principles of the Agile Manifesto is precisely: responding to change over following a plan.
The next time that you hear a politician describe his plan for the following 5 or 10 years I recommend deep skepticism, he should at least describe simultaneously how he will handle changes in his plan. Similarly, if he’d like to create a new framework with hundreds of specific cases but promises that a control institution will be setup to ensure things will be put at the right places you should wonder if the new institution will not cost more than the target objective.
3. Fully tested is always a perspective
When cause-and-effect relationships are easily traceable you can follow the link to see what causes an element to change its state. But in complex systems tracing the cause of a given state change, or even if a given state is reachable, is in general impossible. And from the perspective of computing theory, worst-case coverage means investigating every possible possible state configuration at every step of the program. For a smallish program we are already into a test inventory way beyond the number of molecules in the universe.
Sometimes, different bugs even compensate themselves so that the system appears to work as expected. As a consequence, once a bug is resolved, others appear out of nowhere. Another example here, I worked on a 3D visualization software connected to a simulator and discovered that all orientation computations performed by the simulator were actually reversed. Nobody noticed it for years because the previous output analysis software did not provide 3D visualization at all and also reverted the orientation by using a wrong convention in a textual display. Sometimes, a single tiny failure could break a whole system due to the complex interaction between its components. If you don’t believe me simply read about the top ten software bugs ever.
If a politician claims to have a fully-tested solution to a problem ask him to prove it first, usually Murphy’s law is a best bet.
4. Maintaining is harder than building
Contrary to what you might believe, software maintenance is a broader activity than software development. Indeed, it includes error correction, enhancements, preventing obsolete capabilities, optimization, etc. Because change is inevitable, mechanisms must be developed to evaluate, control and make modifications. For instance, according to estimates, the US Federal government plans to spend more than 80 percent of the IT budget on Operations and Maintenance.
I am always surprised how much political programs focus on creating new laws (i.e. development) instead of the repeal or enhancement of obsolete laws (i.e. maintenance), although the most important cost is always to manage the “legacy system”.
5. Centralization fails to scale
In order to build software that scales, i.e. which supports massive usage or complexity, developers favor a distributed/decentralized model versus a monolith/centralized model. Indeed, this increases tolerance to faults, attacks and prevent collusion (sub-groups coordinating “against” others, one simple example is the need for antitrust laws). From a political point of view the last point is probably the most important — how many individuals are ultimately able to control the social system?
“Decentralization” is one of the words that is used in the politics space the most frequently these days. This proves that most countries have first developed centralized governmental-administrative structures. While a lot of politicians have announced decentralizing reforms, it is likely that many of these have been undermined. Indeed, decentralization requires centralized structures to undermine their own position by releasing power over to the periphery, but these are the people with least interest in its success.
Conclusion
Like software the human condition is diverse, shaped by billions of variables, many of which human had little control over (call it “context”), so that they can’t/don’t do the right thing at each and every moment in their life. And in this article I’ve tried to highlight the most important lessons software developers have learnt the hard way and we should not forget as human beings: rely on collaboration instead of control and be humble about what we can achieve.
If you liked this article, hit the applause button below, share with your audience, follow me on Medium or read more for insights about: the raise and next step of Artificial Intelligence, the evolution of Computer Science, the goodness of unlearning.