Reflecting on 15 years of experience in tech: Businesses do not care about technical debt.
August 15, 2020
They just don’t, OK? This may be an upsetting thought for some, it was for me for a very long time until something clicked in my mind and instead of asking why don’t they care, I decided to ask a different question: Should they care? I want to take you on a journey behind my thinking and explain to you why I currently believe they shouldn’t. Before you jump to conclusions and decide to burn me as a heretic let me explain myself.
Let’s start with asking ourselves the following question:
What is a business and how it operates?
Depending on the type of business they will probably be selling products or services which will follow some sort of business-specific rules that are hopefully making the company money.
Let’s take Amazon for example.
If you ignore the Amazon Web Services side of things, Amazon is a big e-commerce platform that not only sells products but also offers others a platform to do the same.
In this case, the (simplified) business rules are the following:
- Customer can pick a product
- Customer pays for the product using one of the options available
- Customer can add the address where the product will get delivered
- Order is placed with Amazon
- Warehouse receives a request to ship the product
- Courier collects the product from the warehouse
- Courier delivers the product to the customer
Notice that none of these rules talks about how the customer has picked the product, how they made the payment or how they provided the address to Amazon. The majority of people are familiar with Amazon and would have probably thought of the mobile phone application or the website. But these days you can also place an order through Amazon Alexa using your voice. The beauty of business rules is that they apply whatever the medium of interaction is for the customer.
Imagine that the internet disappears tomorrow, what would happen to Amazon? My guess is they would continue operating by pivoting to another medium. For example, they could open a phone line or a physical catalogue and start accepting cheques via post. In a weird twist of fate Amazon could even start opening up brick and mortar shops. Of course, the lead time would increase to days if not weeks rather than the current hours or a day.
Hopefully, by this point you are starting to ask yourself a question: But why does it matter?
Well, in my opinion, the reason it matters is that in the technology world we tend to concentrate on technology rather than what value it provides to the world or the business. The main value that it provides is that it enables the world around us making things easier and more accessible.
Technology allows us to cut lead times from weeks to hours or minutes. This is true for a product being in your hands, breaking the news and events on social media, or spinning up a new service.
What sort of technology you pick is mostly irrelevant, as long as it is a good fit to solve the correct problem.
A business will run and succeed regardless of what technology you pick: .NET, JavaScript, React, Angular, Python, AWS, Google Cloud, Heroku, etc.
Business needs
Now that we have covered the idea of business rules, let’s talk about the needs of a business. One of the main ones is that the business needs to make money at some point. How the business achieves this will be very context-specific but one thing is certain: The business will always need new features to help its customers which should improve the bottom line.
In this day and age with so many startups all over the world, no business can sit still and constantly has to adapt and adjust to find new ways to not fall behind. If you sit still and do nothing - you are probably falling behind as your competition is moving forward and is either closing the gap or surpassing you.
Now if you keep that in mind, it will come as no surprise that when it comes to priorities, as soon as you have achieved or finished one goal, there is another one knocking on your door.
The more ideas the business can test the better chance of solving real problems for both business and customers. And I want you to read that statement again, as a lot of features that you will test will prove not worthwhile to improve or even keep. This is specifically about trying out ideas and understanding what works.
Where does it go wrong
So now that we hopefully have some vague idea of why businesses cannot sit still, as they are under a constant pressure to stay competitive or be more profitable, we can turn our attention to the software delivery teams.
A common theme I have seen (please don’t read this as a rule) is that business pressure translates to pressure on the software delivery teams, which leads to shortcuts being taken to deliver something faster.
Those shortcuts usually create something that can be defined as technical debt.
From the graph, we can see that the more technical debt we accumulate the bigger the cost of changes becomes, which in practice means that small changes become big changes as the system is not robust enough.
Now given what we know about the business and their needs of constantly moving forward, and the fact that technical debt would get in the way of achieving that, a logical conclusion would be to let the teams handle technical debt.
However we do not live in a perfect world, and fixing technical debt is often pushed back on as not something that delivers value, and often perceived as the very thing that the business does not want to do: Slowing down.
So now we are in a weird situation of trying to understand the best way to deal with technical debt and also convince the business on why it is important.
Some of the common points that people are making is that ignoring technical debt will make the code harder to deal with and more expensive to change which will increase lead times, slowing the teams and business down over the long run. And in business, time is money.
Longer lead times mean fewer features, which in turn translate to a lesser chance of solving real problems for both business and customers.
Systemic failure to address technical debt may eventually slow down the delivery pipeline to a point where drastic measures have to be taken and the business may be forced to slow down or even stop adding any new features. Although I could argue that there is no need to ever fully stop but I have seen this happen.
Now, although all of the points logically make sense, often all the business hears is “slow down” and pushes back saying they need the features as soon as possible and technical debt can wait, or that you can pay the technical debt later on which of course does not happen to the extent that would make a difference, because, well: New goals await.
The reality is that even though the businesses do not care about the technical debt they do care about being able to deliver at a certain pace, usually - fast.
Sending the wrong message
OK, if you are a software engineer reading this, and thinking “Yeah, the business just does not get it”. Well, I may have some bad news for you. They do get it. And the reason we are in this hot mess is highly likely your behaviour.
If you are constantly unhappy about the state of the system(s) and codebase you have to work with, you complain how slow and annoying it is, you claim it is crying for a refactor, yet when the business asks you to ship new features you manage to find new ways to pile it on top of the mounting technical debt: You are sending a wrong message to the business, and that message is two-fold.
- There are ways to deliver faster
- The problem(s) you are talking about is not really a problem And if we dig further, there is another problem that is in plain sight. You are waiting to be permitted to do the right thing.
To help you understand what I mean, think about the following: When you take your car for a service or go to see a doctor, do they ask: “Is it OK with you if they do their job well?”
I sincerely hope the answer is “No.” So why do you feel that you need to have that permission?
Businesses are too busy to care
Now, when I said that businesses do not care I may be a little bit harsh on them. After all, this is just the way things are, they do not care because they do not know they should care. If anything, I believe businesses would love to care but they cannot as they are too busy doing business things i.e. working out new features and what problems need solving and a million other things that it takes to run a business.
Not all is lost, as there is a reason they hired you. They hired you to help them try out the ideas they have and they hired you to care about the things that they cannot for the lack of time, but not a will.
Remember we covered the concept that technology is an enabler?
If technology is an enabler and you know how to control and use it well, you are, by proxy, an enabler for the business.
This is a job requirement and expectation that is always present but is hardly ever spelt out in job description or responsibilities.
From my experience, when you talk to business about all the things we have covered so far, they are mostly on board. After all, if you know you can go faster and test more things, what business would say no to that?
The trickier question is not how you explain it to business, but what can you do about it to prove that it will work.
So what can we/you do about it?
All of the above is a good philosophical discussion, but without practical advice that is all to it, so in this section, we are going to cover some terms and things that you can do to help you navigate technical debt.
Refactoring
By definition: a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behaviour.
Most refactoring can and should be completed in a few minutes to a few hours.
If a refactor will take days or weeks the team needs to come up with a plan of how to address it as they should continue to deliver value.
The above have been defined in the Refactoring book by Martin Fowler [1]
An interesting point to note on refactoring is that often when you try to communicate the risks and benefits of it to the management they may still find it hard to see the benefits or accept the need for refactoring. This means it can be hard to get the right time for large technical debt items (A. Martini et al.) [2]. But fear not as the next points should help you make your arguments stronger.
Have a plan
To make sure the team moves in the right direction and supports the business, find like-minded people and come up with a plan. You can always find some time across teams that can work together towards a goal, and if you want to make the change - you will find the time in your work day to chip away at the tasks. You can read more about some of my views on roadmaps.
Good enough in your context
You will never have a perfect system, nor should you as such a thing does not exist. So when coming up with a plan of action, one of the things to think about is what is good enough in your situation?
It is perfectly fine to move parts of the system out and leave it in the current state if there are bigger wins elsewhere. Sometimes it is best to isolate a part of the system or problem so that it can be tackled later on when it becomes a pressing issue.
Measure it
Before you set on this journey think on how you can measure the impact you will be achieving with your changes. If you can measure and prove that your work brings value to the company it will be easier to have a conversation with the business and win them over in terms of bringing on a wider change of not having to ask for the time to tackle technical debt but rather it being a mutual understanding within the business.
20% time
You may have heard of the practice in many organisations where teams get some time to work on anything they want, which often leads to some pain points in the system being squashed.
It may be a hard sell to ask the business for 20% time to focus on technical debt, that is why the previous point on measuring the impact is very important. If you can prove that things are getting better it is easier to make a bid for some plan.
Another thing to consider, is how much “interest” are you paying now? How much longer each feature is taking versus how long it should take. You may be unknowingly spending a large proportion of your time fighting against the system when fixing it may be a wiser time investment (think of paying back technical debt).
Research by A. Martini et al (2018) [2] found that the companies who track the time spent on managing technical debt spend on average a quarter of their time on such activities with a median value being 15-20%.
What to do when you cannot get 20% time
20% time is not the only way to chip away at technical debt. You can agree to have someone at your team to be on-call every week. That person’s focus that week can be technical debt and bug fixes over features. In a self-organised team that should not be an issue, right? The least you could do is try.
For a wider level of technical debt, as long as you have a plan, you can share the work across the team affected. Agree that Monday Team A spends 3 hours, Tuesday Team B, etc. This will bring other challenges like how easy is it to handover but if you crack it the outcome will be not only less tech debt but a better cross-team collaboration.
Once again referring to the research done by A. Martini et al (2018) [2] the research found that technical debt initiatives often start with an individual effort. You can be that individual in your organisation.
Use estimates to your advantage
A piece of advice I often give teams that do estimation: Use it to your advantage. They are there to tell the business how long things will take. If you think that some technical debt should be addressed, add it to your estimate. You are the person doing the work.
Don’t abuse this but use it to build up leverage and most importantly trust.
Buy if you can, otherwise consider open-source
One of the ways to split parts of the system out is by replacing them with ready software-as-a-service solutions. This may help you may cut down on maintenance costs and can let you focus on building the parts of the business that set you apart from competition.
If you don’t have the budget to buy, there are plenty of great open source solutions out there for a lot of problems, they may not always be the best for your business - but the paid-for don’t always offer all the things you need either. It is really important to remember that if you decide to run something in-house there will be maintenance costs associated with it, which in some cases may outweigh the software-as-a-service price that may seem too steep at first glance.
Finally, you can cut costs if you open-source parts of your codebase, as a lot of software-as-a-service platforms provide free licenses to open-source software.
Burn it and build it again / Greenfield?
Sometimes, in very rare scenarios, the state of the current system may be quite sad that unravelling it may outweigh the costs of starting afresh.
In my opinion there usually are some smart ways to help the current systems in place, but on very rare occasions the system may be so tightly coupled it makes more sense to build something new which is more resilient to changes.
A good level of thinking and planning has to go into this before making the call. Give it your best shot before deciding to go greenfield. The initial cost of going greenfield can be high, so it is only fair to spend some time thinking and planning to understand the full complexity involved.
Think about how you are going to go about greenfield. What caused the original system to fail? If you bring the same attitudes and problems across then no amounts of greenfield will solve the original problem.
If you do decide to go greenfield, think about the potential challenges going ahead. What crucial features you have on a roadmap that still have to be added to the old system? Can the old system go into maintenance mode? How are you going to release and test the new shiny version to the world? How do you gather feedback and what metrics matter to you? Do you need to port all of the functionality and features across or can you sunset some of them [3]? Do you have the data to make the right decisions?
Going greenfield is always very tempting, but the reality is that for a business it’s a very big decision as well as time and money investment.
Don’t be a hero
It can be very tempting to jump in and try and fix things by yourself which is a great first step, as we learned previously the change often starts with an individual. But always remember the goal is to bring about a wider change.
Constantly reflect on the state of affairs and make sure you are not the only individual that cares about doing the right thing and more importantly make sure you are not the person everyone goes to with their tech debt problems. Short term it may work but long-term it may burn you out.
Find like-minded people in the team or company and foster the change on a wider scale. This will benefit the business and your sanity in the longer term.
Summary
Businesses are too busy to care about the things they hired you to care about for them. They have their challenges and issues they have to face every single day.
Some of the best performing teams I have ever worked with had the following motto: Ask for forgiveness, not permission. Which I think is just another way to express bias for action. Looking back at my career so far, the things I wish I had done differently always come down to lack of an action at the time.
So next time you feel down at work and think that the business does not get it, remember the only person stopping you is usually yourself.
And finally, I leave you with the following proverb: “Where there is a will, there is a way”.
If you are interested in this topic please reach out to me either on: LinkedIn or Twitter