But what exactly is technical debt? From an IT operations perspective, it is a metaphor used to broadly cover the compounding problem of aging IT infrastructure and limited IT team members or the requisite skills necessary to stay on top of IT management and support responsibilities.”
One thing we already know about technical debt is that it’s always there. Just like regular debt, technical debt is under the surface, waiting for inactivity, deferred equipment upgrades, or simply letting ongoing IT maintenance tasks slide for any length of time. Technical debt typically builds slowly with one small decision after another leading to a day where the IT department is choked to death by its own stagnation or inability to stay on top of IT operations’ needs.
Don't have time to read the entire article? You could take the first step toward reducing your company's technical debt with the click of a button. Learn how to get it under control with a Whitehat IT Assessment:
Projects and forward momentum start to slow. A project that should have taken three weeks to complete, say a VDI upgrade, takes six months because to upgrade the VDI environment, we have to upgrade the desktop operating system. But to do that, we have to upgrade the hypervisor. But to do that, we have to upgrade the host servers. And on and on it goes.
Many conversations surrounding technical debt sound extremely hypothetical. Businesses want to know, “How will it directly impact me?” Older equipment takes more time to support. Warranties expire so replacement becomes more complicated and expensive and unstructured work, like critical support tickets, always takes priority over structured work, or project work. If the Support Desk can’t get to all the issues, the escalation and project team resources begin to find themselves engaged in support issues. This can slow or even stop the forward momentum of an entire organization.
Over time, the IT budget begins to shift from a desired state of 60-70 percent of an IT budget going toward support and maintenance of legacy IT purchases and 30-40 percent going toward new company initiatives and projects to 80-90 percent going toward maintenance and 10-20 percent spent developing new capabilities.
Forward momentum slows as the cash necessary for movement is not available to fund new projects. If nothing is done, this can turn into a death spiral for companies, and reducing technical debt gets harder and harder to do. If there are staffing shortages limiting IT’s ability to pile IT resources on the problems and gain control of the situation, pushing cost reduction initiatives to the bottom of the list.
In this case, we have a recent case study and practical example. Picture this: A company with 1,000 employees that use computers every day to do their job. The servers and IT infrastructure they rely on to do their work are extremely outdated for the most part. Windows Server 2008 R2 and Windows 7 still have a heavy presence in the environment, which went End of Life (EOL)/End of Support (EOS) in January 2020. The servers that deliver the applications and workloads are also at EOL and were already five years old in 2018.
Who cares if there is no impact on the end-users, right?
Sure, all this equipment is old and it is slowing down some upgrades, but is it really impacting the bottom line of the business in any meaningful way?
Reviewing ticket data for the last two years and charting it, the actual impact of technical debt on the company’s productivity begins to emerge.
But before I dig too deep into our example, let me set the stage a bit to give you some context. MetricNet, a company that compiles Support Desk metrics across all industries, company sizes etc., gives a desired benchmark of 0.5 human created tickets per employee per month. World-class level is 0.25 tickets per employee per month, but let’s not get carried away too quickly. The recommended target baseline is 1 ticket per employee per month, at a minimum. According to MetricNet, the average cost of a Level 1 ticket in the U.S. is $24 (IT labor, productivity lost, etc.).
So, let’s say the company in our example is hoping for a 0.5 tickets per employee per month. According to MetricNet, this customer, should shoot to have no more than 500 tickets per month based on their employee count. However, the closest they have gotten to that over the period measured is 860 tickets in November 2020. As more and more of their infrastructure has aged out and gone EOL, there has been a steady uptick in ticket volume and more volatility, signifying major problems.
If all these 500 tickets were just Level 1, basic tickets, like password resets, session unlocks, etc., they should be burning about $12,000 (0.5 tickets x 1,000 employees x $24 per ticket) a month in IT labor and lost productivity. As it is, over this window of time they have averaged 1,607 tickets per month, meaning they are on a burn rate of $38,568 (1,607 tickets x $24 per ticket) per month in lost productivity and IT support costs. It stands to reason that not all of these are Level 1 tickets, some are more complex Level 2 and even Level 3 issues that may require travel, all adding cost. By NOT keeping their IT infrastructure current and having competent capable people at the wheel managing this, there is a “hole” in their “profit bucket” dripping $26,568 ($38,568 - $12,000) a month that they can’t even see.
Keeping our same numbers, January of 2022 cost them more than $93,000 in burnt IT time and end-user productivity, again assuming all Level 1 tickets, or more than $81,000 over what we would consider “good.” And the problem is the hole in the bucket is getting bigger.
Now, let’s dream a little bit and consider what would happen if they had a world class IT team/infrastructure that would allow the customer to achieve the 0.25 tickets per employee per month. We would expect their ticketing costs to be around $6,000 a month. That dollar amount is much better than the amounts this customer has been dealing with the last two years.
This real-life example demonstrates that as the technical debt continued to rise, in the form of outdated, EOL hardware and software, IT eventually loses its ability to stay on top of the support issues because the issues are growing faster than IT support staff can set in place to deal with them. The compounding effect of technical debt and staff shortages eventually lead to ratcheting support ticket counts each month as end-users experienced more and more support issues.
More and more of the IT budget is going toward keeping systems going. Forward momentum, in this instance, has all but stopped for the internal IT team. Forward growth is only happening with third-party vendors supplying staff to execute project work. The cash is not there to fund new projects.
Eventually the impact will spread to daily/monthly maintenance, support, and patching, while security monitoring functions slide as IT must pour more resources into taking care of support issues, now bumping up against 4,000 support tickets/incidents per month – a full eight times more than what they should be.
If nothing is done, this will shift from a slide to a spiral – a death spiral.
As it is, in our living example, a large purchase of new infrastructure is the first step to begin to right the ship. There are not enough IT resources with sufficient skills in all the support areas necessary to remediate all the technical debt in a reasonable amount of time, which introduces some challenge in reaching a remedy to the situation. This internal IT team will have to be augmented with external resources to work through the problem.
This is just one simple example of what organizations worldwide face. They lack visibility into their IT infrastructure and find it difficult to identify and prioritize technical debt, which is hard to see directly if no one is maintaining a refresh schedule. But they do experience symptoms – long lead times for implementing new IT, missed deadlines, and pressure on the technical support team. There are approaches and techniques that can lead to measurable savings and decreased risks. With the right approach, organizations can increase their efficiency by managing technical debt.
How to See Technical Debt Coming
If there is not an annual refresh schedule for some percentage of the IT infrastructure, or if a significant portion of the IT infrastructure is out of warranty, over five years old, or on an operating system that is at or nearing EOL, then there is a strong possibility that a tide of technical debt is building.
If you routinely hear, “Why does it take IT so long to complete a project?” or “Projects are always delayed or late with IT,” there is a good possibility that direct technical debt is impeding the progress of IT projects and IT support needs to address it.
When reviewing your IT budget, pay attention to what percentage of the budget is paying for legacy infrastructure. This would include Support Desk operations and associated costs, license renewals, support renewals, maintenance costs, parts replacement cost, and your cost-per-ticket type metrics.
For many, assuming Server 2008 R2 and Windows 7 have been retired, the next most impactful EOL or EOS date with be Microsoft Server 2012, first available November 2013 and reaching End of Extended Support Oct 9, 2023.
For more information on how Whitehat Virtual can help your business reduce technical debt, take the first step toward getting help today:
Whitehat Virtual Technologies is one of the best Managed IT Service providers, and we even have an award to prove it! We're based out of Austin, Texas, and we serve companies across the United States.