Skip to contentAnemo
EN
Contact

How to Check Software Quality When You Cannot Read Code

· 5 min read

Software quality leaves traces outside the code. How often it can be changed safely, how long a new developer takes to become useful, what happens when the person who built it is away, how many bugs come back after being fixed. All of these are readable without opening a single file, and together they tell you more than most people expect.

You are not going to assess architecture this way. You are going to find out whether the system can be changed without breaking, which is the only quality question with direct business consequences.

Key takeaways

Check 1: how often changes reach production safely

Ask for the number of deployments in the last three months, and how many had to be rolled back or hotfixed within a day.

Frequent, uneventful deployment is the strongest single indicator of a healthy system. It means the team can change things without holding their breath, which means tests exist, the release process is understood, and the parts are separable. Rare deployment with drama around each one means the opposite, whatever anyone says about code standards.

Check 2: how long a new developer takes to be useful

Ask how long it took the last person who joined to ship something to production on their own.

A few days to two weeks suggests the system is comprehensible and set up properly. Two months or more means either the documentation does not exist, the setup is undocumented folklore, or the system is entangled enough that nobody can work in one part without understanding all of it. All three are expensive, and all three get worse.

If nobody has joined recently, ask how long they think it would take. The estimate itself is informative.

Check 3: what happens when one particular person is away

Ask directly: if your most senior developer took a month off, what stops?

Every team has concentration of knowledge and a little is unavoidable. But if the honest answer is that releases stop, or that one area becomes untouchable, you have a business risk sitting in one person's head. That is a quality finding, and it is the one most likely to hurt you at the worst moment.

Check 4: how many bugs come back

Ask for the count of issues closed and then reopened in the last quarter, or fixed twice in different forms.

A fix that does not hold usually means the cause was not found, only the symptom. A system with a lot of returning bugs is one where nobody can predict the effect of a change, which is the practical definition of poor quality. This number is usually available from whatever issue tracker the team uses and almost nobody looks at it.

Check 5: whether test and live behave the same

Ask whether there is an environment that matches production, and how often something works in testing and fails live.

Teams without a realistic test environment are doing their testing in production, whether or not they describe it that way. This shows up as a pattern of releases that look fine and then need an urgent fix, and it is one of the most common causes of the deployment drama in check 1.

Check 6: how old the foundations are

Ask when the main framework, language version and libraries were last updated, and whether any are past their supported life.

Out-of-date foundations are not automatically a problem; plenty of stable systems run on older versions deliberately. It becomes a problem when nobody knows the answer, or when the reason for not updating is that nobody dares. Unsupported components also stop receiving security fixes, which turns a technical decision into a business exposure.

Reading the results together

No single check condemns a system. The pattern to worry about is a system that deploys rarely, takes months to onboard anyone, depends on one person, and has bugs that keep returning. That combination describes a codebase that cannot be changed safely, and it will slow every future piece of work whatever you pay for it.

The opposite pattern, frequent deployment and fast onboarding, usually holds up even when the code has rough edges. Tidy code that nobody can deploy is worth less than untidy code that ships.

If the answers are bad and you need to know how bad, that is what an audit is for. We cover the scope of one in what a technical audit involves.

Frequently asked questions

Will my team resent being asked these?

Most will not, because none of the six questions implies blame and most have answers the team is already aware of. Ask them as a set, explain that you are trying to understand risk rather than assign fault, and ask for the numbers rather than opinions. If a team refuses all six, that response is more informative than any answer would have been.

What if the answers are bad but the product works fine today?

That is the common case, and it is not an emergency. Poor changeability does not stop a working system from working; it makes everything you want to do next slower and more expensive. Treat it as a cost that shows up in future quotes, and decide how much of it is worth fixing before the next major piece of work.

Can I run these checks on a supplier before hiring them?

Not on your own system, since it does not exist yet, but you can ask the same questions about how they work: deployment frequency on recent projects, how they onboard, what they hand over. The answers are a good proxy, and vague answers are a real signal.

Who can do this for me if I would rather not ask?

Someone technical who has no stake in the answer. If you want us to run these checks and tell you what we find, get in touch.

How we would work on this

Related reading

Building the product for what comes next

We would rather deliver one product that holds up than three that have to be rebuilt. That standard is the same on every project, whatever its size.

Ali Boran GazelCEO

Contact us