A Foundation for Successful Problem Solving - The Diagnosis
Resist the urge the jump to action - take time to understand the reality you are operating in
“Let’s just try something and go from there, the most important thing is that we take action”
I’ve muttered these words so many times, probably to the dismay of my own team, and I apologize for the error of my ways…
Other little ditties that fit into this pattern:
“What measurements do we have today? Let’s look at those first”
“We already have a pretty good idea of what needs to improve”
“We just need to tackle this one thing”
How often have you heard these phrases, followed by a few months of work, which is then followed by quiet resignation because the solution didn’t have the impact you expected?
In this post I’ll share why writing a diagnosis is a critical tool that I use to get clarity on a problem:
What is a diagnosis and how does it help shape your problem solving?
How to know when a diagnosis is needed, and how to differentiate a diagnosis from a problem statement.
How to collaboratively develop a diagnosis with your team and stakeholders, and how to know when you are done.
We are wired to solve problems, and we love the idea that our dice roll at the craps table is going to yield instant success. Casinos, AI Code generators, and Pokemon Card Packs all take advantage of this quirk of human intuition. Unfortunately, the majority of the time we’re working with an incomplete picture of the problem space, and this leads to wasted time, money, and damaged team reputation.
In an uncertain world - the diagnosis should be your go-to starting point for discovery and understanding. I’ll grant that on smaller things sometimes the problem is clear. Burned out lightbulb - probably not much diagnosis necessary. But for larger problems, how do you know?
Diagnosing the Diagnosis problem
Diagnosis takes time and too often, isn’t seen as taking action. I think this is the most common reason it gets skipped. We see ourselves as pressed for time, needing to act urgently. There’s no time to think. Do or do not, there is no try. Just do it.
How many projects waste months or years of effort only to discover they were missing huge chunks of context from the start?
Using a diagnosis is a critical step to starting from a foundation of clarity and understanding. If you have concerns about not addressing a problem accurately, it’s worth investing the time to diagnose.
What is a diagnosis?
A diagnosis is a strategic thinking tool. A good diagnosis captures the reality of the problem space you are working in. This goes beyond the specific problem you are working to solve to include surrounding pressures you may or may not have control over. It is an acknowledgement of the landscape you are operating in, and the obstacles that you may have to overcome.
In practice, a diagnosis is just a list of true statements about your environment, your culture, the project, and any other dimension of your reality that may result in challenges solving the problem you are focused on. Some examples of good diagnosis material:
Cultural, Technical, or Policy challenges that may make success difficult - whether or not you can influence them
Realities that can be worked around, but that need to be taken into account
Evidence from past efforts that inform potential risks for the project
I’ve included an example further down in this document, but for now the important takeaway is that a diagnosis is intended to represent the reality you are operating in, and the obstacles you will need to be aware of and potentially overcome.
When do I need a diagnosis?
A diagnosis helps you capture different perspectives about the problem and the realities of trying to improve that situation. When any of these sorts of questions come up for you, a diagnosis might be in order:
Is there more going on here than I realize?
What’s the risk that this fails for reasons we haven’t thought about?
When has this been tried before and why didn’t that work?
A diagnosis can be anything from a 5 minute whiteboard session to a multi-session effort to explore all possible options. In my experience, the act of calling the activity a “diagnosis” and opening the door to input from others will tell you how much is there to learn.
But I have a problem statement, that’s the same right?
A problem statement tends to describe a challenge that you’d like to solve, and should do so in an open-minded way that allows for multiple potential future solutions to exist. Problem statements are future looking and should avoid creating too many constraints on the solution space.
A diagnosis, on the other hand, is a set of statements of fact or sentiment about a situation which may or may not be solvable. Diagnosis statements should describe constraints, risks, and challenges. Diagnosis statements are designed to lay out obstacles so that your problem solving and strategic execution can consider how to address those obstacles.
Both are important tools in attacking a problem, but they are distinct and serve different purposes.
Developing a Diagnosis
So you are convinced, you are going to go write a diagnosis statement - how do you do it? When do you stop? How do you begin?
You can spend a lot of time on this activity, but I would encourage you to start with a few basic questions and when they are answered - move on. You can always come back and revisit the diagnosis if you discover something new, and I would encourage you to treat the diagnosis as a living document and feed it regularly.
What are those basic questions, you ask:
What specific behaviors or outcomes are we trying to change?
What forces are working against that change?
What assumptions are we treating as true?
What cultural norms are working against us making this change?
What about our environment or situation is unpredictable and could impact this work in unpredictable ways?
Seek out all contributors
When answering each of these questions, take a “Five Whys” approach with each answer. Each question will probably have multiple answers, and for each answer you should challenge yourself to come up with all the underlying contributors.
For example:
There is a perception that our software development teams deliver software too slowly. Why?
Our teams spend a lot of their time discussing changes they are planning to make and can be slow to make decisions. Why?
Team conversations are often focused on opinion and “what if” scenarios instead of next steps and evidence gathering. Why?
Team members do not always have good behavior modeling to help them know when to take action vs. discuss. Why?
Usually this is because we don’t have a team lead present to facilitate the conversation and/or the development manager isn’t very technical. Why?
Team leads are spread very thin, and managers are hired for their people management skills and not necessarily their technical skills.
This conversation could have gone a lot of different directions in different companies, but you can see how we started with an observation and exposed a number of contributors to the problem:
Team discussions may be taking more time than they should.
Team leads are spread thin,
Team members aren’t always getting good behavior modeling from team ceremonies.
Managers aren’t always technical, yet we put them in positions where that can have a negative impact on the team.
These are realities of this particular situation, and some of them we have more control over than others. From a strategic perspective, attacking the problem of software development speed without understanding these nuances would likely have you solving a very different set of problems.
Write it down, share it out
An important part of developing a diagnosis is writing it down. This is true for a few reasons:
Different people process information differently, some people will engage during conversations and some will engage with a document.
Small nuances can come out when you read all the diagnosis statements together. This can be a powerful unlock to discovering all of the contributors to a challenge.
You want a written artifact you can reflect on later to see what has changed, what you’ve improved, and evolve the diagnosis to represent your reality down the road.
Putting the diagnosis down in a document and sharing with others is a key step in making sure you’ve captured a wide set of perspectives.
How do I know when my diagnosis is complete?
A diagnosis is rarely 100% complete, but it should be clear when you’ve reached a point of diminishing returns. There is high value in the early stages of diagnosis discovery. When you sense that contributors are arguing semantics or little nuances, you’ve probably explored far enough. It’s ok to be imperfect. Here are some signals I look for and questions I’ll ask to identify if I’ve gone far enough.
If someone new picked up this problem, is there a big missing area of risk they wouldn’t know about?
Do we have feedback from anyone who is likely to have a divergent view of this diagnosis?
As we expose the diagnosis to wider and wider audiences, are we getting big new insights?
Bringing the Discipline to Pause and Diagnose
We all have a responsibility to lead in ambiguous situations. The best leaders I’ve worked with will spot when a diagnosis is missing and pause the conversation.
“What are we trying to change here? What’s happening today, and are we sure we understand why?”
“What’s the behavior we’re seeing today that is a concern? What are the drivers causing that behavior?”
“How do we know this action we’re talking about is the one with the greatest impact?”
Don’t just do this for your own teams, lend a bit of courage to the teams around you by asking the hard question when you see this happening. You might be surprised how often speaking up empowers others to enter the conversation. There are a lot of reasons a diagnosis gets skipped, and speaking up isn’t going to work all the time, but it’s important to try.
A Note on Safety and Candor
No post on talking about challenges would be complete without mentioning psychological safety. In the case of developing an accurate diagnosis, safety is critical, and it can be the difference between an accurate and inaccurate diagnosis. Have you ever watched a project fail because, while it was technically correct, it lacked the necessary support to overcome unspoken institutional challenges?
Many of the challenges of our environments are hard to talk about, that’s often why they are challenges. Writing a diagnosis that ignores that which shall not be named is far less valuable. In fact, the act of writing down those really hard to speak about challenges in your organization is the first step to acknowledging them and addressing them.
My advice here is to find a way to write them down that feels slightly uncomfortable, while avoiding being too edgy or calling anyone out specifically. Not sure how to do that? Then find people who are good at influential writing who can help you find a way to get these challenges down on paper.
Other Inputs to a Diagnosis
Besides the work of writing down diagnosis statements and getting feedback from your team, there are other sources of input to a diagnosis that many teams have already:
Surveys: These are excellent sources of information for a diagnosis. Because they are sentiment based and are qualitative in nature, they aren’t often seen as “data” - but they are a rich source of insight into whether you’ve captured the full picture of challenges in your organization.
Bugs, Incidents, Retrospectives: What other sources of evidence do you have that may provide clues about the challenge you are taking on? Can you incorporate data from these sources into your diagnosis?
Industry patterns and research: If you spot a situation where your industry has a solution that isn’t being applied in your situation, it’s worth calling that out.
It’s easy to get into a cycle of analysis paralysis when considering all viewpoints. Review the feedback above about how to know when you are done and ship it!
Reflecting on Key Points
Let’s wrap this up - I really appreciate you reading this far!
Keep in mind the signals that suggest a diagnosis might help and don’t be afraid to ask challenging questions of your teams to see if this step is needed.
Write down your diagnosis and involve others to collaboratively build your full diagnosis. Pull in other sources of data if it’s useful, but don’t get bogged down in analysis.
Keep an open mind, be curious, and create a safe environment to hear candid feedback about challenges that are difficult to talk about in your organization. Find a way to write those down.
For additional reading, I recommend the book “Good Strategy / Bad Strategy” by Richard Rumelt, which covers combining a diagnosis with guiding policy and cohesive action to create a comprehensive strategy. For software engineering specific disciplines, Will Larson will be releasing a new book called “Crafting Engineering Strategy” soon which covers the diagnosis process in much more depth.
I hope this helps with your next challenging problem - drop me a note and let me know if it helped or if you ran into problems.