RSS

Category Archives: Quality

Quality Coaching: Preventing production issues

Despite our best efforts there will be bugs in production. For bugs which impact customer perceptions of quality, it’s important that we learn from them and try to prevent similar bugs.

As a Test Manager, I would regularly review newly-reported bugs and analyse what testers could do differently to improve our processes, our skills and our approach. Recently, while working more closely with developers, I’ve revised my approach to prevent bugs earlier and therefore more efficiently.

Initial bug review process 

While working as a Senior Quality Coach at Octopus Deploy, I had freedom to experiment with different approaches. This was my initial approach for bug reviews:

  1. Review recently raised bugs, working alone
  2. Categorise them by likely root cause
  3. Plan improvements to testing processes
  4. Provide recommendations to the team

Very quickly, some major flaws became obvious with this approach. You may have spotted them already.

  1. “… working alone” – We don’t know what we don’t know. I had major blind spots due to relying only on my own knowledge and experience.
  2. “… likely root cause” – i.e. possibly not the actual root cause.
  3. “Plan improvements …” – if the identified root cause is incorrect, how effective can these improvements be?
  4. “Provide recommendations …” – I’ll leave you to imagine how this went, based on all of the above.

Quality Coaching mindset

I worked with the Engineering Manager to develop a new approach to bug reviews, which was much more effective. I took off my Test Manager hat and put my Coaching hat back on. This new approach could be summarised as “Ask, don’t tell”. Our goal remained the same: Understand the root cause, how the bug was detected, how can we prevent similar bugs and/or detect them sooner.

Monthly I would meet with each developer for 30 minutes, and together we’d review ~3 bugs they’d fixed recently. This was the base set of questions we’d cover for each bug:

  • How difficult/easy was it to investigate and discover the cause of the issue? 
  • Could we have prevented this issue from reaching main branch with a safety net*? 
  • Would a safety net for this issue cost more than it’s worth? 
  • Can we detect this type of issue before our customers do? 
  • Did you find related issues while fixing this one? If so, were they known issues? 

Lessons learned

There’s no need to review every issue, or you’ll end up with a long list of improvements and no time to implement them all. In our sessions the developers selected the bugs we’d discuss (before or during the meeting) and I found that this made the session more engaging for them and therefore more productive overall.

Our bug review meetings were equally about coaching and learning for me. From the coaching side, I could prompt developers to consider security, performance, scale, and other quality aspects of their code. At the same time I gained a deeper understanding of specific features and how they’re implemented. For example, on an issue where I would normally have suggested additional performance testing, I learned about static analysis checks which could prevent the issue – much faster and cheaper!

After the first session or two, this “prevention and detection” quality mindset becomes an automatic part of bug fix work. Future sessions with the same developers felt more like a reminder or prompt to prioritise quality during the normal course of their work. The sessions also provided an opportunity to share any new progress, concerns or learnings regarding quality methods.

With experienced developers who are already quality-focused, these meetings might not yield much benefit and could start to feel like a hollow ceremony. Nobody enjoys a meeting that feels like a checkbox and doesn’t deliver tangible value. If you find that your senior developers already know exactly how to prevent similar issues from occurring, they may instead appreciate support in advocating to prioritise that work, or in preparing training for others in the company. You could cancel future sessions, stick with them as a way for you to learn more, and/or meet less often.

If you have multiple coaching sessions where you both agree that “xyz” needs to be done, and no progress is being made in that area, it has a negative effect on the energy and motivation for future bug review sessions. It may be time to switch gears, and focus your efforts on gathering leadership support to prioritise getting “xyz” done.

There are pros and cons of when to review the root cause of an issue. For high severity issues where customers are experiencing loss of critical functionality, of course we fix them urgently and discover the root cause as soon as possible. With that exception, reviewing bugs monthly after they’ve been resolved was the process I found most useful.

*A safety net is a test or process which will catch a bug before it impacts customers. Eg, unit test, dogfooding, user testing, etc.

This is the last post in a short series of posts on quality coaching.

 
Leave a comment

Posted by on September 4, 2021 in Coaching, Quality, Software Testing

 

Tags: , ,

Coaching Testers vs. Developers

After many years leading and coaching testers, for the past 6 months I’ve been working with developers as a quality coach. I’m keen to share lessons learned from my experience so far and to learn from others working in Quality Assistance and Quality Coaching roles.

What’s different?

Enthusiasm

Testers are eager to learn more about testing, it’s their passion. The “pull method” of providing coaching and training on demand worked very well, where I made it known that I’m available to the team and they approached me to arrange sessions. Testers reached out to ask for advice, and I was invited to meetings, asked to consult or collaborate, or asked to review work. In some cases my reputation in the testing community has preceded me to a new role, and testers were keen to collaborate from the moment I joined the company.

Great developers are also open to learning more about testing, but the word ‘enthusiastic’ doesn’t spring to mind. If I was to rely solely on the pull method of waiting to be asked for coaching, I would have a lot of spare time on my hands. Instead I attend standup meetings, planning sessions and retros, and I look for opportunities to propose coaching and pairing sessions. It requires me to be more confident and assertive, and somewhat persistent. It’s fair to say that my reputation doesn’t precede me with developers!

Arranging the first session with each developer is the biggest hurdle. The quality assistance/coaching model is relatively new for most people, and there’s uncertainty around what will be involved. As with most things, it gets easier each time.

Training method

For a team of testers, I could present three new concepts in three weeks. For example: mind-mapping test scenarios, focus/defocus exploratory testing technique, and consistency heuristics. Testers love learning these techniques, and incorporating them into their regular work. I picture them adding new tools to their toolbox, to help them find quality issues more efficiently and effectively. The value and usage of each new tool is self-apparent, meaning it’s obvious to the testers where and when each new concept will be useful.

Working with developers, any coaching on testing methods will be more successful if demonstrated and proven in the context of the code/feature/product they’re currently working on. I’ve found that I’m not able to update and reuse my existing presentations and materials which are theory-based and use practise testing websites. It’s almost as though the method you’re demonstrating needs to find a bug in your own product during the training session, to be considered a technique worth learning. I picture developers with a small extra toolbox for testing that’s already full, and they’re not convinced yet they need to purchase a larger toolbox!

My own expertise

When doing one-on-one coaching I learn new things every single time, such as domain knowledge, keyboard shortcuts, useful browser extensions, etc. Even so, I’m typically guiding more than I am learning.

While coaching developers it feels more like a two-way learning process. While asking leading questions about quality, scope, and risk, I’m also learning about the operating environment, debugging methods, code structure, in-house test tools, and more. Importantly, I’m seeing how many bugs can be traced back to code patterns and structure, and can therefore be prevented at that level during development. It’s exciting to learn new ways of preventing issues rather than detecting them later.

Which is better?

When I’m working with testers I automatically assume the role of mentor, trainer, leader. For years this has been my comfort zone – testers are my tribe, my people, my community. Personally and professionally, I thrive when I’m just outside my comfort zone. In the past that has lead me to pursue consulting, contracting, senior management roles, public speaking, organising meetups, hosting training courses… Now taking on this new role has allowed me to use my experience in the software industry with quality and coaching, while pushing me to learn about development more deeply.

I’m hopeful that by coaching developers I can have a greater impact on preventing/reducing product quality issues and therefore help to produce quality software faster. It’s still early days.

Please reach out if you’re also in a Quality Coach role and would like to share notes.

Part 1 in a short series of posts about Quality Coaching.

 
Leave a comment

Posted by on June 13, 2021 in Coaching, Growth, Learning, Quality

 

Tags: , , , ,

 
%d bloggers like this: