RSS

Tag Archives: Quality Coach

Quality Coaching: Preventing production issues

Despite our best efforts there will be bugs in production. For bugs which impact customer perceptions of quality, it’s important that we learn from them and try to prevent similar bugs.

As a Test Manager, I would regularly review newly-reported bugs and analyse what testers could do differently to improve our processes, our skills and our approach. Recently, while working more closely with developers, I’ve revised my approach to prevent bugs earlier and therefore more efficiently.

Initial bug review process 

While working as a Senior Quality Coach at Octopus Deploy, I had freedom to experiment with different approaches. This was my initial approach for bug reviews:

  1. Review recently raised bugs, working alone
  2. Categorise them by likely root cause
  3. Plan improvements to testing processes
  4. Provide recommendations to the team

Very quickly, some major flaws became obvious with this approach. You may have spotted them already.

  1. “… working alone” – We don’t know what we don’t know. I had major blind spots due to relying only on my own knowledge and experience.
  2. “… likely root cause” – i.e. possibly not the actual root cause.
  3. “Plan improvements …” – if the identified root cause is incorrect, how effective can these improvements be?
  4. “Provide recommendations …” – I’ll leave you to imagine how this went, based on all of the above.

Quality Coaching mindset

I worked with the Engineering Manager to develop a new approach to bug reviews, which was much more effective. I took off my Test Manager hat and put my Coaching hat back on. This new approach could be summarised as “Ask, don’t tell”. Our goal remained the same: Understand the root cause, how the bug was detected, how can we prevent similar bugs and/or detect them sooner.

Monthly I would meet with each developer for 30 minutes, and together we’d review ~3 bugs they’d fixed recently. This was the base set of questions we’d cover for each bug:

  • How difficult/easy was it to investigate and discover the cause of the issue? 
  • Could we have prevented this issue from reaching main branch with a safety net*? 
  • Would a safety net for this issue cost more than it’s worth? 
  • Can we detect this type of issue before our customers do? 
  • Did you find related issues while fixing this one? If so, were they known issues? 

Lessons learned

There’s no need to review every issue, or you’ll end up with a long list of improvements and no time to implement them all. In our sessions the developers selected the bugs we’d discuss (before or during the meeting) and I found that this made the session more engaging for them and therefore more productive overall.

Our bug review meetings were equally about coaching and learning for me. From the coaching side, I could prompt developers to consider security, performance, scale, and other quality aspects of their code. At the same time I gained a deeper understanding of specific features and how they’re implemented. For example, on an issue where I would normally have suggested additional performance testing, I learned about static analysis checks which could prevent the issue – much faster and cheaper!

After the first session or two, this “prevention and detection” quality mindset becomes an automatic part of bug fix work. Future sessions with the same developers felt more like a reminder or prompt to prioritise quality during the normal course of their work. The sessions also provided an opportunity to share any new progress, concerns or learnings regarding quality methods.

With experienced developers who are already quality-focused, these meetings might not yield much benefit and could start to feel like a hollow ceremony. Nobody enjoys a meeting that feels like a checkbox and doesn’t deliver tangible value. If you find that your senior developers already know exactly how to prevent similar issues from occurring, they may instead appreciate support in advocating to prioritise that work, or in preparing training for others in the company. You could cancel future sessions, stick with them as a way for you to learn more, and/or meet less often.

If you have multiple coaching sessions where you both agree that “xyz” needs to be done, and no progress is being made in that area, it has a negative effect on the energy and motivation for future bug review sessions. It may be time to switch gears, and focus your efforts on gathering leadership support to prioritise getting “xyz” done.

There are pros and cons of when to review the root cause of an issue. For high severity issues where customers are experiencing loss of critical functionality, of course we fix them urgently and discover the root cause as soon as possible. With that exception, reviewing bugs monthly after they’ve been resolved was the process I found most useful.

*A safety net is a test or process which will catch a bug before it impacts customers. Eg, unit test, dogfooding, user testing, etc.

This is the last post in a short series of posts on quality coaching.

 
Leave a comment

Posted by on September 4, 2021 in Coaching, Quality, Software Testing

 

Tags: , ,

New Feature Development: Working with a Quality Coach

In the previous post I’ve listed some responsibilities of the Quality Coach role. One activity I’ve performed to meet those goals was running with a team – joining a development team for 1-2 months to produce high quality results together, and to build quality into the product. 

As an experienced software tester, this was the quality coaching activity which felt most familiar and comfortable for me. I worked as part of an agile team to develop acceptance criteria, and helped guide the team on creating a test strategy and approach to suit their project. We used Atlassian’s quality assistance model as a starting point and adapted the process over time to meet our needs.

QA Kickoff

For individual stories I would attend a QA Kickoff meeting to map out test scenarios with the developer and product manager. During the meeting I’d ask a lot of questions, and together we would consider risks, probe edge cases to help determine scope, and consider various relevant aspects of software quality. Most of the mind map contents were generated together during meetings so that sessions were more collaborative and creative, to encourage thought and input from different perspectives. 

Initially when running with a new team I’d try to ask specific, leading questions during the QA Kickoff, similar to the questions asked by testers. Then over time I would slowly switch to coaching mode, asking broader questions to promote a longer-lasting quality mindset.

Software Testing QuestionsQuality Coaching Questions
What will happen if 2+ users click this button at the same time?Do you have any concerns around concurrency?
Together, can we review the tests which cover these risks?
Will data integrity be maintained if this process is interrupted? Are there any risks to data integrity?
How are they being mitigated/prevented?
Which file types are unsupported?
Is an error displayed when an unsupported file is selected?
Should the error message contain more details?
What can go wrong?
How will the user know something is wrong?
Will they have enough information to fix the issue?
I’ve added a UI automation test for this bug. How can we prevent this type of bug from reoccurring?

The software testing questions remind me of coaching sessions with junior testers in past roles. By asking specific questions, you’re leading them towards a quality risk area which you’ve already identified. The quality coaching questions remind me of reviewing a test plan prepared by a senior tester. You ask more open questions and then work out the answers while learning together.  

Unlike a software tester, the quality coach then relies on the developer to perform the actual testing and provide answers to the questions raised during the QA Kickoff meeting.

QA Review

For faster feedback loops, the developer should test their own changes prior to the QA Review. These meetings can also include a product manager, a senior/peer developer and a designer. The developer demonstrates that all scenarios from the QA Kickoff have been covered, and explains any changes made to scope/functionality since the QA Kickoff meeting. The quality coach can provide coaching during the session, usually in relation to test coverage gaps and automation test code patterns. 

I’ve found these sessions to be effective in helping developers shift perspective from the lines of code back to the user experience, and to look at the bigger picture.

But “developers can’t test their own work, can they?

To be blunt, developers being unable to test their own work is a fallacy that has suited the software testing industry very well. It has also been a convenient theory for those software developers who prefer to let someone else do their testing for them. My own opinion on this has changed in recent years, and it’s sheer luck if I don’t have a poorly-aged blog post stating that developers can’t test their own code.

I’ve seen that the best developers can and do test their own work. In an unhealthy culture, those developers can be viewed by their peers as slow and less productive, despite having better code quality and less bugs found by testers, and therefore less rework required. In those unhealthy environments, the testers and engineering managers have different ideas about which developers are the more valuable and productive members of the team. Senior testers, when paired with developers who test their own code, are able to adopt more of a quality coaching mindset and help the developer improve their own testing skills and code quality.

Quality coaching mindset

Developers on your team may be used to working with testers and be uncertain of what quality coaching means for them. It helps to clearly describe and demonstrate what quality coaching involves, and specifically what it doesn’t include – i.e. testing their changes for them.

Having a quality coach work with the same team for too long can blur the lines between quality coach and software tester. I found that by changing teams regularly, it reinforced the message that the whole team is responsible for quality. It also reminded me to act as a quality coach, especially at times when I was tempted to jump in and do some testing myself. It encourages a subtle focus shift from having product quality as the only goal, to helping the team reduce their feedback loops and independently produce quality software.

Knowing that there are no testers in the team, and that the quality coach will be back on the team again in a few months, hopefully helps to encourage a lasting quality mindset amongst the developers in the team. 

This is Part 3 in a short series of posts about quality coaching. The next posts describe some additional activities performed by a Quality Coach. 

 
Leave a comment

Posted by on August 15, 2021 in Software Testing

 

Tags: ,

The Role of a Quality Coach

Context 

For eight months I’ve worked with software development teams in the role of senior quality coach. We work in agile teams, without ever using the word agile. Most teams include software developers, a product designer, a product manager and an engineering manager. There are no testers or business analysts, and everyone on the team can write code to some degree. 

Quality coaching is not software testing

Quality coaches do not fill the role of software testers. While it’s commonly said that each person in an agile team owns quality, this truth can be undermined by the presence of a tester in the team.*

To state it more clearly: quality coaches do not own the test strategy, automation test suites, build pipelines, lists of defects, test environments, or test data. Quality coaches are not responsible for investigating failed test results, performing exploratory testing, or writing up bug reports. 

Every one of these artefacts and activities is owned and maintained by the development teams. 

Quality Coach Responsibilities

Create and promote a quality mindset within the team. 

Work with stakeholders to discover/determine quality goals. 

Explore the product to gain context. 

Provide coaching, guidance and training on software quality and testing.  

Collaborate with other teams including Support, Sales and Marketing to better understand user goals and behaviours.

Stay curious, ask questions, follow up on loose threads. 

Help facilitate risk workshops, bug bashes, post-mortems and other brainstorming sessions.

Advocate for process improvements.

This is Part 2 in a short series of posts about quality coaching. The next posts elaborate on the activities performed by a Quality Coach. 

*Embedding a tester in an agile team can work very well, for example when the bulk of testing is performed while pairing with the developer who wrote the code.

 
Leave a comment

Posted by on July 11, 2021 in Software Testing

 

Tags:

Coaching Testers vs. Developers

After many years leading and coaching testers, for the past 6 months I’ve been working with developers as a quality coach. I’m keen to share lessons learned from my experience so far and to learn from others working in Quality Assistance and Quality Coaching roles.

What’s different?

Enthusiasm

Testers are eager to learn more about testing, it’s their passion. The “pull method” of providing coaching and training on demand worked very well, where I made it known that I’m available to the team and they approached me to arrange sessions. Testers reached out to ask for advice, and I was invited to meetings, asked to consult or collaborate, or asked to review work. In some cases my reputation in the testing community has preceded me to a new role, and testers were keen to collaborate from the moment I joined the company.

Great developers are also open to learning more about testing, but the word ‘enthusiastic’ doesn’t spring to mind. If I was to rely solely on the pull method of waiting to be asked for coaching, I would have a lot of spare time on my hands. Instead I attend standup meetings, planning sessions and retros, and I look for opportunities to propose coaching and pairing sessions. It requires me to be more confident and assertive, and somewhat persistent. It’s fair to say that my reputation doesn’t precede me with developers!

Arranging the first session with each developer is the biggest hurdle. The quality assistance/coaching model is relatively new for most people, and there’s uncertainty around what will be involved. As with most things, it gets easier each time.

Training method

For a team of testers, I could present three new concepts in three weeks. For example: mind-mapping test scenarios, focus/defocus exploratory testing technique, and consistency heuristics. Testers love learning these techniques, and incorporating them into their regular work. I picture them adding new tools to their toolbox, to help them find quality issues more efficiently and effectively. The value and usage of each new tool is self-apparent, meaning it’s obvious to the testers where and when each new concept will be useful.

Working with developers, any coaching on testing methods will be more successful if demonstrated and proven in the context of the code/feature/product they’re currently working on. I’ve found that I’m not able to update and reuse my existing presentations and materials which are theory-based and use practise testing websites. It’s almost as though the method you’re demonstrating needs to find a bug in your own product during the training session, to be considered a technique worth learning. I picture developers with a small extra toolbox for testing that’s already full, and they’re not convinced yet they need to purchase a larger toolbox!

My own expertise

When doing one-on-one coaching I learn new things every single time, such as domain knowledge, keyboard shortcuts, useful browser extensions, etc. Even so, I’m typically guiding more than I am learning.

While coaching developers it feels more like a two-way learning process. While asking leading questions about quality, scope, and risk, I’m also learning about the operating environment, debugging methods, code structure, in-house test tools, and more. Importantly, I’m seeing how many bugs can be traced back to code patterns and structure, and can therefore be prevented at that level during development. It’s exciting to learn new ways of preventing issues rather than detecting them later.

Which is better?

When I’m working with testers I automatically assume the role of mentor, trainer, leader. For years this has been my comfort zone – testers are my tribe, my people, my community. Personally and professionally, I thrive when I’m just outside my comfort zone. In the past that has lead me to pursue consulting, contracting, senior management roles, public speaking, organising meetups, hosting training courses… Now taking on this new role has allowed me to use my experience in the software industry with quality and coaching, while pushing me to learn about development more deeply.

I’m hopeful that by coaching developers I can have a greater impact on preventing/reducing product quality issues and therefore help to produce quality software faster. It’s still early days.

Please reach out if you’re also in a Quality Coach role and would like to share notes.

Part 1 in a short series of posts about Quality Coaching.

 
Leave a comment

Posted by on June 13, 2021 in Coaching, Growth, Learning, Quality

 

Tags: , , , ,

 
%d bloggers like this: