RSS

Category Archives: Software Testing

Rapid Software Testing – Reading Recommendations

Having just completed Rapid Software Testing twice in two weeks with James Bach, I’m feeling motivated and inspired to continue learning.

Here’s a list of books recommended by James during the course. These will enhance your skills and change the way you look at testing.

RSTReading

The first book may be the most important, and the most difficult to read. I’m still getting through my copy. The content is excellent, and there’s a lot to take in.
The next 4 books are real page-turners, explaining important and complex information is a way that’s enjoyable to read.
I haven’t yet read the last book on this list.

An Introduction to General Systems Thinking by Gerald Weinberg
Thinking, Fast and Slow by Daniel Kahneman
Tacit and Explicit Knowledge by Harry Collins
Lessons Learned in Software Testing: A Context-Driven Approach by Cem Kaner, James Bach, Bret Pettichord
The Secrets of Consulting: A Guide to Giving and Getting Advice Successfully by Gerald Weinberg
Discussion of the Method: Conducting the Engineer’s Approach to Problem Solving by Billy Koen

If you’ve already read these books, I’m interested to hear your thoughts. For example, what was the biggest takeaway you got from each book, and how has that helped you with software testing?

 
7 Comments

Posted by on August 14, 2015 in Learning, Software Testing

 

Tags: , , , ,

What’s Your Context? – Workshop with Fiona Charles

How do you discover the differences in context between clients and projects, and whether those differences matter? While intuition is important, unconscious analysis and choices lead to unconscious assumptions – Fiona Charles

Ask a tester which approach is the best way to test software. The typical response will be “It depends”. But what does it depend on, and why? How will those facors affect testing?

Fiona presented her “What’s Your Context?” workshop to the Auckland WeTest meetup group. We split into 6 groups to brainstorm the elements of context that affect our approach to testing. It’s difficult to report on the value gained from attending a workshop, as the learning comes from being involved in the discussion. Here I’ve recorded my brief notes on the “Aha!” moments described by others at the end of the workshop.

Me: I’ll be doing this exercise regularly for projects I work on. More aware how much context can change during the course of the project.

Natalia: It was very useful to learn about other tester’s contexts.

Pete: Two favourite sayings are ‘It depends’ and ‘Why?’. For example, ‘Why are we doing this?’.

Morris: We say ‘team’ a lot. Testing is a team sport.

Vikas: Highlights the importance of thinking about context deliberately, instead of repeating past processes.

Shaheem: Maintain a risk focus, which things can derail or negatively impact the project.

Chris: The definition of Minimum Viable Product is heavily context-dependent.

Georgia: Focus on the problem being solved. Take history into account. Appreciate the benefit of informal communications (this arose in the context of working with offsite teams).

Vincent: Like the experience of working with his team, learning from their experiences, and the way they grouped elements of context together into personal, product, team and development methodology.

John: So many sources of information beyond just requirements. Stop and consider context first before you get started.

I encourage other testers to take Fiona‘s workshop. Her questions, insights and stories brought the exercise to the next level.
In the meantime consider pairing up with one or more testers and asking yourselves, “Which elements of your context affect your testing? Why, and how? How can you use this information to improve your testing approach for your current project?”

 
1 Comment

Posted by on November 10, 2014 in Software Testing

 

Tags: , , ,

Let’s Test Oz – Closing Keynote from Fiona Charles

The closing keynote speech of Let’s Test Oz 2014 was “The Battle for Our Hearts and Minds” by Fiona Charles.

This was the first time I’ve attended one of Fiona’s talks in person. My impressions of Fiona after this conference are that she is honest, practical, a strategic thinker and that she doesn’t mince words.

“I have seen 50-page test strategies without an ounce of strategic thinking”

Fiona Charles

“I’m not going to do bad work” Fiona Charles

The theme of this keynote was that attempts to standardise testing are stifling creativity and value, and that it’s time for testers to take back our craft. Fiona spoke of the need for testers to have the courage and tenacity to speak up about important issues when others remain quiet. This included being willing to ‘blow the whistle‘ where necessary to expose important issues which could affect people’s lives.

“We need to be able to say things that nobody wants to hear, because that’s our job”

The topic of testing standards came up more than once, as a primary cause of the long-term de-skilling of the testing workforce and the current overall state of testing processes and documentation. Using the example of a 25-page IEEE 829 compliant Test Plan, Fiona saw no project-specific content until page 12. The time taken to produce these documents is costing companies money, and contributes to testing being viewed as ‘too expensive’. The focus of testing should be on adding value to the project and to the company.

“The Master Test Plan is probably the most useless document since printing was invented”

Most of the people behind the creation of ISO 29119 stand to profit if the standard is introduced. Interestingly, Fiona’s opposition to ISO 29119 comes despite her anticipation that she’ll profit from the standard if it’s introduced. Fiona described how she has seen first-hand the damage caused by compliance to the IEEE 829 test documentation standard. She has been called in to multiple organisations to mop up the damage which that standard leaves in its wake, and she has every reason to believe that ISO 29119 would create more of the same damage.

“The quest for certainty collides with the reality of software development”

Fiona introduced the concept of “healthy uncertainty vs unhealthy certainty” while debunking the notion that popular test metrics are useful. She covered some key attributes of great testers, and they’re not the ones you see listed in jobs ads: Integrity, Independence of Mind, Courage, Engagement…

I really enjoyed this talk. It was motivational, inspirational and a call to action for all testers.

Recommended reading\viewing:
The slides from this keynote are available from the Let’s Test Oz website.
Breaking the Tyranny of Form blog post – Fiona Charles
Delivering Unwelcome Messages EuroSTAR webinar – Fiona Charles
Slides from We are the 99% – Anne-Marie Charrett

All quotes in this post are from Fiona Charles’ keynote.

 
2 Comments

Posted by on October 10, 2014 in Software Testing

 

Tags: , , , ,

ER: Learning exercise on the Implicit Principles of CDT

This is a detailed experience report on a learning exercise I completed recently, following my post about James Bach‘s opening keynote at Let’s Test Oz (LTO) 2014:

…There was one slide in particular which I could’ve questioned James on for another hour, called Implicit principles of the Context-Driven School of Testing. This slide contains ideas which could fill a book, if James had time to write another book.. I think I need to read some more books before I can fully fathom the concepts presented! The beauty of this conference is that I have many opportunities to find James in the hotel and ask about this slide in more detail…

On day 3 of LTO I tracked down James and we spent the better part of an hour walking through these principles. Please click on the image below and take a moment to review them in detail.

Implicit Principles of CDT - Slide from James Bach's Keynote at Let's Test Oz 2014

Presented by James Bach at Let’s Test Oz 2014

At first glance I found these Implicit principles intimidating. In hindsight, I’ve attributed this to 3 main reasons:

1. My immediate impression was that the principles are based on a lot of assumed knowledge. I’ve since concluded that these ideas stand alone, and the impression of assumed knowledge was due to the number of unfamiliar terms. In fact the only assumed knowledge is a basic understanding of software testing and an above average English vocabulary.

2. The principles include words which I couldn’t define, such as primacy, non-linearity, cybernetic and authorship. I’ll define these simply here, in my own words:
Primacy – Most important
Non-linearity – Unpredictability
Cybernetic – React to observations
Authorship – Creation

3. I couldn’t understand the reasons behind the creation of these principles. If James covered this in his talk then I missed it. I was wondering what was wrong with the original list of seven basic principles (shown below). I wanted to understand not just the new principles themselves, but also the need for them, and the thought processes that went into creating them. As an aspiring CDT practitioner, understanding this process of articulating the CDT approach is important to me.

Original CDT Principles by Cem Kaner and James Bach

Original CDT Principles by Cem Kaner and James Bach


The Learning Exercise

While discussing these principles with Anna Royzman over dinner, I mentioned that the new list seemed unapproachable for the masses. Anna agreed with the need for a shorter and more simplistic version, which I’ve called the marketing version. Anyone reading the marketing version and looking for more information could read the full version of these principles to find out more context and detail.

I approached James about this on Skype and offered to create a first draft for review, to get the ball rolling. James disagreed with the need for a marketing version of the principles, but he did agree to support me in the process of creating it. I was quite confident in the need for a new version…

Draft 1 of my marketing version

I have to remind myself now that I was proud of this version when I initially created it, while the ink was still wet. I felt that I’d captured the key points from James’ 10 Implicit principles and presented them in a user-friendly way. Here’s what I came up with:

  1. We constantly adapt test processes to changes in our real-world environment (covers point 1 from updated list)
  2. We learn through investigation and present facts based on evidence (covers points 2 & 4)
  3. Due to systems complexity, we observe and react to uncertainty (point 3)
  4. Systems are developed by people, for people (point 5)
  5. Testers have a duty to add value and behave ethically (point 6)
  6. Testers work with the team and share responsibility for quality (point 7)
  7. Testers must continue to learn a variety of skills and practices, in order to adapt processes to suit each project (covers points 8, 9 and 10)

I sent this list to James for review, and we started to review point 1 together. Based on James’ questions and comments, I had a basis to further review my own work. These are my brief notes from that review.

Self-review 1:
1.
2. ‘Facts based on evidence’ doesn’t seem to tie in with heuristics.
3. No longer sure that this sentence makes my point clear
4. Therefore… what?
5. I want\need to do this, it’s not just a duty. This sentence [as written] currently applies to everyone in the company…
6. Sounds like I’m describing Agile testers. “The team” – which team?
7. This point isn’t terrible.

This version is vacuous. I’ve abstracted too far, it could almost be describing procedure-driven testing. Start again…

Taking a Step Back

During Skype coaching James posed the question, “If you had to tell someone a few things that would get across to them the *gist* of CDT what would that be?” This expanded the learning exercise for me, as I’d have the freedom to create a whole new list, rather than devising a simplified version of the 10 implicit principles.

I decided to first work out which concepts were lacking in the 7 Basic CDT Principles, in order to understand the need for the 10 Implicit CDT Principles. I came up with a working list of the differences (shown in the image below, initially without the information in italics).

Comparison of CDT Concepts

Comparison of CDT Concepts

While reviewing both lists so closely, I gained a greater appreciation for the initial list of principles. I really started to doubt the need for a new short version, and I also doubted whether my creation could come anywhere close to the original.

When reviewing some of the differences I’d identified with James, he said something very interesting. His updated list of 10 principles was not intended to replace the original list! Therefore the original list still stands, and my marketing version is redundant. The new list was designed to clarify the implicit concepts which were behind the creation of the original principles.

The implicit principles were created by the main principles…
… I wrote the ten by asking myself what did I mean by the seven and what must necessarily be true to achieve them.

This was my aha moment. Now I understood that there was no need for a new short list. The original list was not being replaced at all, only expanded upon in more detail.

Conclusion

Although I’d started out on a fool’s errand due to a misunderstanding, this turned out to be a productive learning exercise. I can now speak confidently about the principles of context-driven testing and will continue to champion them. I’ve suggested two changes to James which he has agreed with. The first was the re-introduction of the concept of software solving a problem. The second is that community status was not implicit in the original principles and has been added in this new version.

This exercise has prompted me to finish reading Thinking, Fast and Slow by Daniel Kahneman, analysing my own thought processes.
And finally, it helped me to “keep my brain sharp” between contract roles.

Notes on Complex Language

I remain concerned that the complex language used in these principles will hinder the widespread communication and adoption of the concepts.

My concerns are valid, according to this paper by Daniel M. Oppenheimer – “Consequences of Erudite Vernacular Utilized Irrespective of Necessity: Problems with Using Long Words Needlessly”. Essentially, as the level of complexity increases the text becomes harder to understand and less likely to be accepted by the reader.

Oppenheimer’s emphasis here is on ‘needlessly’ complex words.

…there are many times when a long word is appropriate, because it is more precise or concise…
…select the most appropriate word for a given argument such that decreases in fluency are overridden by increases by other positive attributes…

What are your thoughts?

Are precision and brevity (conciseness) more important than first impressions in this case?
Does the original set of principles serve the needs of newcomers adequately, lessening the need for the new version to be approachable?

 
4 Comments

Posted by on October 7, 2014 in Software Testing

 

Tags: , ,

Let’s Test Oz 2014 – Day 1

It’s 11PM and day 1 hasn’t finished yet, there are activities still happening around the hotel. This is really a different style of conference than I’m used to. All participants stay at the same hotel where the keynotes, breakout sessions and test lab take place, so the conference doesn’t actually end at a specific time each night.

I arrived this morning in the picturesque Blue Mountains outside of Sydney. I was excited, nervous, intimidated, keen, and relieved to have made it here. There was a contingent of testers already here and the conferring had begun before the conference’s opening keynote. About 5 minutes after entering the hotel, most thoughts of intimidation and nervousness were gone. This is where I was meant to be. I mingled and promoted twitter as a means for learning more about the context-driven testing community.

The opening keynote was delivered by James Bach:

How do I know I am context-driven?

What followed was a wealth of information based on years of research, hands-on experience and debates, condensed into a one-hour talk. This was an excellent summary of what it means to be context-driven, from one of the founders of the context-driven testing community. There was one slide in particular which I could’ve questioned James on for another hour, called Implicit principles of the Context-Driven School of Testing. This slide contains ideas which could fill a book, if James had time to write another book.. I think I need to read some more books before I can fully fathom the concepts presented! The beauty of this conference is that I have many opportunities to find James in the hotel and ask about this slide in more detail, ask for advice on recommended further reading, and discuss testing in depth.

As usual, I found James’ talk personally motivating and compelling. Specifically, the categorisation of levels of involvement in the context-driven community felt to me like a call to action and I’ve treated it as such. I will be actively ensuring that I fall into the Committed Practitioner category, and probably also Committed Student as I love to keep learning.

These are some of my favourite quotes from James‘ keynote speech:

“A professional society of people trying to be the best they can be” – Yes! This is a growing crowd which I’m proud to be a part of.

“Respect and nurture people who are learning” – James noted the Greeting vs. Challenging methods of introducing testers to the context-driven community, and his tendency towards the latter. There are other leaders in the test community who patiently introduce those who are newly discovering professional testing approaches.

“The product is a solution. If the problem isn’t solved, the product doesn’t work”. Hallelujah!

“Testing has parallels with martial arts, you need to practise, and EARN respect” – I’m paraphrasing here.

“Context-driven testers must be able to answer the question ‘What’s your approach to testing?'” – Oops. I have some homework to do.

“Instead of best-practice, say a practice. For example, we use a practice for defect management.” That’s a huge improvement! This would make the software world a better place 🙂

“You don’t need to promise, quantify or lie. See Keith Klain for more information.” I’m paraphrasing again. This was another call to action for me.

In conclusion…

This is what I got from the FIRST HOUR of this 3 day conference. I’m glad I made the effort to attend, I’d have deeply regretted missing out.

Stay tuned, more to come. But not tonight 🙂

 
3 Comments

Posted by on September 15, 2014 in Software Testing

 

Tags: , ,

Should you take the Rapid Software Testing course?

Yes.

This year I flew to Australia from New Zealand to attend the Rapid Software Testing course with James Bach, hosted by Testing Times.

The course is crammed with practical tips that you can start using immediately on your current projects. James has concocted mnemonic devices that really work.

Practical exercises are a great way to practice and demonstrate what you’ve just learned, and being in the ‘hot seat’ is a challenging and memorable experience. Once James dons his cap the teacher has gone, and the pointy-haired boss has arrived. I highly recommend volunteering for a hot-seat challenge, to help get the most out of your experience.

I’d already done some of the RST practical exercises during a Skype coaching session with James, so I sat out the first exercise… for about 30 seconds. Then I realised* I could pick up where I’d left off with the exercise, taking my learning to the next level. I found my old Skype chat transcripts, pulled up my previous test findings, and continued digging deeper into the exercise. I was even able to channel Rich and ‘Go Meta‘ which I was very excited about.

During the course I also learned transferable skills from James’ tips, mentioned as asides on coaching, critical thinking, presenting, professionalism, accountability, responsibility… There is such a wealth of information packed into this course! I was completely engaged throughout the entire 3 day course, and in that sense it was exhausting.

Everything in this class is a trap – Dean Mackenzie

As a participant I felt like a contestant on QI – if you answer questions with the first thought that springs to mind, expect to hear the Klaxon. Only instead of a klaxon, you’ll be treated to an extensive set of follow-up questions until you realise how shallow your initial answer was, and relish the opportunity to provide a more thought-out response.

On the second night I arranged dinner with some of the attendees and we talked testing over wine. Then in a fancy Sydney restaurant we started playing the dice game. I love the tenacity I saw from my fellow testers that night.

I graduated from RST with 2 stars on my certificate, and James following me on twitter. I’m proud of both 🙂
I also met some intelligent and enthusiastic testers, had fun, learned a lot, and filled my writing pad with notes which I regularly refer to. I’m thrilled that I decided to invest in my own education and make the trip back to Sydney.

P1130462_crop    JBtwitterRST cert

*I remembered reading another RST grad’s blog where they did exactly this about 10 minutes into an exercise. With the benefit of their experience, I only wasted 30 seconds. Thank you blogger-whose-post-I-can-no-longer-find!

I recommend reading other’s blog posts on their RST experiences when deciding whether to go, and in preparation for attending. Such as the one you’re reading now, and some of these. Martial Tester has a great list of posts.

 

 
1 Comment

Posted by on September 11, 2014 in Software Testing

 

Simplified Estimation

I’ve wasted a lot of my time estimating future testing activities down to the number of days, for minor and major releases of complex systems. Now I’m considering a new approach.

I will review whatever materials are available for the project, speak to stakeholders about their primary values and concerns for the project, gauge the budget for number of test analysts and choose one of the following estimates:
– 6 weeks
– 3 months
– 6 months

The numbers may change slightly of course, but essentially this is the level of estimate breakdown that I’m prepared to provide at the outset.

My reasoning is that any up-front estimate will be rendered inaccurate due to a large number of unknown factors, and past experience at detailed estimation efforts have proven to be futile. I can continue wasting my client’s time and money, or we can take a broader view of estimation and revise that estimate when I’m elbow-deep in the project and have a clearer understanding of the risks involved.

A key success-factor for this approach will be transparency and regular reporting. To be effective in my role as test manager, I need the support of the project manager and stakeholders. They are much more likely to be supportive when provided with a clear understanding of the top few goals, achievements, risks and issues facing the team each week.

So, how can I tell if testing is running on-time or going over schedule? Well, I’d have to ask you to consider – what does that question even mean? If your testing is running to plan (somehow) and you find a defect which causes a decent-sized code re-write, is your testing still running on time? Was it in any way useful to report that you were running “on time”? Instead I report clearly on what has been tested, the most important outstanding issues and the product areas which are yet to be tested.

Another key success factor is commitment to quality and efficiency. Only when the whole team are focused on delivering a quality product efficiently can this approach to simplified\iterative estimation be successful. Simplified estimation is not about laziness, it’s about expending efforts where they will be of most benefit.

Thoughts?

 

 
7 Comments

Posted by on June 8, 2014 in Software Testing

 

Tags: ,

 
%d bloggers like this: