RSS

Should you take the Rapid Software Testing course?

11 Sep

Yes.

This year I flew to Australia from New Zealand to attend the Rapid Software Testing course with James Bach, hosted by Testing Times.

The course is crammed with practical tips that you can start using immediately on your current projects. James has concocted mnemonic devices that really work.

Practical exercises are a great way to practice and demonstrate what you’ve just learned, and being in the ‘hot seat’ is a challenging and memorable experience. Once James dons his cap the teacher has gone, and the pointy-haired boss has arrived. I highly recommend volunteering for a hot-seat challenge, to help get the most out of your experience.

I’d already done some of the RST practical exercises during a Skype coaching session with James, so I sat out the first exercise… for about 30 seconds. Then I realised* I could pick up where I’d left off with the exercise, taking my learning to the next level. I found my old Skype chat transcripts, pulled up my previous test findings, and continued digging deeper into the exercise. I was even able to channel Rich and ‘Go Meta‘ which I was very excited about.

During the course I also learned transferable skills from James’ tips, mentioned as asides on coaching, critical thinking, presenting, professionalism, accountability, responsibility… There is such a wealth of information packed into this course! I was completely engaged throughout the entire 3 day course, and in that sense it was exhausting.

Everything in this class is a trap – Dean Mackenzie

As a participant I felt like a contestant on QI – if you answer questions with the first thought that springs to mind, expect to hear the Klaxon. Only instead of a klaxon, you’ll be treated to an extensive set of follow-up questions until you realise how shallow your initial answer was, and relish the opportunity to provide a more thought-out response.

On the second night I arranged dinner with some of the attendees and we talked testing over wine. Then in a fancy Sydney restaurant we started playing the dice game. I love the tenacity I saw from my fellow testers that night.

I graduated from RST with 2 stars on my certificate, and James following me on twitter. I’m proud of both 🙂
I also met some intelligent and enthusiastic testers, had fun, learned a lot, and filled my writing pad with notes which I regularly refer to. I’m thrilled that I decided to invest in my own education and make the trip back to Sydney.

P1130462_crop    JBtwitterRST cert

*I remembered reading another RST grad’s blog where they did exactly this about 10 minutes into an exercise. With the benefit of their experience, I only wasted 30 seconds. Thank you blogger-whose-post-I-can-no-longer-find!

I recommend reading other’s blog posts on their RST experiences when deciding whether to go, and in preparation for attending. Such as the one you’re reading now, and some of these. Martial Tester has a great list of posts.

 

 
1 Comment

Posted by on September 11, 2014 in Software Testing

 

Simplified Estimation

08 Jun

I’ve wasted a lot of my time estimating future testing activities down to the number of days, for minor and major releases of complex systems. Now I’m considering a new approach.

I will review whatever materials are available for the project, speak to stakeholders about their primary values and concerns for the project, gauge the budget for number of test analysts and choose one of the following estimates:
– 6 weeks
– 3 months
– 6 months

The numbers may change slightly of course, but essentially this is the level of estimate breakdown that I’m prepared to provide at the outset.

My reasoning is that any up-front estimate will be rendered inaccurate due to a large number of unknown factors, and past experience at detailed estimation efforts have proven to be futile. I can continue wasting my client’s time and money, or we can take a broader view of estimation and revise that estimate when I’m elbow-deep in the project and have a clearer understanding of the risks involved.

A key success-factor for this approach will be transparency and regular reporting. To be effective in my role as test manager, I need the support of the project manager and stakeholders. They are much more likely to be supportive when provided with a clear understanding of the top few goals, achievements, risks and issues facing the team each week.

So, how can I tell if testing is running on-time or going over schedule? Well, I’d have to ask you to consider – what does that question even mean? If your testing is running to plan (somehow) and you find a defect which causes a decent-sized code re-write, is your testing still running on time? Was it in any way useful to report that you were running “on time”? Instead I report clearly on what has been tested, the most important outstanding issues and the product areas which are yet to be tested.

Another key success factor is commitment to quality and efficiency. Only when the whole team are focused on delivering a quality product efficiently can this approach to simplified\iterative estimation be successful. Simplified estimation is not about laziness, it’s about expending efforts where they will be of most benefit.

Thoughts?

 

 
7 Comments

Posted by on June 8, 2014 in Software Testing

 

Tags: ,

Judging the Software Testing World Cup – Oceania 2014

17 May

Preparation
When I was asked if I’d be interested in judging the Software Testing World Cup Oceania event this year, I said “Sure, why not?”. I actually hadn’t heard of it before. From the little information available on the website at the time, I didn’t have a good understanding of the sheer scale of this event.

When I received the first group email communication from Maik I realised that preparations were already well underway. I was overwhelmed at first by the large number of judges who were already involved, and impressed when I recognised some of their names from the online software testing community.

Once we had all agreed on dates for the regional competitions, it seemed like no time at all until the first competition was held in North America. Immediately after the competition there were group emails flying back and forth about test reports, bug details, reproducibility, product owner engagement… I read every email but most of them went over my head. It didn’t occur to me that I could ask Maik & Matt for access to the test reports and the bug tracking tool in order to follow the process more closely. I assumed I wouldn’t be allowed access by the product owners. I also know now that there’s a recorded YouTube stream video of each competition available for public viewing, which I didn’t realise at the time. Mostly I was just feeling overwhelmed and thinking, “We’re next”.

A few days later the Oceania competition was approaching and I asked if the North America judges had any blog posts or ‘Lessons Learned’ to share with the judges of upcoming events. But of course they were still in full-swing of actual judging, on top of their day jobs, and hadn’t had a chance to put anything together yet.

Competition
On the night of the competition, I sent out an email 1 hour before the start time basically asking, “What should I be doing?”.
That lead to some emails from Matt and 30 minutes later I had about 7 tabs open in Firefox on my laptop and I was feeling very confused. We were using Skype, Twitter, Google Drive, YouTube, email, Google Hangouts, HP Agile Manager and SUT concurrently.  I started familiarising myself with SocialText, our SUT. I hadn’t realised I’d be on camera and recorded on YouTube, but luckily I wasn’t in my pyjamas.

Due to some technical issues (read: user error – long story) I was hearing the audio live from Google Hangouts and with a 5 second delay in stereo from YouTube. I wasn’t able to mute one without also muting the other. When I explained the problem Sigge sent me a link to view comments on the live stream without viewing the video, and then I could start to participate properly. I’d lost more than 20 minutes worth of product explanation from the product owner unfortunately. The other regional judge Dean joined just after I did, I think he was having his own technical issues.

For the remainder of the competition, I was Alt-Tabbing my way through Firefox:
– Monitoring YouTube comments and reporting participant questions to the product owners verbally
– Typing up answers to the questions back on YouTube for quick reference
– Constantly switching YouTube comments view from Top Comments to Newest First so I could see if any new questions had been asked (grumble grumble)
– Reviewing the judging categories in Google Drive
– Posting to Twitter, just for fun
– Watching the product owner video stream and chatting to other judges in Google Hangouts
– Reading the bugs that were being raised
– Trying to repro some of the more interesting bugs in the SUT
– Emailing participants who were having technical issues
– I was also sending the occasional one-on-one chat message to other judges using Skype.

About 2 hours into the competition, the participant questions had settled down while they got on with raising defects and writing their reports. As soon as we had a moment to think, all of the regional judges in Oceania were asking ourselves the same question:

If this was so complicated and hectic for 3 regional judges with xx a cup load of teams, how will the 3 regional judges for Asia cope with xxx a bucket load of teams?

Judging
From the outset we had two goals for judging the competition:
1. Judge the participating teams and agree on a winner for the region.
2. Think of ways to make the Asia competition manageable for judges and participants.

Every bug was read by one or more judges. Like the other judges, I tried to reproduce the interesting bugs on my own systems. Before even reading the test reports, I could get a feel for which areas each team had focussed their time on.

Every team’s entry was judged by at least 3-4 of the 5 judges. If the discrepancy between scores was above average, we’d judge that team’s entry again until we agreed on the overall score. This is a very fair approach, and also very time consuming.

Each day after work, the local judges were online judging entries. We didn’t coordinate to judge at the same time, we all just tried to complete the task expediently, around our existing schedules. We could see each other online in Skype, updating the judges notes online, creating strange things in the SUT and updating the status of defect reports. I found it helpful to be able to jump on Skype and ask for opinions and clarifications in real-time.

During the judging I found more than a few bugs in the defect management tool. As a thankyou for their sponsorship (read: because the bugs were annoying me so much), I was planning to report the bugs to HP somehow. But now that I’ve finished using the tool I’ve lost motivation for reporting the bugs to HP.

I don’t like to say too much more about the judging publicly until after the final STWC event in November. I’m happy to talk it through with other judges in our Google Group in the meantime.

After the judging I put together some metrics to show myself once again just how useful metrics are. Suffice to say, there was no metric I could find that correctly predicted the winning team. I didn’t try very hard though. Add a comment below if there’s a metric you’d like me to generate from our competition data, and I’ll let you know if it would have proven useful for judging the quality of the teams’ work.

Tips for Judges
1. Like most volunteer work this was very rewarding, particularly when you find people in your region who are as passionate about testing as you are.

2. This is an excellent learning experience, as you would expect. As with participants, this is your chance to show your professionalism, work with peers in your region, and learn about different approaches to testing.

3. You don’t need to be in the same region as the competition in order to judge it. There are xxx many teams registered for the upcoming Asia competition and I see 3 local judges listed on the website. Are you free to assist with judging? Why not contact Maik or Matt today and offer to help (perhaps include a link to your LinkedIn profile or twitter account).

4. Why not write a blog about your experience? It helps you to capture and remember what you’ve learned, and it may help a judge of the next competition who is feeling unprepared and unsure of what to expect.

5. Have some drinks and snacks within reach (or in my case, a great partner within earshot).

6. This is your regional competition. Be actively involved during the competition, speak up during the live chat, ask clarifying questions…

7. Set aside a few days\evenings after the competition to focus on judging.

8. If you have an idea to improve the judging in any way, speak up to Matt and Maik. They’re experienced in this process now, and will be able to discuss it with you further.

9. Sigge, Dean and I have come up with some ideas for processes and improvements, and fed those back to Matt and Maik for consideration.

Some tips for Participants
1. This competition is intensive and participant’s attention is pulled in multiple directions for the duration of the competition. None of the Oceania teams with 1 member completed their Test Report. Most of the 2 member teams also failed to complete the competition. In my opinion, if you are a team of 1 or 2 people, please consider joining up to form teams of 3 or 4 people.

2. If at all possible, co-locate with team members for the competition. Or if you have video conferencing available at the office, use it. There is a lot going on at once, and ease of communication with team members is key.

3. Bug quality is so much more important that quantity. I could write a whole blog post about this point alone, but I shouldn’t really have to for this target audience 🙂 Brush up on your bug advocacy skills.

4. Have multiple monitors available if you can. At a minimum, try to have one screen where the whole team can see and hear the live stream video of the product owner, in addition to everyone having their own screen to work on.

5. Get familiar with the defect reporting tool in advance. Learn how to attach screenshots to issues, for example.

6. The competition has a strict finish time. That’s the deadline to have all bugs and test reports submitted by, not 30 mins later and not the next day 🙂

7. You can find additional tips on twitter if you search for #STWC2014 (If you’re not on twitter yet, read this and then join twitter).

8. Have fun with your team! This is an opportunity to work closely with colleagues and peers. While it’s a simulation of a high-pressure project with a very short timeline, it is just a simulation.

Good luck!

 
2 Comments

Posted by on May 17, 2014 in Software Testing

 

Tags: , ,

Confidence in your work

13 Apr

If you’re looking for reasons to doubt the quality of your own work, you’ll find them.

At one client, I was managing a team of five testers and working with an agile development team overseas. There was no time of day where the developers and testers were online at the same time. Our primary modes of communication were emails and defect comments.

When the Programme Manager called me into his office and said that my team was adding no value to his project, I was dismayed. All I could think of was that I’d just returned to the workforce after a year-long maternity break, and my first project was an epic failure after just six weeks. My role was to provide information about product quality to the PM, and he had just given me the worst feedback that I could ever imagine giving a team:

It will be better if you do nothing, because you’re distracting the development team with all of your questions. They’re very busy, they’ve got a lot of work to get through for this sprint, and you’re wasting their time.

I was so surprised and upset by this that I excused myself and thought about where I could have gone so wrong on this project.
For about the next 40 minutes, I found plenty of reasons to doubt the quality of my work.

I was working with a new cloud application, and having some trouble completely understanding the backend structure.
The questions we were asking development made sense to me, but maybe we should have added more detail to avoid misunderstandings, in the absence of face-to-face or at least real-time conversations.
I’d been asking for feedback every two weeks and it had all been positive, how could I have missed something this big?
I’d just been off work for a year, and maybe all my instincts were off.

At about that point, I wondered why no one from my team had questioned me on our approach, and whether they’d also been doing anything wrong.

My team had been doing all that they could with the information available during the day.
They’d been sending through a single email of concise questions daily, when our team couldn’t work out the answers amongst ourselves.
They’d been notifying me daily about issues blocking their main priority tasks, and finding other things to test in the meantime.

I came to the conclusion that my team had done nothing wrong, and had been adding as much value as they could given that they were usually blocked for at least half of each day on testing the highest priority product areas.

Then I finally realised, I had been adding value too. Quite a lot of value actually. Perhaps if I’d been at the top of my game, I would have come to this conclusion during the meeting, rather than 40 minutes later.

I had been uncovering major oversights in terms of product development and deployment planning, some of which had a large impact on the project budget.
I’d worked hard to develop solid test plans in the face of ever-changing information.
My decision to use a visual coverage map approach to test planning had provided thousands of dollars in savings, from the effort we saved when changing the plans as often as was necessary.
I had worked closely with the Training Department, to ensure they had early access to the system.
We were regularly blocked by lack of real-time access to the development team, and sent them daily summary emails with lists of questions and requests for clarifications.

I’ve taken away a few lessons from this unpleasant experience. Some are lessons that I’ve learned before, like ‘Get everything in writing’ and ‘Seriously, get it all in writing’.
And that when a team say they are ‘agile’, that can mean just about anything.
And that some people interpret my questions about project gaps as blaming, rather than genuine interest and concern in taking corrective actions, which is their problem and not mine.
But those lessons are off-topic for this post.

A new lesson I’ve learned is to criticise my own work instead of relying on feedback, and not wait to be criticised by someone else. This will improve my confidence in my own work, and give me some practise at promoting and/or defending my test approach. Here’s hoping that next time I hear statements as ridiculous as the ones above, I will react with appropriate confidence and indignation.

UPDATE: Writing this blog helped me a lot. The next time my work was called into question, I responded appropriately with confidence, evidence, indignation and pride.
If I’m ever in a similar situation with a manager who buckles under the immense stress placed on them by stakeholders, and that manager decides to lash out at the team in a moment of weakness, I will defend myself and my team.

 
6 Comments

Posted by on April 13, 2014 in Software Testing

 

Preparation for the Rapid Software Testing Course

16 Mar

In June this year I will be attending the Rapid Software Testing (RST) Course with James Bach, through Testing Times in Australia.

To say I’m excited is an understatement. I have high expectations of this course, based on many excellent reviews that I’ve read in blog posts from testers whom I respect.

What do I intend to achieve from taking the RST course?

  • Improve my testing skills, and add new skills to my testing toolbox
  • Explain my approach to software testing in a professional way
  • Discuss and debate various testing approaches with confidence
  • Approach unexpected challenges with confidence
  • Regularly re-prioritise my testing efforts towards areas of highest risk
  • Learn to grasp the key elements of new concepts quickly
  • Improve my critical thinking skills and questioning skills, particularly when faced with illogical statements that appear reasonable at first glance
  • Write a blog post discussing aspects of the question in my blog URL: Is it good enough yet?

This is quite a list of things to expect from a 3 day course on testing!
From everything I’ve read about this course from past attendees it is also completely achievable, which makes this course unlike any other.

Why didn’t I ask my company to send me? 

This question came up on Twitter. Adding course cost, international flights and 4 days off work including travel time, RST costs a lot of money. Yet I’ve chosen to send myself to the course. Why?

  • Fear of being told no if I had asked my company to send me (i.e. I fear I might find out that I work for a company which doesn’t recognise the innate value of this course)
  • Lack of confidence in my negotiating skills (James Bach is not usually mentioned in a positive way at my company, so I suspect that I’d have some convincing to do…)
  • To prove to myself that I take my career seriously
  • Knowledge that this is a worthwhile investment, and excellent value for money

How am I preparing for the course?

Wow. This started as a short list, and the more I researched, the longer it grew. If I do even half of the things on this list I’ll be a much better tester, without even taking the course! I need to remind myself that they don’t ALL need to be done in the next three months, and practise prioritising.

  • Read some blog posts and resources on satisfice.com
  • Read blog posts about RST experiences (- tick)
  • Lessons Learned in Software Testing – flick through it again
  • Read RST slides and appendices
  • Keep playing the dice game
  • Basically try to prepare myself so that James can’t catch me out. Then acknowledge and accept that I will be caught out, that I will feel uncomfortable, and that I’ll learn something from it
  • Read ‘Thinking fast and slow’ (- in progress)
  • Read ‘Explore It!’ and practice as I read (- in progress)
  • Learn about experiment design
  • Watch 7 Samurai movie (?? This came from David Greenlees)
  • Read “Things that make us smarter”
  • General Systems Thinking – read it again
  • Bill Anders (Also from David’s blog post. “Failure is not an option”? I need to do more research here)
  • Learn some basic Ruby. Everyday Scripting with Ruby: For Teams, Testers, and You, by Brian Marick
  • What are Decision tables?
  • Practise drawing State Models
  • Brush off my dusty Unix shell scripting knowledge (I hope it’s still in there somewhere)

What do I need to remember most during the course?

  • Don’t be afraid to fall into a trap and learn your way out of it
  • Ask for help when you need help
  • Ask questions when you need more information
  • Verify or state your assumptions
  • Don’t get defensive
  • Don’t take anything personally

This post was written by me, for me. If you’ve also received any value out of it – awesome! Let me know in the Comments section.

 
10 Comments

Posted by on March 16, 2014 in Uncategorized

 

Tags: , , ,

Test Tweeting

23 Feb

This is a Lightning Talk which I prepared for the Sydney Testers Meetup but was unable to present due to illness.

It is intended for testers who are Twitter newbies and Twitter virgins. My aim is to help get more testers online, tweeting, and contributing to the online global testing community.

I plan to present this talk at an Auckland Testers Meetup. In the meantime I’m asked about this topic almost weekly while I’m meeting new testers and spreading the word about CDT, so I’m making my unpolished slides available:

Test Tweeting slides – updated May 2014

I’ve had some great feedback already to add information about Lists, Apps and Live Tweeting at events.
Beyond that, if you have any feedback, or suggestions of your favourite lists and apps for Twitter, please comment below. Thanks!

Update: I presented this talk last night at #WeTest and it was a success (as defined and judged by me, arbitrarily).
Thanks everyone who provided feedback, and to everyone who attended last night and asked some excellent questions.

To those of you who haven’t signed up to Twitter yet – what are you waiting for? 🙂

 
7 Comments

Posted by on February 23, 2014 in Uncategorized

 

Tags: , , , , ,

Context Driven Testing in a Bureaucratic Environment

20 Aug

After taking the Black Box Software Testing (BBST) Foundations course, I decided to try taking a CDT approach with my next project. As it happened, my next client project was part of a larger Enterprise release, at a large bureaucratic organisation, with multiple other test teams already using established testing processes and procedures.

I was brought in as the Test Team Lead with 2 Test Analysts. On Day 1, the Programme Test Manager (PTM)  handed our team an incomplete and out-of-date requirements document, which had been prepared by the development team and signed off by the business (yes, really). This document had been approved by all stakeholders, despite the most pertinent section of the document consisting of only a heading, with no content. This is when I really understood James Bach‘s comment about testers having a super power – the ability to read.

By lunch time I was asked by the PTM how many test cases my team would need for this release. I protested the futility and absurdity of the request (I was more diplomatic at the time). Why is the number of test cases a useful measure, when tests might take 10 minutes or 2 days to execute? If we do write a certain number of test cases, won’t that number be ever-growing anyway as testers learn more about the product? How can I predict total number of test cases using only an incomplete requirements document? But in the end, a number was needed, so I offered to make up a number – 300 test cases. I was frowned at, and told that a more complex product in the same Enterprise release had predicted only 250 test cases, and that surely we would need less than them. I changed my answer to 200 test cases and that got a smile. I felt that I had given the right answer. But my CDT approach was not off to a great start.

Note: I first presented this information at OZWST 1 in the form of an Experience Report.

 
3 Comments

Posted by on August 20, 2013 in Software Testing

 

Tags: , , ,

Observation as a source of truth

18 Aug

Seeing something with your own eyes does not necessarily make it true.

In one of my previous testing roles, I was testing a software application which produced
image output files using custom hardware input devices. While I was working with a
developer at his computer, he opened an image file which exhibited characteristics I
hadn’t seen before when testing this product. I went to discuss the abnormality with our
image processing expert (let’s call him Jon), and he requested a copy of the file to help
determine whether this was the result of a software bug or a hardware fault.

I returned to the developer with this request, and watched as he copied the correct file to
a shared network drive. However, when I retraced my steps back to Jon’s desk and he
opened the image file, he saw only random ASCII characters. Jon said that this was not
a valid image file, and for a while I insisted that it was…

Eventually I returned to the developer’s desk seeking support, only to find out that he
could no longer open the original file on his computer either. Why? It turned out that his
computer had malfunctioned, and the file had become corrupted sometime between us
viewing it and him copying it to the network.

At this point I wondered why I had been questioning one of the country’s leading experts
on digital image processing, over something so basic as whether or not an image file
was valid. I’d been totally biased by the fact that I had observed the image with my own
eyes, rather than thinking logically about what piece of the puzzle I was missing in this
scenario.

First published here.

 
Leave a comment

Posted by on August 18, 2013 in Software Testing

 

Tags:

BBST Foundations – Will you take the red pill?

18 Aug

In 2011 I had the good fortune to work for Anne-Marie Charrett, as a Test Team Lead. At the time when other testers were already lining up for her Skype coaching services, I had not yet heard of Anne-Marie or context-driven testing. I still thought that Exploratory Testing meant ad-hoc testing, and that it was impossible to achieve good regression testing without test cases. I think Anne-Marie had a glimmer of hope for me because I held ‘Lessons Learned in Software Testing’ in high regard, and I was genuinely interested in software quality.

Bit-by-bit Anne-Marie managed to lure me away from my false-security blanket of regression test cases and test metrics. Over the course of a few months she left interesting articles on context-driven testing on my desk, suggested that I attend the Sydney Testers meetups, arranged corporate funding for AST membership and held a few re-training sessions for our test team. Eventually I took the leap, and enrolled in the BBST Foundations course to help me understand how I could possibly test the product thoroughly without my regression test cases.

Wow.

I have heard this course described as “taking the red pill” and I think that’s apt. I found the course very challenging, and I greatly enjoyed debating testing ideas with testers around the globe. Concepts which seemed logical to me before I took the course now seem absurd. To think that I have announced in past meetings that testing was 80% complete, or that a release would take 6 weeks to test because that’s how long it took last time, all seems naive now.

Now I think I have the drive and support to be great at what I do, although I find myself currently unemployed. So the challenge I’m facing is – can I get hired at a company that wants great testers or test managers who use a context-driven approach, even though I’m not quite there yet in terms of experience? Or do I find a job within my previous comfort zone of standard test practices, which is like a favourite pair of jeans that no longer fit me quite right?

Either way, I’m glad that I eventually took the red pill.

(First published here)

 
4 Comments

Posted by on August 18, 2013 in Software Testing

 

Tags: ,

Love the one you’re with

18 Aug

“Choose a job you love, and you will never have to work a day in your life” – Confucius.

To me this phrase was taunting…

I regularly saw this phrase proven to be true for other people, and it became a background goal for me (which is a convenient term I’ve coined meaning a goal which you think about seldom and do nothing to achieve, eg. Walk the Kokoda trail).

I’ve always enjoyed doing things which I’m good at. Over the years I was successfully advancing my testing career, but all the while I was attempting to figure out what I really wanted to do in life. What do I love to do that I can actually get paid for?

With the help of the online software testing community I have recently discovered that testing can be more creative, engaging and effective than I had thought possible. In the past I enjoyed testing mainly because I was a good tester. Now I look forward to being a great tester because I enjoy what I do.

So the answer to my problem was simpler than I imagined, and it’s along the lines of “Love the one you’re with”.

(First published here)

 
Leave a comment

Posted by on August 18, 2013 in Software Testing

 

Tags: ,

Newer posts →
 
%d bloggers like this: