Learning about learning by teaching

I've undergone a tough education in software testing:

• 15 months long
• Tests 3 times a week, 7 hours each and in front of a crowd
• If you skip a test you'll have to do it again, typically within 5 days.
• The expected level of competence is: "Good enough to teach others"
• Little or no chance of "redo:s", you better do it right the first time, every time

In other words: I've been teaching a class in software testing.
The intense experience of teaching testing like this has of course taught me tons of things and with this post I want to share the positive effects this particular job had on my own learning. Each benefit (of teaching) comes with a "why I find it useful" and "how you can apply this knowledge in a more everyday context".

Benefit: Curriculum

Why?

I'm not a fan of committing to a plan, especially not when it comes to my learning. However, the education's curriculum did force me to look into many topics/material I would otherwise had skipped due to laziness, lack of interest or lack of understanding (not thinking it was useful to me). Some of these have definitely made me a more complete/skilled tester such as test framing, deeper understanding of test techniques and a better understanding of bias.

How?

• Use this existing curriculum in your own self-education.
• Complete the BBST courses (just for reference: I haven't done this myself)
• Create your own list of topics and focus on these in whatever order you prefer.
  Important though: It's easy to do this in a shallow way, go deep to learn something new!
  Some good resources to guide you: James Bach - Self-education, James Bach - My view of testing professionalism (especially page 7) and Huib Schoots - A road to awesomeness.

Benefit: Go deep

Why?

I've read tons of articles, watched hundreds of hours of presentations/videos and spent a lot of time practically practicing testing. However, I often end up looking at topics in a quite shallow way especially when I find the topic a bit boring (may still be very useful). When you are to talk about a specific topic for just a couple of hours, you're okey, there's little "need" to go deep. When you have to prepare several weeks of study material though, that's a whole different beast! Being forced to go deep into topics has enables me to better question, improve, explain and argue for the various choices I make (for example why I chose to place my testing mission at the top in a certain status report).

How?

• Dig deep into fundamental questions e.g. what is testing, why do we test, what is an oracle etc.
• Look into related topics. Say you want to improve your written reporting skills then look into e.g. rhetoric, design, how people read printed and digital documents, how people interpret colors or tutorials for your word processor/mind mapping tool/whatever. The point is: don't limit yourself to articles specifically about test reporting.
• Set a "topic of the month" and try to become as skilled as you can in this topic. Don't stop because you feel "done", continue beyond that.

Benefit: Giving feedback

Why?

An important part of my job is helping students understand what they do well and what they might need to improve. To do this I have to observe and analyze what they've done, what they think they've accomplished, what actually made them accomplish what they've accomplished etc., all this I have to do rather thorough in order to be able to explain it to them. This helps me create an understanding that is beyond "do this or do that because it works better".
An example of this is when grading various assignments and projects as students, at least on a general level, need to understand what they did good and what they would had to do to get a better grade. If they get the highest grade they need to know why, so they both know what to continue doing and what to improve. As testers we need these kinds of observation and communication skills all the time when working with developers, project managers etc.

How?

• Study your own testing and try to explain why it was good and how it could be improved.
• One area where I've found this pretty easy to practice (can't prove that the practice translates to other parts of testing but I think it does) is watching presentations (e.g. YouTube) and try to give feedback to the presenter. What specifically did she/he do good and bad?
• Study other testers and try to figure out why you find them good/bad testers. Be as specific as you can.
• When testing, try to find positive and negative patterns: "The log entries are (almost) always relevant and well described making my work so much easier" or "The UI components often have poor vertical alignment".

Benefit: Teaching

Why?

Teaching in itself is a great technique for learning. You have to rephrase the content to match your own language, you hear yourself speak about the topic and you get questions pinpointing gaps in your understanding and/or explanation.

How?

• Do teach colleagues and friends about the various topics you've practiced.
• Write an educational article/blog post about what you've learned (you don't need to publish it to anyone to still get many of the benefits).
• Talk at a local test meetup and if there isn't one, arrange one.

Benefit: Peer

Why?

Working with Maria Kedemo and Martin Nilsson have allowed me to get feedback on the ideas I'm about to share, feedback on my interpretation of various topics and someone to speak with when I feel stuck. It has also allowed me to learn from their knowledge and experience of testing.

How?

• Speak with a colleague
• Join a local tester meetup
• Go to a test conference
• Join the active community on Twitter
• Try your own version of Transpection Tuesday (my post, Helena's post: 1, 2)
• More ideas...

Benefit: Observe testing

Why?

I've spent a significant amount of time both observing testers test (as a group), observed testers test (the actual testing done by an individual) and listened to testers speak about their testing. All three exposed me to new ideas and made me question my own approach. It's also interesting because you get to see a specific problem solved in many different ways which helps you understand what actually impacts the result; e.g. "what is the common denominator in these solutions, is there anything I can learn from that?" or "they all had different ways to setup but all ended up with the same solution, which setup worked best/most efficient and can I learn something from that?".

How?

• Pair testing
• Look at other testers' notes, reports etc.
• Do call for and attend debriefs no matter if you use the concept of test sessions or not
• Offer to review things
• Volunteer to mentor/coach another tester; this will enable you to observe another tester as well as get several of the other benefits mentioned in this post

Benefit: Consistency

Why?

To sit down and learn about various topics every day for over a year has definitely added some welcomed consistency to my self-education.

How?

• Create a routine, such as the already mentioned Transpection Tuesdays.
• Follow a curriculum
• Create your own skill development list

Benefit: Questioning basic assumptions

Why?

Explaining fundamental concepts is incredibly hard but rewarding! As an experienced tester I take quite a few things for granted and explaining concepts built on these assumptions to someone without experience lead to wonderful questions like "but why do we need testing at all", "what does it actually mean to test something", "why can testers find bugs if developers who know the code can't (as in why do bugs happen at all)?". Answering these questions without being able to rely on "experience based assumptions" has led to more than a few epiphanies (and a lot of frustration of course).

How?

• Talk testing with people having a different frame of reference (developers, management etc.)
• Talk testing with people who don't work in the industry; for instance try to explain what you do to a relative.
• Teach new testers at the company or teach e.g. developers in testing
• Talk testing with new, inexperienced testers

Benefit: Ask yourself "how do you train this skill"

Why?

Reading and listening is nice but sometimes you need to actually practice skills to be able to learn them. When teaching I've spent a fair amount of time trying to figure out exercises pinpointing a specific skill I want the students to practice or just exercises/projects in general helping students practice relevant testing skills. This experience help me now both when less experienced testers want help learning a skill, when I try to explain/teach something and when I try to teach myself something.

How?

• After e.g. a blog post, YouTube video or book; think about how you can incorporate the new concepts you've just learned about into your own work.
• Try various exercises and try to replicate various experiments yourself; such as: 1, 2, 3, 4; to help kickstart your brain.
• Whenever you're asked to explain something; try to come up with an exercise or experiment that helps demonstrating whatever you are to explain.

Benefit: Getting questions

Why?

I've already touched upon this but getting questions from the students on anything that's not clear to them is incredibly challenging but rewarding. It has helped me realize flaws in my own understanding, forced me to question my own assumptions and challenged me to find new ways to explain certain concepts in.

How?

• Explain concepts to others
• Ask for feedback
• Ask questions yourself; this both inspires others and help you ask questions "to yourself"
• When reading/watching e.g. a book, presentation (video) or article; pause periodically and ask yourself: "what did I just read/watch and what of that is unclear/seems strange to me?"

Benefit: Having the time and expectation to learn

Why?

When in the middle of deadlines, huge backlogs and conflicting priorities it's easy to forget learning. Having the explicit expectation to learn new things has been an interesting experience and I feel confident saying I leave the teacher assignment as a much more competent tester. Spending as much time as I did on learning is not possible in most working contexts but I think "expectation to learn" is the key concept here as it helps making it happen at all.

How?

• Ask your boss: "How much time am I expected (or at least allowed) to spend on education?"
• When doing backlog grooming (if you do this); add learning stories as dependencies e.g. "before we implement the new video player we need to learn a bit about streaming, video formats and performance testing related to streaming". If you end up never having time for these learning dependencies, try timeboxing them to make the expected time invested in learning more explicit.
• Remember learning is a fundamental part of testing.
• Differentiate between the learning that's necessary to solve your current task and learning with more long term, strategic value (e.g. learning more about testing in general, about web security, about test planning etc.). The "strategic learning" is often important to keep you and the company progressing but can easily be forgotten if put in the same "budget" as the task solving learning.

Final word

I removed several additional benefits I had initially included just to finally get this blog post published (it's been stuck in "draft" for over a year) so just to be clear: You can learn so much more about learning by teaching; this is just a mere introduction.

Next step

After a colleague pointed out I'm not 29 years old anymore I had to revisit my About page. While reading the rest of the text this sentence made me stop...

Next step is to spread the knowledge I've obtained and be a thinking practitioner who can hopefully do what James once did for me.

I wrote that sentence over 4 years ago, before any tester outside Linköping knew I existed. So I took a moment just to reflect.

"Spread the knowledge I've obtained"
I've been teaching students in Software Testing at a vocational university for 1,5 years, I've given numerous lectures, presentations and workshops on various topics at conferences (national and international), meetups and at my workplaces and I'm now test coach at Verisure, a role in which I'm basically paid to share the knowledge I've acquired so far. Finally I've, for most of the period, been a fairly active blogger.

"Be a thinking practitioner"
Transpection Tuesday, my blog, my peer conference appearances, my many dinner discussions with top notch testers (most recently Göran Bakken), my (former) activity on Twitter all add to the "thinking" part while my testing and test related experiments at Ericsson, Verisure and Zenterio, as well as my effort during Software Testing World Cup all add to the "practitioner" part.

Most important though: The two are not separate processes. The best example of this is probably Transpection Tuesday. During Transpection Tuesday, Helena and I often discuss a challenge one or both of us have, together we work out potential solutions or experiments to run, we go back to work to try these solutions/run the experiments and finally we share/ask for help to evaluate the results at a second Transpection Tuesday. Like I said, one process.

"Who can hopefully do what James once did for me"
After graduation I got emails from two former students, both made me feel I've accomplished exactly this...

... hmm, is it raining inside or why are my eyes moist all of a sudden...

On top of that other former students, testers and other colleagues (most recently a developer) have all helped me understand my efforts to inspire, guide and mentor have actually made a difference.

It's not without pride I say: In four years I've wildly exceeded my own expectations based on my memories of what I hoped to achieve 2012. Not just my expectations for the next four years but potentially for my whole career. Shit... I need a break.

... pause music...

What's my "next step"

Took quite a bit of thinking but here's my updated "next step", or mission if you prefer that:

I've presented to, spoken with, inspired, coached and mentored testers in testing and quality. I want to continue this but to a broader audience, in ways different from what has been done before and inspire others to present, speak, inspire, coach and mentor as well.

Clarification:
Broader audience refers to e.g. developers, students (not studying testing), managers etc.

If you happen to read this in few years, do remind me to report my status, please.

What's your "next step"

Enough about me, what's your next step?

Testing Education: Curriculum

I promised a long time ago I would write about each of the subjects in the testing education in detail. I've learned that's not as straight forward as I thought. I need to be careful about what I say/imply about the students, course content, exercises etc. (law, policies, respect for the school (content ownership) etc.).

Due to that I lost the energy to do the detailed posts. But, since I receive a lot of questions about curriculum, here's a post describing the curriculum on a more general level. The subjects below are arranged in the same order as they are taught to the students but the items in each of the bullet lists are not sorted in any particular way. Also the lists are not complete in any way, they're basically what I find noteworthy among all the big and small things brought up. Finally from test design and forward it's not as detailed since those subjects haven't been finished/started yet.

Introduction to testing (4 weeks)

• What is testing and why do we test software?
• Note taking and Visualization (mainly mind maps)
• Heuristics
• Basic test design
• Risk
• Oracles
• Coverage
• Bugs and bug reports
• When are you done?
• Tools (e.g. Sikuli, Selenium, VirtualBox, Apache and jMeter)

We started this course by; first thing, first day; give the students a flash game to test, a basic testing mission and a bare minimum of introduction to get started, just to let them get a minimal practical idea of what testing is. The rest of this course was a combination of theory and practical testing; sometimes we started with letting them test and then explained some concept they had used (e.g. oracles) and sometimes we did the other way around; e.g. when explaining risk we first gave a lecture and then let them create a simple risk analysis of an application before they actually got to test it.

The testing in this course was (of course) very superficial and the goal was to introduce many different concepts to give them a "foundation" rather than focus on teaching them one part really well, All in all the students, in one way or another, worked with ~10 different applications including desktop (Windows/iOS depending on their laptop), web, mobile and, in some cases, Linux desktop applications using VirtualBox.

You have to remember that the students in many cases came fresh from high school and/or did not have any technical background so it was just a brief introduction to the concepts and tools mentioned.

Think like a tester (6 weeks)

• Critical thinking
• Lateral thinking
• Heuristics
• Bias and Fallacies
• Problem solving
• Test polarities
• Models and mental models
• Information gathering/learning

The general setup was similar to the one used in the introduction course however, during the "think like a tester" course we added a lot of general exercises (e.g. what can you do with a brick, the alien tour exercise and many other) to compliment the theory and practical testing.

During this course, James Bach visited the class in Malmö and my class in Örebro joined in via video. A great opportunity for the students to see one of the experts often referenced, in real life. The highlight was James testing the same application as the students had tested as part of their examination for the introduction course. Malmö had several visits from prominent testers (thanks to Öredev) but I leave it to Martin and Maria to speak about those.

Project contexts (3 weeks)

• Lean and Agile
• Scrum and Kanban
• Waterfall and V/W-modell
• Outsourcing
• Testing in agile
• Common challenges for testers

The most interesting part of this course was probably a pretty detailed lecture and discussion on agile testing followed up by a big exercise where students were asked to identify risk in various contexts (like isolation etc.) and what to do to mitigate these/solve the problem.

Programming (6 weeks)

• Java
• Automation
• File systems
• Programming theory (e.g. compilers, bin/hex and memory allocation)
• TDD/unit tests

Most of this course was used by the students to program their own testing tool (tool to generate and interpret test data). I'm working on publishing some of them. This has by far been the most challenging course making students work day and night to get their examination applications ready.

Test methods (4 weeks)

• Schools of testing
• ISTQB
• TMap
• Context driven testing
• Myths

We teach a context driven mindset and the primary objective with this course was for the students to learn about other ways of looking at testing. Most focus was spent on ISTQB since it's so common and the students got to read the syllabus in detail as well as discuss its strengths, weaknesses, goals etc. in true "critical evaluation style".

Test design (24 weeks, 50%)

• Test approaches
• Test techniques
• Technology
• Heuristics
• Risk based testing
• Coverage
• Oracles
• When to use what?
• Security testing
• Performance and reliability testing
• Usability and UX testing

This course runs in parallel with the test strategy, bug reporting and test reporting courses described below and is (by far) the biggest course in the education. The goal is to make it rather practical letting the students use their newly acquired knowledge as quick and much as possible, thus the courses will kind of intertwine as it would be pretty wasteful/fabricated not to do test design, test strategy and test reporting when practicing bug reporting etc.

Test strategy (12 weeks, 50%)

• Risk based test management
• Heuristic test strategy model
• Visualisation
• Testability
• Test framing
• Test processes

This is the other course starting this week. The most important goal is to make students comfortable when requested to create or present their strategy.

Bug reporting (6 weeks, 50%)

• Content
• Different receivers/readers
• Bug pinpointing
• Reasons not to correct errors
• Oracles
• Bug trackers
• Style and structure
• What's a bug and what is not?

Test reporting (6 weeks, 50%)

• Rhetoric
• What are we reporting and why?
• Metrics
• How not to misguide the reader
• Style and structure
• Different receivers/readers/listeners

Testing tools (6 weeks)

• Various categories of tools
• Introduction to common tools

The main reason for this course is students in earlier vocal university software testing educations (not taught by us) felt practical testing and a basic knowledge of tools were the biggest parts lacking in their education. Apart from tools, being the last course we will have to spare some time to talk about practical tips and tricks preparing the students for the internship and working full time as testers.

Internship (8 weeks)

To finish off the education students are sent on internship at various companies. If you're interested in taking on a student, feel free to contact me in whatever way you prefer (email, Twitter, LinkedIn, as comment, buy me a beer at some conference etc,). By the way; I've not forgotten about you Karlo, students have been informed.

Learning more

You can read about the education (Swedish) on EC Education's homepage. If you have any further questions or are interested in attending the next iteration of the education (Malmö or Örebro) in September, don't hesitate to ask.

Report from a software testing education

(written in the middle of the night so you'll have to excuse any strange language or grammar)

Background

I'm currently teaching a 1½ years software testing education. The education is situated in Örebro, Sweden, and there's a similar one in Malmö, where Martin Nilsson is the teacher. In this blog post, I just want to briefly describe the education since many have shown an interest in what Martin and I are doing. This post will not go into any depth about exercises or lectures so in case that's your only interest you'll have to wait. The plan is to cover individual subjects in separate blog posts.

The Education

The education has a clear context driven touch to it, which is no surprise considering the teachers and that the curriculum was created by Henrik Andersson. Since it's the first year its run the material is created in parallel with the education. This has turned out to be a lot of work for me and Martin but it's incredibly rewarding.

Teaching style

This is a vocational university education meaning the students are expected to acquire a high level of practical proficiency or, as I prefer to describe it: Become awesome testers. Our approach to achieve this has been to mix lectures, discussions, exercises and a lot practical testing. Exercises both include exercises closely related to testing (like the critical thinking exercise I've written about before) as well as exercises designed to teach some aspect in a more general way. I'll tell you more about the exercises in later posts.

A lot of emphasis has also been put on creating an atmosphere where students feel safe to stand up, ask questions, question material and even, at times, get cross-examined. Considering it's a class of 30 students that's quite a challenge but we're getting there and I must say students are doing their part of the job splendidly.

The Students

While on the topic; the students range from fresh high schoolers to people taking breaks from well paid jobs. So far the students have completely blown our minds! I had 1/3 of the class attending a local test meetup last week in their spare time, their school results are amazing, during the first course me and Martin had to up the tempo (compared to our plan) roughly with a factor of 4 to keep up with the students and there's no sign of anyone slowing down.

So far, just in Örebro, they've sent in test reports to two companies, reported bugs to two more and I'm planning a project with Securitas Direct where they will briefly be part of a live project (focus is on learning project contexts and improve their ability to test but showing off for a potential future employer is never bad as an added bonus). And in Malmö all students attended two days of Öredev (thanks to Maria Kedemo), but I'll leave it to Maria and Martin to talk more about that.

The Subjects

So far two courses are finished: Introduction to testing and Thinking like a tester. I will cover the content, outcome and lessons learned from those in separate posts. A third is underway: Project contexts and finally there's a Programming course before the year ends. Next semester there's Test methods, Test design and techniques, Test strategy, Bug reporting and the final semester consists of Test reporting, Tools and finally there's eight weeks of internship to round up the education.

The Content

How do you set up 20 hours of teacher-led education in 10-20 hours (sometimes in topics you have very limited experience in)? Well, first off, you don't; but together, me and Martin are slowly closing in on 40 hours a week. A few things I've learned:

1. I created content ridiculously quick a few times and that quickly raised my general tempo (a bit like how you practice speed reading), so boring, but practice makes perfect.
2. Since I don't have time to practice what I'm about to say I try to think about key points I want to make and never put more than one on each slide to help me stay focused,
3. Use the community! Ask friends for help, get some coaching, copy or get inspired by other testers' exercises etc.
4. Google image is your friend when you dislike text in slides!
5. When designing my own exercises, I typically start with the one thing I want the students to learn (there are almost always added bonuses but I don't plan for those), From there I try to get to some context where this is very common and finally try to figure out a way to simulate this where I remove distractions or I try to find a way this can be isolated/highlighted while testing. I'll go into this in greater detail in later posts.
6. I force myself to focus on what's important. I often get caught up in, for example, trying to find that one perfect image or real life example but I've learned if I can't find it in 1 minute I will unlikely find it in 30 so I need to (if not critical) reconsider what I'm looking for.
7. I always strive for simple! It's tempting to show off by teaching the students unnecessary complicated things or making exercises overly delicate. They will forget a lot of what I say so time is, in my experience, better spent trying to teach them what's important, in an as clear and focused way as possible and repeating the key things over and over is typically good as long as you change approach (lectures, exercises, practical testing etc.).
8. I often find that exercises can, after run once, be slightly tweaked to either cement what I've been trying to show/teach the students, let them use their new knowledge/test that they've learned from their mistakes or to teach another aspect. And tweaking an existing exercise is typically much less work that to find/figure out a new one!

Sources are anything from articles and books to blogs and experience (from testing, workshops etc.). Finding good, reliable sources is also an area where I've gradually improved. Stories seem to be easier to relate to so I try to use my own experiences as much as possible. To make sharing between me a Martin easier we typically create simple, elegant slides (one or a few images and avoiding text) with a lot of notes. This works well for us when presenting the other person's material as well as for students when they have to read up (speaking about the notes right now). I should say the fact I have a deaf student in my class also is a factor here. She asked me early on to write notes for the slides since she had problems looking at the slides, the interpreter and take notes at the same time. So she should have a lot of credit as well for the detailed notes.

There's no limit to what I could say about creating content but let's stop there.

Company contact

Being an education requested from companies in the area, it's pretty natural to have a close cooperation with them. So far I'm still building my testing network in Örebro (I live, and typically work, 12 kilometers away, in Linköping) but I'm getting there. The goal long term is to get students connected with various companies long before the internship starts. We'll see how that goal turns out.

Internship

If you keep reading about the education and think it sounds interesting keep the internship in mind as many students are open, or even actively seek, to work somewhere else than in Örebro. Mostly in Sweden (Gothenburg, Stockholm, Västerås, Karlskoga/Karlstad and Helsingborg) but also abroad such as Australia, Austria, USA (especially San Francisco and Columbus) and Russia, just contact me if you wanna know more or feel like you could use one of these students at your company. The internship doesn't start until October next year but never hurts to get the connection done well in advance.

Wrap up

So that's the boring part, next up is "Introduction to testing"; what, how and why we taught the students the things we did and what we (Martin and I) learned from giving this course.

By the way: The education use #YHTest on Twitter in case you want to see what the students have created (used sparsely so far).