Practice: Note taking

Yesterday I decided I wanted to improve my note taking skills (my finance stopped trusting my memory a long time ago and it's time I do the same). I also decided books, blogs and similar was forbidden ground cause I constantly find myself reading instead of practicing when I try to learn something, which I don't think is a good thing. So I picked a presentation, in this case Elisabeth Hendrickson's keynote from Cast 2012 (The Thinking Tester, Evolved), and forced myself to take enough notes to be able to retell her key points a week later. At the bottom of this post I've added my actual notes as pictures, not very easy to read but hopefully good enough (with my handwriting when writing fast, I don't think any picture quality would make a difference though)

First a few general realizations:

• It was fun!
• It was easier than I thought! I don't know if this was because it was fun, due to the specific topic or some other reason but I guess time will tell.
• It felt efficient! I have a hard time imagining it would be possible for me to learn as much in the same amount of time by reading (for this topic I should add).
• Just like reading makes you practice other skills, like focus, this was an efficient way to practice several other useful skills for instance:
• Observation
  Both during and after I observed everything I did in a conscious way.
• Reflection
  From the observations I draw conclusions and tried to figure out what I could learn from these conclusions.
• Focus/Discipline
  This was, for me, a much more powerful way of practicing focus than reading.
• Listening
  In this particular practice I soon noticed my way of listening to what Elisabeth actually said got better and better leading me to believe this could be a powerful way of practicing listening as well.
• I feel motivated to do it again!

So what did I learn about my way of taking notes (the quick version):

• I'm not bad at taking notes when I really try, my problem is rather that I don't try
• I tend to structurally close sections (for instance adding borders) before they are really finished adding some interruption in flow when I have to fix this.
• I change style when I mentally change section. That's actually quite useful since it makes different parts easier to distinguish. Compared to my borders above it's much easier to come back to sections when style, rather than boxes, differentiate them.
• I like to headline my sections, sometimes that becomes counterproductive, for example I sometimes make up headlines before I really understand what a section is about making my final result confusing.
• In the beginning I wrote down stuff as I heard them, the more into the note taking I got the more I listened to a whole section before adding notes, this made the notes more relevant but sometimes I missed out on details. In most cases I would prioritize relevance, but I have to be aware it's not always the best choice.
• I organize my notes on paper as I organize the information in my head. I have a hard time writing down my notes in a more "reading structured way". This could sometimes be useful (I believe) so trying a similar practice with a more "live blogging" approach would be very interesting. Also, this could be an area where reading would be really helpful. I've not read so much of Markus Gärtner's work yet but I've understood from comments he's the king of live blogging so that could be one place to start...
• I believe some parts could have been better represented with visualizations. This is definitely an area to improve, maybe by doing a similar practice with just different constraints.

I'm pretty sure more lessons will be learned tonight when I sit down and look at my notes again (only did that briefly yesterday) but no matter the outcome tonight, this feels like a success! Hope it can help someone else as well.




... and by the way, right now retelling key points next week seems really within reach...

The context driven ad

Would you rather have...

... your doctor say:
Nothing is broken, just rest
-or-
Surgical suture, a cast and amputation, you can never be too sure

... your soccer coach say:
Due to their strong central defense...
-or-
4-4-2 like always!

... your kid say:
Will you teach me that?
-or-
Fuck you, school 's over!

... your mechanic say:
Of course I'm responsibility for my work
-or-
Broken? Sorry but according to...

... your scout leader say:
Let's put out the camp fire
-or-
Step 1, look for trapped survivors...

Would you rather have a:
Context driven tester
-or-
Not

(if anyone reads my posts, feel free to comment with your own favorites)

Test certification thoughts

Expiring date
Skill diminishes over time if not properly maintained, or at least I strongly believe that. Since a certificate says nothing about how the receiver develops afterwards it's usefulness as proof of anything diminishes over time. What would you today score on tests you passed in school? What's the value of a several years old certificate?

Memorizing information
The standard examination format is about memorizing all the, to the questionnaire, correct answers. That is not a proof of any skill but the skill of memorization.  Hopefully you've picked up stuff along the way but still, from a test skill perspective, nothing is proven.

What's the educator's target
An educational dilemma is schools being compared based on the average grades they issue. For the pupils, higher grades mean more options for higher education and jobs which is often higher valued than actual good education. I imagine the dilemma exists for certifiers as well. The certificate is what is used when applying for jobs so getting the certificate could easily become more important than actually learning anything. So would you as a certifier focus on teaching how to pass the exam or how to become a great tester?

Memorization vs Understanding
First, read the amazing post About learning by Janet Gregory.

I'm very interested in kids' learning. One thing I strongly believe in is, if you want to build understanding, helping kids finding their own solution rather than providing a solution is key. I also believe the same goes for educating testers. Providing answers is not very educating, coaching testers to think and question believes is. Certificate training is unfortunately the former, it's all about reading and accepting other peoples strategies in testing. Don't get me wrong, I still think there's a value in knowing what highly valued people think but only as guidance to your own solutions which are based on practice, critical thinking and experience.

Paying for what is free
Why pay for something that is free? What, apart from a silly paper, do you get by taking the exam? The information is free. You can even find tons of questions available online for free. In reality it means paying for nothing that helps you improve as tester. Might be reasonable for someone searching for a job (since it works) but companies? Send out a link to the syllabus and invest your money in a proven test coach/teacher instead of paying for something that is only valuable when looking for a new job. By the way, donations are a great way of funding great work so I'm not against the paying in itself just what you actually pay for.

What to look for as a recruiter
Certificates are probably nice, if nothing else it shows the person wants to work with software testing. However, a blog, twitter, recommendations, publications, experience, enthusiasm etc. are all way more valuable in my point of view.

ISTQB and certified testers
I do not agree with the ideas presented by ISTQB but that doesn't mean they are wrong or not valuable (or at least I can't tell). I'm also not saying testers with a certificate is worse testers than those without. The only thing I'm questioning is the value of a certificate. What does it really say about a tester's skill? My opinion: virtually nothing.

Why I've signed up for RST instead of buying a kick ass sofa and new computer

I tried to get this course paid by my employer but due to budget cuts and the fact another testing course was ordered to the company it was rejected. Instead I paid for it myself. So what drives me to this:

Most expensive things bought so far

1. My current apartment
2. My current car
3. Previous car
4. Previous car
5. RST
6. Stroller/baby buggy
7. TV
8. Sofa
9. My fiancee's computer
10. Bed

So why RST?

• Because the testers I admire and know in real life praise it
• Because the testers I admire and study recommend it
• Because James Bach has been the single most important person in my transformation from "well it's a job" to "I love this, give me more"
• Because it's the most respected course in the branch of testing I believe in
• Because I don't get a chance to be challenged very often at work and that's a fundamental part in how James teach
• Because I've studied the work of both James Bach, Michael Bolton and Cem Kaner with great interest
• Because I long term believe this will help me form a career I will really enjoy
• Because I'm in the middle of making my current work fun (not only for me) and I believe this course can teach me valuable tools for that
• Because I've never found a negative review
• Because I believe in skill and experience rather than assumptions
• Because it's a chance to meet other highly motivated testers
• Because I believe this will make me a more professional tester
• ... but most of all...

I've always been fascinated by people who turn what they do into an art, like a waiter who makes you wanna come back even if the food is crappy or the girl who cleans our stairwell who cares more about the building than all of us living here combined. The passion these people radiate and the level of satisfaction they seem to enjoy has always inspired me. That's what I'm aiming for, I don't intend to become better than everybody else, I just want to become the best I can be and inspire others to do the same... and that's something I believe RST can help me with.

... 25 days left.

Guided by Fun

This is actually a rewrite of my old blog post: Using Fun as your guideline. Reason being that I think the idea is good but it was poorly presented and too connected to our unique context in the old version.

Background
Documentation is generally not fun, so focusing on fun would leave us with chaos
// colleague

This summer we ran an unintended experiment in my team where almost all our time was spent testing and documentation only existed as casual notes not really stored anywhere.

What happened was we couldn't explain to stakeholders what we had done, why and what our current status was. However, we did get a lot of testing done so we pretty much failed to support stakeholders but did support programmers.

Was it fun? No! Why? It was frustrating and we felt unprofessional.

So we added some very basic documentation. When we encountered questions we couldn't answer, we first asked ourselves: Is this question really something we should be able to answer. If yes we looked at what we had to improve to be able to provide that answer, if no we politely explained this (I will touch the "it's not a choice" response in further down).

Guideline
At some point we realized that generally the better we worked the more fun we had. Turing it around would be: The more fun we had the better we seemed to work. Fun was a lot easier to relate to than "better" or "more efficient" so we stated what we felt was fun, asked ourselves if it seemed like a good way to work and, if so, tried to reinforce it. Here are our results:

• More time for testing = fun
• Understanding what we do (continuous learning) = fun
• Being able to explain our testing = required to have fun
• Knowing what is done and what is left = required to have fun

In practice we try to fulfill the bottom two requirements, in an as lightweight but sufficient way as possible so we can put as much time as possible into testing and learning (association to exploratory testing anyone?).

It's not a choice

When you don't really see a point in answering a certain question the response might be:

It's not a choice

Without going into reasons for that, both valid and invalid, let's talk about how to approach it. First, never (if possible) ask for what information someone wants, ask what questions you should be able to answer. This not only helps you find alternative information that might be sufficient but it's also something not too often asked so the stakeholder can't just go back and use the default answer.

If this isn't enough try to understand what the information/answer is used for (by asking questions, not by making assumptions). This may:

• Make it less frustrating to provide (since you understand its value)
• Help you find a better alternative
• Make the stakeholder more aware of gain compared to cost

Summary

Always trying to enforce things that make testing more fun is not a solution to every problem, it's a heuristic, it can be very useful in many situations but not all. Please also notice the opposite of fun is boring, not serious or professional. Instead I claim doing something in a boring way when there exists an equally good, or better, fun alternative, that is truly unprofessional.

An interesting set of skills

Look at the following set of skill:

• Ability to see unplanned ways of using a function
• Ability to see beyond obvious functions and attack "passive parts" as if they were functions
• Curious
• Continuing even though something is "generally considered finished", to see what happens
• Attacks any problem with an open mind
• Doesn't assume a common function necessarily works as it's commonly expected
• Won't give up just because something seems complicated or someone says it's impossible
• Finds great pleasure in finding new unexpected behavior i products
• Always have an idea about what to test next
• Deliberately and constantly practicing presenting any achieved result and how it was achieved
• Love to present ideas without demanding others to use them
• Always willing to help
• Creative
• Systematically covering function after function and when "done" unfocusing, looking for any abstract/general functions that might have been overlooked when looking at the details.

Sounds like the description of a great tester doesn't it?

So who is it?

A random one year old kid I observed while waiting at the hospital of course.

Think we have stuff to learn from her?

I do!

Think we're born with a huge amount of testing skill?

I do!

So where did it all go wrong...?

...

Using Fun as your guideline

This post is rewritten, please read the new guided by fun post first.

Very interesting statement from colleague after my presentation on Exploratory Testing long ago:
Documentation is generally not fun, so focusing on fun would leave us with chaos

Last spring I started on a new team as part of an organizational change. The goal was to create something less cumbersome and depressing than our current waterfall implementation. In our old way of working tester's have had time to prepare detailed test cases with instructions and god knows what information.

Things just went bad. We were standing in a situation where a poorly written strategy was all we had while code was handed to us: "Please test so we don't have to wait for months before faults are discovered and we have to dig into code we already dislike" (like we did before) plead the programmers.

So we did the only reasonable thing: We tested, we tested and we tested. Life was great! No public documentation! Success!

Time went by and the first hand over was made between me and the other tester in the team due to vacations. Chaos! I knew to some extent what was tested but I couldn't really say if anything was "done". Soon after he handed the work back to me as he went on vacation. Chaos once again. The team even started to ask us for test cases just to get some kind of idea of what was tested.

Was it fun spending all our time testing? No! Why? Because it lead to tons of frustration and a feeling (a well motivated one) of not being professional as we couldn't answer fundamental questions about our work.

Fun as guideline
Today we use fun as our main guideline in our internal test process.

These are our criterias for fun:

• More time for testing = more fun 
• Understanding what we test = more fun 
• Knowing what's been done = basic requirement for fun 
• Knowing where we're heading = basic requirement for fun 
• Being able to answer questions about our work = basic requirement for fun 

To summarize: On top of actual testing (incl. learning anything we need to learn), we put a layer of administration, as small as possible, to reach the basic requirements for fun.

In practice

• More time for testing
  Exploratory testing, removing anything that does not provide value to us, asking stakeholders what questions why want us to answer rather than what documents they want.
• Understanding what we test
  Experience database, ask questions, ask for presentations, involve programmers in testing discussions, read, discuss, pair testing 
• Knowing what's been done
  Test matrix (I'll explain that in a later blog post), SBTM inspired team internal test management (still have a lot to learn about SBTM though) and (at least) weekly test discussions
• Knowing what's next to test
  Compact test strategy, test matrix  and (at least) weekly test discussions
• Being able to answer questions about our testing
  Test matrix, bug tracker, backlog, test strategy

Things we fell we've achieved

• We are more motivated than before 
• We are a lot more efficient than before 
• We know what we're doing to a much higher degree (surprising result!) 
• We constantly learn things about testing and our product (= more fun) 
• Less frustration 
• We understand the code better and can write our own corrections / add testability ourselves more often (in our case this is a bit special since our product supports injecting assember instructions in runtime). 
• The test matrix creates more interesting questions than the test case progress reports ever did (once again: I'll write a post on our Test Matrix later) 
• Minimal waste upon changes. This makes it far easier to work in an iterative way 
• More lightweight administration makes test more available to programmers 
• Quicker feedback to programmers 
• So far we think we've found more important bugs this way, especially important bugs that are indirectly a consequence of what we've implemented (impacts we could not had anticipated) 
• We take responsibility for our testing to a much higher degree ("we probably triggered that case" is not enough anymore, instead we learn how to do it, implement the necessary testability or admit it's not tested and explains why). 
• We've stopped considering issues as something we need to "revert from"/work around and rather consider them as a chance to learn and improve. We've also learned that we can't really know for sure what we actually need before we've lost it a valueable lesson.

Testing your own code

There is a different mindset in programming and testing, as a programmer you want your code to work while as a tester you want to discover in what ways the code is not working. Sounds sane, right? Well, I'm starting to believe this is not necessarily true.

Lately I've been developing a lot in PHP. At one point I thought about the starting statement of this post and realized it might not really apply. I'd noticed I was just as thorough when testing my own code as any other's code, and not only that, I even cared more that my work was functioning well than "I feel I generally care when testing some random product". Why? Shouldn't I simply look at parts and say: "Well, I know that is working so no testing required there"?

Soon after a programmer at my work said something that got me thinking. To give you a quick background, he's a really great programmer that has excelled quite a lot lately in testing as well. Anyway, he was testing his latest work on quite a high functional level, far away from his usual domain. When we talked about testing in general he said: "I really love this testing business! Code is the one thing I produce and when I hand it over I want it to be as awesome as it can be".

That made me realize something important: "I assume people using what I've programmed will judge me personally for it's function and therefore I want it to be as well functioning as humanly possible in the time given to produce it". So rather than "wanting it to work" I "wanted to make sure I found every significant bug there is".

Of course this is just two observations, I know you find what you want to find and I'm aware my first statement often reads as "you should not be the only one to test your code", and I'm not challenging that. But I do think it's an interesting point of view. I believe if you can make every programmer on your team thinking: "I take personal pride in my code working as good as it can in the time given" and combine that with an awareness of test, I think your job as a tester would be a lot simpler.