31 October 2012

Practice: Note taking

Yesterday I decided I wanted to improve my note taking skills (my finance stopped trusting my memory a long time ago and it's time I do the same). I also decided books, blogs and similar was forbidden ground cause I constantly find myself reading instead of practicing when I try to learn something, which I don't think is a good thing. So I picked a presentation, in this case Elisabeth Hendrickson's keynote from Cast 2012 (The Thinking Tester, Evolved), and forced myself to take enough notes to be able to retell her key points a week later. At the bottom of this post I've added my actual notes as pictures, not very easy to read but hopefully good enough (with my handwriting when writing fast, I don't think any picture quality would make a difference though)

First a few general realizations:
  • It was fun!
  • It was easier than I thought! I don't know if this was because it was fun, due to the specific topic or some other reason but I guess time will tell.
  • It felt efficient! I have a hard time imagining it would be possible for me to learn as much in the same amount of time by reading (for this topic I should add).
  • Just like reading makes you practice other skills, like focus, this was an efficient way to practice several other useful skills for instance:
    • Observation
      Both during and after I observed everything I did in a conscious way.
    • Reflection
      From the observations I draw conclusions and tried to figure out what I could learn from these conclusions.
    • Focus/Discipline
      This was, for me, a much more powerful way of practicing focus than reading.
    • Listening
      In this particular practice I soon noticed my way of listening to what Elisabeth actually said got better and better leading me to believe this could be a powerful way of practicing listening as well.
  • I feel motivated to do it again!

So what did I learn about my way of taking notes (the quick version):
  • I'm not bad at taking notes when I really try, my problem is rather that I don't try
  • I tend to structurally close sections (for instance adding borders) before they are really finished adding some interruption in flow when I have to fix this.
  • I change style when I mentally change section. That's actually quite useful since it makes different parts easier to distinguish. Compared to my borders above it's much easier to come back to sections when style, rather than boxes, differentiate them.
  • I like to headline my sections, sometimes that becomes counterproductive, for example I sometimes make up headlines before I really understand what a section is about making my final result confusing.
  • In the beginning I wrote down stuff as I heard them, the more into the note taking I got the more I listened to a whole section before adding notes, this made the notes more relevant but sometimes I missed out on details. In most cases I would prioritize relevance, but I have to be aware it's not always the best choice.
  • I organize my notes on paper as I organize the information in my head. I have a hard time writing down my notes in a more "reading structured way". This could sometimes be useful (I believe) so trying a similar practice with a more "live blogging" approach would be very interesting. Also, this could be an area where reading would be really helpful. I've not read so much of Markus Gärtner's work yet but I've understood from comments he's the king of live blogging so that could be one place to start...
  • I believe some parts could have been better represented with visualizations. This is definitely an area to improve, maybe by doing a similar practice with just different constraints.
I'm pretty sure more lessons will be learned tonight when I sit down and look at my notes again (only did that briefly yesterday) but no matter the outcome tonight, this feels like a success! Hope it can help someone else as well.




... and by the way, right now retelling key points next week seems really within reach...

17 October 2012

The context driven ad

Would you rather have...

... your doctor say:
Nothing is broken, just rest
-or-
Surgical suture, a cast and amputation, you can never be too sure

... your soccer coach say:
Due to their strong central defense...
-or-
4-4-2 like always!

... your kid say:
Will you teach me that?
-or-
Fuck you, school 's over!

... your mechanic say:
Of course I'm responsibility for my work
-or-
Broken? Sorry but according to...

... your scout leader say:
Let's put out the camp fire
-or-
Step 1, look for trapped survivors...

Would you rather have a:
Context driven tester
-or-
Not

(if anyone reads my posts, feel free to comment with your own favorites)

12 October 2012

Test certification thoughts

Expiring date
Skill diminishes over time if not properly maintained, or at least I strongly believe that. Since a certificate says nothing about how the receiver develops afterwards it's usefulness as proof of anything diminishes over time. What would you today score on tests you passed in school? What's the value of a several years old certificate?

Memorizing information
The standard examination format is about memorizing all the, to the questionnaire, correct answers. That is not a proof of any skill but the skill of memorization.  Hopefully you've picked up stuff along the way but still, from a test skill perspective, nothing is proven.

What's the educator's target
An educational dilemma is schools being compared based on the average grades they issue. For the pupils, higher grades mean more options for higher education and jobs which is often higher valued than actual good education. I imagine the dilemma exists for certifiers as well. The certificate is what is used when applying for jobs so getting the certificate could easily become more important than actually learning anything. So would you as a certifier focus on teaching how to pass the exam or how to become a great tester?

Memorization vs Understanding
First, read the amazing post About learning by Janet Gregory.

I'm very interested in kids' learning. One thing I strongly believe in is, if you want to build understanding, helping kids finding their own solution rather than providing a solution is key. I also believe the same goes for educating testers. Providing answers is not very educating, coaching testers to think and question believes is. Certificate training is unfortunately the former, it's all about reading and accepting other peoples strategies in testing. Don't get me wrong, I still think there's a value in knowing what highly valued people think but only as guidance to your own solutions which are based on practice, critical thinking and experience.

Paying for what is free
Why pay for something that is free? What, apart from a silly paper, do you get by taking the exam? The information is free. You can even find tons of questions available online for free. In reality it means paying for nothing that helps you improve as tester. Might be reasonable for someone searching for a job (since it works) but companies? Send out a link to the syllabus and invest your money in a proven test coach/teacher instead of paying for something that is only valuable when looking for a new job. By the way, donations are a great way of funding great work so I'm not against the paying in itself just what you actually pay for.

What to look for as a recruiter
Certificates are probably nice, if nothing else it shows the person wants to work with software testing. However, a blog, twitter, recommendations, publications, experience, enthusiasm etc. are all way more valuable in my point of view.

ISTQB and certified testers
I do not agree with the ideas presented by ISTQB but that doesn't mean they are wrong or not valuable (or at least I can't tell). I'm also not saying testers with a certificate is worse testers than those without. The only thing I'm questioning is the value of a certificate. What does it really say about a tester's skill? My opinion: virtually nothing.

10 October 2012

Why I've signed up for RST instead of buying a kick ass sofa and new computer

I tried to get this course paid by my employer but due to budget cuts and the fact another testing course was ordered to the company it was rejected. Instead I paid for it myself. So what drives me to this:

Most expensive things bought so far
  1. My current apartment
  2. My current car
  3. Previous car
  4. Previous car
  5. RST
  6. Stroller/baby buggy
  7. TV
  8. Sofa
  9. My fiancee's computer
  10. Bed
So why RST?
  • Because the testers I admire and know in real life praise it
  • Because the testers I admire and study recommend it
  • Because James Bach has been the single most important person in my transformation from "well it's a job" to "I love this, give me more"
  • Because it's the most respected course in the branch of testing I believe in
  • Because I don't get a chance to be challenged very often at work and that's a fundamental part in how James teach
  • Because I've studied the work of both James Bach, Michael Bolton and Cem Kaner with great interest
  • Because I long term believe this will help me form a career I will really enjoy
  • Because I'm in the middle of making my current work fun (not only for me) and I believe this course can teach me valuable tools for that
  • Because I've never found a negative review
  • Because I believe in skill and experience rather than assumptions
  • Because it's a chance to meet other highly motivated testers
  • Because I believe this will make me a more professional tester
  • ... but most of all...
I've always been fascinated by people who turn what they do into an art, like a waiter who makes you wanna come back even if the food is crappy or the girl who cleans our stairwell who cares more about the building than all of us living here combined. The passion these people radiate and the level of satisfaction they seem to enjoy has always inspired me. That's what I'm aiming for, I don't intend to become better than everybody else, I just want to become the best I can be and inspire others to do the same... and that's something I believe RST can help me with.

... 25 days left.

09 October 2012

Guided by Fun

This is actually a rewrite of my old blog post: Using Fun as your guideline. Reason being that I think the idea is good but it was poorly presented and too connected to our unique context in the old version.

Background
Documentation is generally not fun, so focusing on fun would leave us with chaos
// colleague

This summer we ran an unintended experiment in my team where almost all our time was spent testing and documentation only existed as casual notes not really stored anywhere.

What happened was we couldn't explain to stakeholders what we had done, why and what our current status was. However, we did get a lot of testing done so we pretty much failed to support stakeholders but did support programmers.

Was it fun? No! Why? It was frustrating and we felt unprofessional.

So we added some very basic documentation. When we encountered questions we couldn't answer, we first asked ourselves: Is this question really something we should be able to answer. If yes we looked at what we had to improve to be able to provide that answer, if no we politely explained this (I will touch the "it's not a choice" response in further down).

Guideline
At some point we realized that generally the better we worked the more fun we had. Turing it around would be: The more fun we had the better we seemed to work. Fun was a lot easier to relate to than "better" or "more efficient" so we stated what we felt was fun, asked ourselves if it seemed like a good way to work and, if so, tried to reinforce it. Here are our results:

  • More time for testing = fun
  • Understanding what we do (continuous learning) = fun
  • Being able to explain our testing = required to have fun
  • Knowing what is done and what is left = required to have fun
In practice we try to fulfill the bottom two requirements, in an as lightweight but sufficient way as possible so we can put as much time as possible into testing and learning (association to exploratory testing anyone?).

It's not a choice
When you don't really see a point in answering a certain question the response might be:
It's not a choice

Without going into reasons for that, both valid and invalid, let's talk about how to approach it. First, never (if possible) ask for what information someone wants, ask what questions you should be able to answer. This not only helps you find alternative information that might be sufficient but it's also something not too often asked so the stakeholder can't just go back and use the default answer.

If this isn't enough try to understand what the information/answer is used for (by asking questions, not by making assumptions). This may:
  • Make it less frustrating to provide (since you understand its value)
  • Help you find a better alternative
  • Make the stakeholder more aware of gain compared to cost
Summary
Always trying to enforce things that make testing more fun is not a solution to every problem, it's a heuristic, it can be very useful in many situations but not all. Please also notice the opposite of fun is boring, not serious or professional. Instead I claim doing something in a boring way when there exists an equally good, or better, fun alternative, that is truly unprofessional.

02 October 2012

An interesting set of skills

Look at the following set of skill:

  • Ability to see unplanned ways of using a function
  • Ability to see beyond obvious functions and attack "passive parts" as if they were functions
  • Curious
  • Continuing even though something is "generally considered finished", to see what happens
  • Attacks any problem with an open mind
  • Doesn't assume a common function necessarily works as it's commonly expected
  • Won't give up just because something seems complicated or someone says it's impossible
  • Finds great pleasure in finding new unexpected behavior i products
  • Always have an idea about what to test next
  • Deliberately and constantly practicing presenting any achieved result and how it was achieved
  • Love to present ideas without demanding others to use them
  • Always willing to help
  • Creative
  • Systematically covering function after function and when "done" unfocusing, looking for any abstract/general functions that might have been overlooked when looking at the details.
Sounds like the description of a great tester doesn't it?

So who is it?

A random one year old kid I observed while waiting at the hospital of course.

Think we have stuff to learn from her?

I do!

Think we're born with a huge amount of testing skill?

I do!

So where did it all go wrong...?

...

01 October 2012

Using Fun as your guideline

This post is rewritten, please read the new guided by fun post first.

Very interesting statement from colleague after my presentation on Exploratory Testing long ago:
Documentation is generally not fun, so focusing on fun would leave us with chaos

Last spring I started on a new team as part of an organizational change. The goal was to create something less cumbersome and depressing than our current waterfall implementation. In our old way of working tester's have had time to prepare detailed test cases with instructions and god knows what information.

Things just went bad. We were standing in a situation where a poorly written strategy was all we had while code was handed to us: "Please test so we don't have to wait for months before faults are discovered and we have to dig into code we already dislike" (like we did before) plead the programmers.

So we did the only reasonable thing: We tested, we tested and we tested. Life was great! No public documentation! Success!

Time went by and the first hand over was made between me and the other tester in the team due to vacations. Chaos! I knew to some extent what was tested but I couldn't really say if anything was "done". Soon after he handed the work back to me as he went on vacation. Chaos once again. The team even started to ask us for test cases just to get some kind of idea of what was tested.

Was it fun spending all our time testing? No! Why? Because it lead to tons of frustration and a feeling (a well motivated one) of not being professional as we couldn't answer fundamental questions about our work.

Fun as guideline
Today we use fun as our main guideline in our internal test process.

These are our criterias for fun:
  • More time for testing = more fun 
  • Understanding what we test = more fun 
  • Knowing what's been done = basic requirement for fun 
  • Knowing where we're heading = basic requirement for fun 
  • Being able to answer questions about our work = basic requirement for fun 
To summarize: On top of actual testing (incl. learning anything we need to learn), we put a layer of administration, as small as possible, to reach the basic requirements for fun.

In practice
  • More time for testing
    Exploratory testing, removing anything that does not provide value to us, asking stakeholders what questions why want us to answer rather than what documents they want.
  • Understanding what we test
    Experience database, ask questions, ask for presentations, involve programmers in testing discussions, read, discuss, pair testing 
  • Knowing what's been done
    Test matrix (I'll explain that in a later blog post), SBTM inspired team internal test management (still have a lot to learn about SBTM though) and (at least) weekly test discussions
  • Knowing what's next to test
    Compact test strategy, test matrix  and (at least) weekly test discussions
  • Being able to answer questions about our testing
    Test matrix, bug tracker, backlog, test strategy
Things we fell we've achieved
  • We are more motivated than before 
  • We are a lot more efficient than before 
  • We know what we're doing to a much higher degree (surprising result!) 
  • We constantly learn things about testing and our product (= more fun) 
  • Less frustration 
  • We understand the code better and can write our own corrections / add testability ourselves more often (in our case this is a bit special since our product supports injecting assember instructions in runtime). 
  • The test matrix creates more interesting questions than the test case progress reports ever did (once again: I'll write a post on our Test Matrix later) 
  • Minimal waste upon changes. This makes it far easier to work in an iterative way 
  • More lightweight administration makes test more available to programmers 
  • Quicker feedback to programmers 
  • So far we think we've found more important bugs this way, especially important bugs that are indirectly a consequence of what we've implemented (impacts we could not had anticipated) 
  • We take responsibility for our testing to a much higher degree ("we probably triggered that case" is not enough anymore, instead we learn how to do it, implement the necessary testability or admit it's not tested and explains why). 
  • We've stopped considering issues as something we need to "revert from"/work around and rather consider them as a chance to learn and improve. We've also learned that we can't really know for sure what we actually need before we've lost it a valueable lesson.

Testing your own code

There is a different mindset in programming and testing, as a programmer you want your code to work while as a tester you want to discover in what ways the code is not working. Sounds sane, right? Well, I'm starting to believe this is not necessarily true.

Lately I've been developing a lot in PHP. At one point I thought about the starting statement of this post and realized it might not really apply. I'd noticed I was just as thorough when testing my own code as any other's code, and not only that, I even cared more that my work was functioning well than "I feel I generally care when testing some random product". Why? Shouldn't I simply look at parts and say: "Well, I know that is working so no testing required there"?

Soon after a programmer at my work said something that got me thinking. To give you a quick background, he's a really great programmer that has excelled quite a lot lately in testing as well. Anyway, he was testing his latest work on quite a high functional level, far away from his usual domain. When we talked about testing in general he said: "I really love this testing business! Code is the one thing I produce and when I hand it over I want it to be as awesome as it can be".

That made me realize something important: "I assume people using what I've programmed will judge me personally for it's function and therefore I want it to be as well functioning as humanly possible in the time given to produce it". So rather than "wanting it to work" I "wanted to make sure I found every significant bug there is".

Of course this is just two observations, I know you find what you want to find and I'm aware my first statement often reads as "you should not be the only one to test your code", and I'm not challenging that. But I do think it's an interesting point of view. I believe if you can make every programmer on your team thinking: "I take personal pride in my code working as good as it can in the time given" and combine that with an awareness of test, I think your job as a tester would be a lot simpler.

25 August 2012

What differs between Linux distributions?

This article is a shorter version of my old one in Swedish. If you're interested in the original one, please send me a note.
  • Target Group
    Designing a Linux distribution for a beginner, a office user, a gamer or a hardcore geek is very different.
  • Bleeding edge
    Bleeding edge means support for the latest technology but code is less tested/used generally resulting in more bugs and security flaws.
  • Packet Manager
    There is no "best packet manager", instead there are several with different pros and cons. Mainly however this is an inheritance from the "base distribution" the actual distribution is built upon (most distributions are based on one of the big distributions like Debian, Red Hat or Slackware and these in terms are built around different package management systems). Anyhow, it's a difference between distributions.
  • Interface
    A graphical interface is a core component in modern operating systems. In Linux you have tons of options (just to begin with you have split Window Managing and Desktop Environment) and you can customize most to infinity more or less. One very common difference between distributions is which WM/DE they've choosed and how they've choosed to customize it.
  • Hardware support
    This includes which processor architectures a distribution supports, if any specific hardware is supported, some distributions come with tons of drivers others not and so on.
  • Speed, Looks or Security
    Looks (animations, effects etc.) and Security (encryption, real time watchers etc.) comes at a cost of speed. Different distributions balance this differently, some even go to the extremes like distributions with very lightwieght applications, minimal graphics etc. that provides the user with maximal speed or distributions with every possible graphics effect enabled, HD-res wallpapers etc. to give you an astonishing visual expecience. Most however try to balance this to "at least decent values all around".
  • Preinstalled applications
    Office suites, choice of media players etc. is easily customizeable in Linux but different distributions choose different preinstalled sets, some even come without anything not to "get in the way" of the user.
  • Looks
    Something pretty much every distribution customize. This includes background image, color scheme etc.
  • Completeness
    Some distributions come with tons of applications installed and everything preconfigured while others come bare stripped so that after installation you don't even have a graphical interface. To sum it up: Some distributions try to help and guide the user to a complete set of applications while others try not to get in the way of the user, installing only what's considered necessary for it's target audience.
  • Availability to low level components
    As a beginner you want the system to handle itself, new hardware are automatically detected, settings are made i GUIs etc. If you're a performance geek or just like to have complete control over your system however, you don't want the system to accidently override your tweaks or install stuff you've not choosen yourself.
  • Community / Support
    This generally corresponds to the target audience and popularity of a distribution where more technical ones generally (not always) tend to have communities where you're expected to have a fundamental knowledge about Linux. If asking "too simple" questions you might be handed the RTM answer (Read The Manual). On the other hand you might have beginner forums where too technical questions might be overlooked or quickly swarmed by very basic, easy to answer, questions. Size also matters. Big communities often mean someone knows the answer to your question but it might be easier to get personal help in a less populated community. Once again, this is way to simplistic to really tell the truth, but at least you get an idea.

10 July 2012

Lessons from a one year old - a reflection about reflecting

Some time ago I watched my one year old play with one of those shape sorter toys (put the block through the equally shaped hole to get it into the box). The problem is he's a bit too young so he pretty much tried to bang all the four shapes into the circular hole (which actually worked once for the star shape but that's another story) getting more and more frustrated, especially when all the circular blocks were already in the box. So with a stroke of genius he removed the lid and started filling (and emptying) the box with ease looking up at me to recognize his success. My reaction? "No no no, put the lid back on". A few seconds later I started reflecting upon my reaction. This is the result of that reflection.

Lesson: It's not the receiver's fault when you are unclear
In this case it's hard to know exactly how my son was thinking but looking at his action/reaction my guess is he saw the mission as getting the blocks into the box while my idea was to find the right hole for each block, the getting it in the box was just a proof of correct behavior. So I failed at communicating my model and when he acted according to his interpretation (finding a better solution to my problem) I rejected his solution and more or less degraded him by correcting an error he wasn't even aware of.

Lesson: When someone finds a flaw in what you've communicated, that's creativity, not cheating.
I found this very often to be the case in school. If you are creative, finding a solution not thought of/intended (mostly known as "cheats") you are often rejected. Real example:
In one course we were split into groups and built an autonomous robot. In the spec it said "when hit, the robot must continue straight forward for 3 seconds". So we simply made the robot go forward in 3 seconds (the goal was to send all enemy robots of the arena) but also turned down the speed by a factor of 10 when doing so. My idea of a good reaction reaction from the teacher would've been something like, "Well, you pointed out a flaw in my spec, I'll make a positive note about it but please change back to regular speed forward". His reaction? Something like "Well, now you're just trying to cheat, you know very well what I mean!".

Lesson: You need to monitor if the receiver has actually understood you
When I tried to communicate my view, in this case a very simple one, I failed. Since I just assumed we had the same view I didn't really notice that my kid got happy when the darn block was in the box not when he found the right hole. In retrospect waiting expectantly when he actually found the hole and cheering when the block was in box was not very helpful. Of course my intention was to not reveal the solution but all and all it was just a good intention not good communication.

Lesson: It's important to share the same view
Let's pretend I was in my son's position and could speak. In that case I should have asked a whole bunch of questions, mainly: "Why on earth did you put this stupid lid on the box?", that would have saved both me (him) and the dad (me) the frustration.

Lesson: Models limit our creativity
If someone would have told me to, as quickly as possible, get the blocks in the box I would probably not have removed the lid simply because my model is "blocks go into the box through the holes in the lid". The same task given to my kid would have rendered him removing the lid and hence, beating me. Beaten by a one year old?! Just because I'm too stupid to look beyond my assumptions? And I make big money on my thinking skills every month while he is officially a cost!

Lesson: What's obvious to me is not obvious to everybody else
I didn't even think of the fact that my kid might not share the idea of "how to play with a shape sorter", to me it was too obvious. Of course all, or at least a vast majority, of my colleagues share this view but in that case it could instead be "of course they've thought of handling negative values as input". I think you would go mental if you never were to trust anyone about anything but at least question you assumptions.

Lesson: Assumptions must be under constant review
To ensure we don't end up doing bad assumptions and inefficient problem solving we need to constantly question and review the usefulness and relevant context for the assumptions we make. Since assumptions are shortcuts we fall back on to save us from constant tedious repetitive thinking I guess it's in their nature to be hard to monitor. One thing I find helpful is lateral thinking puzzles, since they by design challenge your assumptions.

Lesson: Courage to question is an important skill
Instead of just going with my idea of putting the blocks through the holes in the lid my kid continued to try to remove the lid. That might not always be the best solution, but at the same time I guess you can excuse a one year old for not starting a sophisticated discussion about our different views on the shape sorter. Anyway, the skill to question someone else's assumptions (and therefore uncovering what might be interpreted as facts rather than assumptions) is an important skill to anyone, one that might get you into trouble in the wrong environment but make you a hero in the right one. If you're in an environment where questioning isn't appreciated you might need to question what you're doing there by the way...

Lesson: Being too helpful is not helpful
I had my best intentions when I corrected my kid and in this particular case it might be useful for him to listen. However helping people too much can lead to less thinking, making them scared to go with their own idea or even stop evolution (as in how solutions to problems can evolve over time). Let people think, let people think and before correcting them, ask why they've gone a certain route, maybe we're more right than you?

Lesson: You need to spot assumptions
Finding assumptions require constant observation. At work I recently uncovered that you could actually simulate a certain type of hardware in our system simply by not just accepting "it doesn't work". Once again, of course you can't spend all your life with trying to question every little aspect of every little statement but to challenge common propositions you've not seen/heard any proof of is, to me, a very healthy behavior.

Lesson: Questions is a tool
Well, I've already touched upon this several times but I think it's important enough to highlight again. Questions are an essential tool that you need to use to clear any uncertainty or ambiguity in whatever someone tries to communicate. "Are you sure that..." is often a good start or, if you really want to be considered a genius pain in the butt, go for "why", "why doesn't it work". Never accept something that seems unclear just because you're too afraid to question or afraid that you'll look stupid (hmm, really nice that I write this as a exhortation to you when I'm one of those who really need to work on this).

The most important lesson...?
"You can learn an awful lot from a one year old"? Well, to put it in a more general context: "You can learn an awful lot by just reflecting on an everyday event". I could have come away with tons of lessons like this from reflecting about virtually anything, how I act when I'm cooking, how me and my girlfriend share work at home, how a colleague react to my bug report etc. You could argue some of this is just associations from associations (not immediately related to the example I reflected from) but I guess that pretty much sums up what thinking is (going from one interesting idea to another). You could also argue that a lot of this is just common knowledge and well, it is, just like it's common knowledge not to eat crappy food, don't make up apologies and go to bed in time, sometimes we need reminders, at least I do.

One final question: What else could you learn from this? Well I didn't even start on the psychosocial aspects, parenting in general, I had a lot more interesting associations to school that I didn't bring up, you could question the toy or how it could be redesigned to better set the rules or challenge the user's creativity... well I guess I need to stop otherwise I won't be able to sleep tonight but there you have some more aspects in case you want to continue .)

So to sum it up: Reflect! It's not just a valuable skill to practice, it can also lead to very interesting results.