My Learning - Part 12 - Experiments

Why

• Experiments are great at highlighting incorrect assumptions I've made.
      
• The experiences/learnings I get from experiments "stick".
      
• Experiments tend to be great conversation starters, which leads to more people, which leads to... well, I think you've heard me say that enough times now.
      
• I love exploration and experiments are certainly an act of exploration which makes them fun!
      

Examples

I'd say there are two categories of experiments I conduct:

Experiments testing a hypothesis and experiments where I don't have a clear expectation, the activity I test simply seems useful and I want to learn more about it.

Example with hypothesis:

I had the hypothesis that when I'm programming very little time is actually spent on writing the final production code. So I picked a specific feature I had programmed under pretty optimal conditions (easy, clear purpose, minimal dependencies etc.) and simply rewrote that code, character by character, into a new file and compared the time between the two activities.

Example without hypothesis:

What happens if I start to get up at 5 AM every morning? How will my body feel after a while? I tried different activities in the morning, I tinkered with different times to go to bed, I altered the time to 4 AM and 6 AM etc. all this while monitoring my mood, activities performed etc.

A few more examples:

• I've experimented with several different ways to take notes
• Sketchnoting
• Mind maps
• Different pens
• Digital notes
• Using colors in different ways
• Different notebook sizes
• Different types of paper
      
• Different ways to read
• Speed reading
• Various memory techniques connected to reading
• Writing summaries for each chapter I complete
• Jumping back and forth between a couple of different books hoping that the variety would make reading more fun (it didn't work...)
• Read super carefully by looking up any word I didn't fully understand, reread sections until I could correctly recall what was in there etc. (worked even worse)
• Only read books with 99 pages or less. The idea being they would better match the way I like to read.
• Basically everything mentioned in Brandon Hakim's excellent Udemy course about reading.
      

What makes it an experiment

At this point it may sound like I call any variation to my approach an "experiment" and to some degree that's true. However, what makes me call these "experiments" is there's a very deliberate method and target with each of them. Before I conduct an experiment I carefully decide what it is I want to try (an hypothesis or an approach that seems reasonable), how I can isolate that activity and how I will evaluate the outcome(s).

So for instance I wouldn't call an exercise someone gives me an "experiment" because I'm not thinking and designing that activity for myself, I simply follow instructions (well, I don't but still...). The same goes if I've forgotten my note taking material and "just take notes" without really having a reason to why I do it the way I do it. This can still teach me something but I wouldn't call it an experiment.
    

How they're initiated

Often when I learn something I want to test how well that thing works for me. Rather than "just doing it" I often try to prepare it like an experiment. For instance before I coached using the NOPRA model I asked myself: "What is it I want to achieve? Why? And how can I evaluate the result?". I then design the actual activity and decided how I would evaluate it.

Another scenario is when I'm in the middle of doing something and realize I'm doing it slightly different than usual. If I'm in the right mood I can stop myself and say "this is interesting, how can I monitor and evaluate this?" (like the dish washing experience described in part 2).
 

Science!

There's no formal process where I write things down, remove all external distractions etc. so few, if any, of my experiment would survive an academic review.

I'm also well aware of e.g. confirmation bias, the Hawthorne effect and sampling bias and that all these will skew my results. So apart from considering this when I design my experiments, which is obviously not enough, I try to also be careful about what I make of the results, especially when I present them to others... I may have gotten carried away more than a few times though.

That being said; the experiments are usually quite harmless and their goal is mostly to help me understand how something works for me, and for that I think my approach is perfectly sufficient to be honest (ignorance is bliss).

Back to the overview

     

Transpection Tuesdays

A couple of weeks ago me and Helena Jeret Mäe sent a few cryptic tweets about "Transpection Tuesday". Since then Helena has written a great summary of our first session and to add to that here are my thoughts so far.

What is Transpection
The best explanations I've found are James Bach's explanation of transpection and Michael Bolton's transcript of a transpection session. But basically it's one person asking questions to another person with the twist that the one asking has already tried to answer those questions. The result is you get two sets of answers, hopefully less biased from each other.

What is Transpection Tuesday
When I first met Helena it was during Let's Test earlier this year. She had tried to grab me a few times but sessions and other conversations had interrupted us. So suddenly during lunch she stood up and said: "You sit here!", and pointed at the chair in front of her. And so I did, which was one of the smarter decisions I've taken as a tester.

Transpection Tuesday happened in a somewhat similar fashion:

Helena: Hey. I was thinking this morning that we could have Transpection Tuesdays (because it rhymes :P) or something. if there's something each of us is trying to work through and solve, then we could try this transpection thing out. It's a bit random thought
Erik: Sounds cool... I'll take care of the kids alone tonight since my fiancé is working so suits me perfect... with the possible problem that a kid wakes up and I have to leave very suddenly and might not come back .)
Helena: I understand that and won't hold a grudge :P
Erik: In that case I'm in! Usually kids are asleep after 19:30 CET, hopefully there's no change tonight

So after some technical difficulties we started a Skype call with video. By the way, video was awesome for this since it made sharing visualization easier, body language is useful when trying to explain something and it helped, at least me, stay focused.

So to summarize:
Transpection Tuesday is simply a Skype session ~3 hours long, happening every Tuesday evening between me and Helena, dedicated to talk about testing.

Confession
Okey, so what we've done so far has been far from just transpection. Instead it has more been like a weekly dose of conferring where we discuss topics that matters to us in various forms; where form depends on mood and topic. You could argue the name thus is misleading, and it probably is... But we like it.

What did we discuss?
Helena shared our combined notes, from the first session. I won't write anything detailed about it now but if something in those notes seems interesting, feel free to leave a comment and I can try to write a future blog post about it (my guess is some of the more "community challenging" stuff would fit that criteria).

How did we discuss?
I recognized five different "styles"/formats used during our talks, plus one suggested that we haven't tried yet:

1. Transpection
   I tried this briefly when speaking about why/if we should send people to courses and Helena did it much more extensively during the last TT when speaking about bias. When talking about these topics one of us asked questions we already had answers to but felt we wanted to improve/challenge. What makes this powerful is the person answering last get slightly less biased since he/she hasn't heard the first person's answers which gives you interesting answers to compare.
   
   
2. Conferring
   Most of the talking was just quick questions being thrown out and then we together tried to answer and discuss them. I would say this is the style we use most (it's like our "go to" style) and for those of you who've been to Let's Test or CAST, this is very similar to the discussions you typically end up in during lunches (which is a good thing!).
   
   
3. Challenging
   I plan to do a lot more of this going forward because it felt like an awesome way to learn. Essentially it's one of us asking the other to clarify something to a great level of detail or challenge a claim being made (similar to what happens during RST for instance). Very powerful as assumptions were uncovered and you constantly had to be on your toes.
   
   
4. Role play
   We take opposite roles (e.g. the manager who questions the professionalism in ET versus a context driven tester considering ET a good approach in the current context) and simply fight it off with the goal to pressure each other to explain/motivate something at a deep level, as well as highlight assumptions. We have actually done this very briefly but I don't think enough to really call it role play, more quick imitations and reactions to that.
   
   
5. Transfer
   Simply one of us telling the other something without any questioning going on; so basically a monologue about a topic. What was interesting to me was the information I received this way really seemed valuable but missing the interactive aspect hampered my ability to connect it to something else/make "lasting connections". Also I often felt the energy was lost a bit when one of us talk for long stretches.
   
   
6. Interview
   Similar to Transfer but more driven by questions so basically the receiver "choosing" what information he/she is interested in. Better than the monologue version but still a lot less valuable than any of the more interactive ways. It was like, even with questions, I didn't get my brain into gears, so new information simply didn't stick as it was when e.g. Conferring.

Improvements
Here are a few things I've noticed:

• Having a clear topic helps as we got fewer side tracks and thus easier could focus on going deep into the current topic. Side tracks are not evil per se, but too many and too big ones seem to make discussions rather shallow (my feeling so far).
• Having prepared followup questions would had been helpful, we have improved this for next week, will be interesting to see the result.
• We did get deeper into the topic during our third TT as we pressured each others to better explain what we meant and why even when agreeing. Think we can do this to much greater level though.
• I need to try to quickly structure the information in my head, before starting long rants/monologues about something.
• I think both of us could benefit from monitoring our own speaking trying to stop ourselves during long monologues. It's much harder for the other person to recognize a good break point.

Wrap up
So, Transpection Tuesdays are not so much about transpection but rather about weekly conferring. I already look forward to next week (preliminary topic: "Arguing for Exploratory Testing") and that's a great sign. I hope this turns into a long lasting tradition and it will definitely be the foundation for many future blog posts!

Observation Exercise / Experiment

Intro
After attending Ilari's tutorial at Let's Test I decided to write a blog post, with the goal to put what he taught us into my own context as a tester. That post is still in a raw draft state but in the meantime, on a similar topic, I want to share a simple observation exercise/experiment I've conducted.

The experiment
For one week I decided to "actively observe" during my 15 minute walks between job and home. I had no idea what "actively observe" would mean but I figured that would be clear as I tried it (which it did).

First walk
I left my house ready to observe. I started to search broadly just sweeping over my surroundings trying to find interesting stuff. I did this pretty much the whole way, finding nothing. That was a disappointment.

Second walk
On my walk home I decided to stop focusing on finding interesting things instead I simply tried to focus on things and see what came out of it. Bingo! Suddenly I stopped as two cemetery gardeners was cutting the grass with big scissors to get a perfectly straight border between the walking path and the lawn. This made me look at the flowers and suddenly I realized how symmetrically they were arranged. This continued as I started to notice interesting grave stones, grave stones I had missed during my first walk when trying to "observing everything".

Rest of the walks
For each day I seemed to lower my walking tempo slightly and I slowly went back to the sweeping strategy except I didn't just try to get an oversight of everything, instead I swept in the sense of rapidly switched focus from one thing to another.

Observations

• Initially my goal was to find interesting stuff (sweep) but that quickly changed to general curiosity (focus). In the end I didn't care about what I would find, the search in itself was way more interesting.
  
  
• For each day I seemed to walk a little slower. Since I didn't clock myself (which would probably bias me too badly to render any useful results anyway) I can't of course be sure, but it felt like I did.
  
  
• In the beginning I seemed scared of missing something important. That basically made me miss everything.
  
  
• When I swept over stuff the only thing I reacted to was very out of place things (which rarely was interesting to discover/observe) or motion. To change this I had to focus specifically on certain details, living with the fact that I probably missed a ton of things.
  
  
• After a few days I started to look forward to the actual walks, they became a somewhat spiritual exercise. Which isn't strange as it reminds me of many mindfulness and meditation practices I've come in contact with.
  
  
• I think I've started doing this kind of "active observations" in other places as well somewhat unconsciously (just noticed it the other day). This would be quite a significant change since I'm very easily distracted and absentminded, I don't really focus on my surroundings.
  
  
• Being tired (two mornings) or annoyed (one afternoon) significantly limited my ability to observe. This is nothing surprising but still interesting to realize for myself.
  
  
• To be able to ignore moving objects when observing stationary objects takes practice, which is also no surprise but once again interesting to realize for myself.
  
  
• The more I practiced the more I figured out what was interesting for me to observe. In the beginning it felt like I just looked at things for the sake of it, but after a while I started to ask questions and build genuine (I feel) curiosity... I also learned it'll take some time to really master this, my 10x15mins only made me realize I have a lot to learn more or less.

Conclusion
Will this make me a better tester? I don't know and I can't think of any way to measure it and even if I could, bias would probably ruin the measurements. But my guess is that some of these lessons and the practices will help, to what extent I don't know though. Examples could be to practice ignoring movement as I focus on stationary details or the need to focus on specific details rather than sweep. All and all it will at least not make me a worse tester (I hope .)

One interesting question after these kind of experiments... will I do this next week, in a month or a year? I have no idea but I hope so. Just like with training it right now feels like it's a sure thing but give it a few stressed days and I might have lost track, we'll see. No matter what; I've learned some interesting things during this week!