Peer conferences, part 1

Warning

This blog series is mainly for people who (would like to) arrange or attend peer conferences (explained below). People not interested in peer conferences will find rather little value in this post. One exception though could be if you want to arrange some workshop/conference at your work and you want inspiration for that. Finally; you can replace testing/software testing below with e.g. "programming", "change management" or something else as the experiences and methods should work for any "topic", I just happen to work with software testing.

Finally, these are my thoughts, not "the truth".

EDIT: If you want more ideas on the topic of peer conferences, check out James Thomas' blog post "Trying to be CEWT" and James Lyndsay's blog post "http://workroomprds.github.io/LEWT".
  

Peer conference

A peer conference in this case refers to a small group of experts, gathered to debate a specific topic, based on their own experiences rather than e.g. abstract models and they do this in a focused manner for at least one day. "Small group" refers to something like 6-16 people. "Expert" is harder to define but means something like "a person with a lot of skill, passion and/or experience in software testing".

With this definition I realize other events I've attended might qualify but I hope it's good enough to at least understand this article.
 

Data

This section is to help you understand what I base my thoughts on. Feel free to skip it if not interested. Do note that I attend a lot of other "peer conference -like" events such as Transpection Tuesday, local meetups etc. and I will take that experience into consideration but haven't listed it in this chapter for the sake of your (and my) sanity.

• SWET 4
  15 attendees
  James Bach present, lot's of well known names overall
  I was very inexperienced (compared to the other participants)
  Language: English
  Setup: LAWST inspired
  Topic: Models in testing
  Location: Sweden, fancy conference center
  Length: 1½ days (+optional 1/3 day before), Saturday morning to Sunday lunch
      
• SWET 7
  10 attendees
  James Bach present, very inexperienced group
  I was one of the most experienced attendees
  Language: English
  Setup: LAWST inspired
  Topic: Test coaching
  Location: Sweden, fancy conference center
  Length: 1½ days (+optional 1/3 day before), Saturday morning to Sunday lunch
    
• PEST 6
  9 attendees
  Michael Bolton present, the best of Estonia (which is pretty awesome btw.)
  I was an "international attendee" so a bit different
  Language: English
  Setup: LAWST inspired
  Topic: Gaining consciousness
  Location: Estonia, at Nortal (company)
  Length: 1½ days, Saturday morning to Sunday lunch
    
• SWETish
  10 attendees
  "regional peer conferences" with attendees mainly from Linköping and Örebro
  I was one of the most experienced attendees and co-organizer
  Language: Swedish
  Setup: LAWST inspired
  Topic: Exploratory testing
  Location: Sweden, fancy conference center
  Length: 1½ days, Saturday morning to Sunday lunch
    
• EASTish
  8 attendees
  Only attendees from Linköping, mainly from two specific companies
  I was one of the most experienced attendees and co-organizer
  Language: Swedish
  Setup: People brought their own topics so no "formal presentations"
  Topic: Any
  Location: Sweden, at Sectra (company)
  Length: 1 day (optional evening), Saturday
    
• SWET 8
  11 attendees
  Experienced group, rather mixed skillsets
  I was one of the more experienced attendees
  Language: Swedish
  Setup: LAWST inspired
  Topic: Testing that's not testing
  Location: Sweden, fancy conference center
  Length: 1 day (+optional 1/3 day before), Saturday morning to Sunday lunch
      

Size

So far my experience is ~9 is the minimum; below that the amount of conflicting ideas and experiences, which are important, starts to become an issue. EASTish was still a great peer conference but I think that conference would had gotten even better with a couple more attendees. PEST was right at the minimum limit but I personally did not feel the amount of people negatively impacted the quality of the conversations. Of course the people matter a lot in this case; more experienced people with more diverse experiences having a lot of passion and willingness to debate will likely mean you need fewer attendees and vice versa.

My personal upper limit is ~13; beyond that it seems like every single person gets too little time; especially if there are a few very talkative individuals in the group.

I would typically aim for 13 and since people will get sick, can't attend in the first place etc. that might make us end up with 11-12 which seems great. Notice that "aim" in this case does not mean "invite". For SWET 8 we had, for instance, at most 18 invitations out simultaneously but ended up with 12 who accepted and in the end 11 attendees as one had to cancel.

Language

I didn't think of language as much of an issue until I attended SWETish, my first peer conference in Swedish. It helps a lot even in a country where people generally speak pretty good English. To me, using the attendees' native language seems to help people "dare" to share more ideas, there seems to be way fewer misunderstandings and the overall flow is much better.

But it's a balance as using attendees native language rather than a language more broadly spoken (such as English) will limit who can attend... a problem we have in Sweden as well as several top notch testers here don't speak Swedish or at least not well enough (yet) to attend a fast paced peer conference in Swedish.
   

Experts

I have huge respect for people like James Bach and Michael Bolton; they always add a ton of value to a conversation about testing and especially given a format like the one typically used at peer conferences. Also seeing how James really helped a bunch of less experienced testers elevate during SWET 7 was awesome... however...

In Sweden it impacts the language, which I think is a problem (see Language). We also have a lot of talented testers so giving a spot to an expert will naturally stop someone else from attending and/or give some people less room to express themselves. Finally my experience is it steals a bit of focus as some people, knowingly or not, try too hard to impress the expert and/or not look stupid, hurting their overall performance at the conference.

I think inviting someone like James or Michael is amazing if the language is English anyway and/or the group is very experienced (hopefully lowering the "need" to impress) and/or it's hard to attract enough attendees so giving away a spot is not an issue while the expert can act as a motivation for other people to attend... but it's not necessary, you can have an amazing peer conferences without international, or even national, "experts" (e.g. at work, in your city or in you "region")... and I say this from the context: Linköping, Sweden; a city where we have a fairly active and skilled test community, just to make that clear.
   

Attendees

Experience (as in how much "testing" and "software development" the person has experienced/seen/participated in) helps as we base the discussions on experience. But mixing in a few rather inexperienced people can really add some interesting new points of view, as long as these inexperienced people feel safe sharing their thoughts. To summarize: Having a lot of  experience helps but lack of it is not a deal breaker.

Skill (as in your actual ability to test and understand testing) is, for me, key. Some people might not be known anywhere outside their own company, they might not have much experience neither as testers or in talking about testing but place them in a situation like this and they will provide value, as long as they themselves understand that their skill level is on par with everyone else's.

So on the topic of "experts", how much experience and skill does the average attendee need to have to make the peer conference amazing? My personal experience is: "a lot less than many seem to think".

Two other interesting attributes to me are passion and (verbal) communication skills.

Passion helps a lot but I think that usually comes naturally with wanting to spend a weekend (during which peer conferences or often organized) "just talking about testing"... be careful though about attendees who just think "it'll look good on their resumés" or who want to attend to advertise their own services or hire skilled testers.

Communication skills are important in general but do not mistake this for "talkative" attendees. The K-card system often used at peer conferences, for instance, can help less talkative people gather their thoughts and help them get into otherwise intense conversation and it can stop people who think in an extroverted way ("while talking") from filling all "the space". Instead, much more important is having attendees who can express something in a concise way, who respect the facilitator/format (primarily who won't speak when they shouldn't) and who can understand when their comments won't help a conversation forward/add value as well as dare to speak up when their comments will.
   

Invitation

My first ever peer conference, SWET 4, had an invite only format. My second had an open invitation (first 15 to sign up), my third had an open invitation (everyone may send in an abstract but a program board will select who will actually get an invitation among those) and later I also attended a "semi invite only" where ~15 people got a few days head start (personal invitation) until an open invitation was sent out. Long story short: You can do it in many different ways.

Invite only is great when you know exactly who you want to invite and want control over the group. I also find this to be most efficient as people feel selected and thus prioritze the conference more. It also helps to get those crucial first two or three attendees you need to start a buzz about the conference. There's also a risk that the group becomes too homogeneous; resulting in fewer conflicting ideas/experiences and thus fewer opportunities for people to challenge their own models.

Open invitation is great when you don't know the people you want to invite and/or who you want to come. It also relieves you from some "why did she get an invitation but not be" comments and allows you to better talk about the conference before it actually starts. However, it may make it harder to get those first attendees to sign up as they don't know if the group will be good; this can be somewhat helped by e.g. make in open invitation in a large group (e.g. a Meetup group) but where all the members should be good candidates. Also, there's a risk "the right people" will ignore the invitation because they don't understand they qualify or they, for whatever other reason, don't feel like they are the ones you're looking for.

I personally prefer invite only, even when I don't get an invite myself. I think that allows the organizers to better create a group that's suitable for the topic at hand... but that's my personal preference.

There are like I described in the beginning of this chapter many hybrids between the two but I think that should at least give you and idea. Finding the right format to send out the invitations is pretty straight forward for invite-only (email is pretty good for this) but depends completely on your context if it's an open invitation.

In the next part I'll share a checklist and example of what I think you should consider adding to an invitation, so stay tuned for part 2.

P.S.
For SWET 8 we wrote a personal note in each invitation explaining why we wanted that particular person to attend. This was definitely a win-win:

• Participants better understood our expectations
• Participants got to feel good about themselves
• It seemed to make more people accept the invitation
• For us, the organizers, it felt good to tell awesome people why they were awesome and it didn't cost much time or effort.

I'll provide an example of this in part 2.
     

Organizers

My preference so far is "as many as possible but not more than 4" (so I guess 3-4). Communication and taking decisions become a problem as soon as you're two but to me it's still worth the benefits (see below) until you're around 4 to 5. Some benefits of having more organizers are:

1. There are simply fewer people (easier) you need to invite to get a full group
2. The first person to accept the invitation will join an already established group
3. You are much less fragile, if one gets sick/life happens the work can still move forward
4. Larger personal network, key to avoid too many like-minded attendees when using invite only
5. Greater presence in social medias etc., key when using an open invitation
6. You have more options considering location, food etc.
7. More people mean each person needs to do less... and I'm lazy
8. It's easier to identify who you want to invite since you better know what's missing in the group
9. Each organizer can relax more during the conference (less pressure on each)
10. A lot less risk as with one sick organizer a peer conference might collapse if only having one or two organizers but it's much easier to handle if 3 or 4.

If the organizers can meet in person I think it's a benefit but having a good communication platform (e.g. Slack, Skype etc.) should be sufficient. For SWET 8 the only physical meeting we had together was the evening when we decided we wanted to organize a test conference, everything else was handled via Slack.

      

Format

All peer conferences, except EASTish, I've attended have used basically the same format:

• All attendees prepare ~20 min presentations based on something they've experienced.
• A few attendees will actually present, usually 2-4, for a 1½ day peer conference.
• After an experience report there's a facilitated discussion around that presentation. The discussion will continue until the group feel done with the topic (usually 1-6 hours).
• At some point there's time for lightning talks: ~5 min talks including open season.

While I think this is a great format I think other formats could work just as well and potentially even better as they have been explored less.

Two suggestions:

• Discussion topics and dot voting instead of presentations
• Solving an actual problem (doing something); with one or more debriefs

At EASTish rather than presentations, attendees got to write down a few topics each (typically in the form of a question or short scenario) and then we dot voted. This basically turned it into a prolonged lean coffee. There were some cool benefits to this format:

• People got help/got to discuss the exact topics/questions they were interested in
• Much less time to prepare for attendees
• People who feel uncomfortable to present didn't have that distraction 

Drawbacks could potentially be more abstract content rather than focus on experience (not my experience from EASTish), if attendees think the format means they don't have to prepare that may negatively impact the topics covered and the presentations (or rather preparation work needed) may act as a useful gate keeper; scaring off people who want to attend for the "wrong reasons". I don't know if any of these drawbacks are actually valid but no matter what; this is a format I would love to try at a peer conference fairly soon; with or without a specific topic/theme, but probably with.

The other suggestion is something I have tried at a local meetup where we split into smaller (mixed) groups, tested a specific application and finally spent a long time debriefing our testing including why we did the testing we did, how we organized ourselves, differences between individuals in the team etc. The idea has also been used at at least one peer conference before: PEST 4.5, where they tried to visualize various reports in testing in new and creative ways. I highly recommend reading about PEST 4.5 and ever since I heard about it I've wanted to attend/arrange something similar.

If you have ideas on other formats that could be useful, please comment and I'll add them to the post. This is one of the areas where I think we could really take the concept of peer conferences to a whole new level!
   

Topic/theme

Peer conferences typically have a topic/theme. This topic/theme is important to keep discussions focused, to get the right people interested in the conference and to help people prepare before the conference.

Generally topics should be broad enough to include some diversity but narrow enough so that we can actually get deep into the topic. You have several examples in the Data chapter and if you search for "peer conference software testing" you'll find more. One issue though is that the one liner rarely tells the full story as there are usually specific aspects you're interested in (the one-liner is too broad). To get a better idea what I mean, do check out the invitation for PEST 6, it explains their topic "Gaining consciousness" in a great way.

       

Presentations

This is relevant if you let people present, skip if you want a different format.

First of all, set a clear deadline for when you want attendees to inform you what they want to talk about. Informing you about what they want to talk about is typically done in the form of an abstract (~1 page description of the talk).

When you know what they want to talk about establish a speaker order and if possible, set who will facilitate each talk. You can read more about facilitation in the story behind K-cards. I'll try to share how I do this in some later part but if I forget, feel free to ask me.

If your peer conference looks anything like the ones I've attended you'll likely fit 2-4 presentations into a 1½ day conference, so my advice is to inform the top 5 participants (1 extra to deal with a potential cancellation) and let the rest focus on lightning talks.

I also recommend having appointed mentors (typically organizers, e.g. the facilitator) available for each speaker. This can help both experienced and inexperienced speakers present the "right thing" (an actual experience rather than some abstract concept).
    

Duration

Every time I've attend 1½ day peer conferences, I feel like half the group says "This sucks, I would like to continue" and the other half says: "I loved this, but now I need some sleep/think for myself" at the end. I don't know if that means the length is perfect or too short (or even too long) but I think it's a sign the length is quite good.

However, to avoid stagnation (few new ideas/low energy) when going beyond a day my experience is you need a larger group than ~10 and/or very passionate/skilled/experienced attendees. I for instance felt a bit of stagnation during SWET 7 but not during SWET 4 and SWET 8.

I would love to try a full 2 day version but when trying to find a good schedule I run into problems. If you want to avoid missing too much time from work (problem typically for consultants) you could either start by lunch on Friday and end by lunch on Sunday, this would cost one extra night (compared to the typical Saturday morning to Sunday lunch setup) and interfere with working hours especially for people with a long ride to the conference... or you could start Saturday morning but end Sunday evening instead, so you basically end after dinner (say everyone leave ~21:00). This would not cost an extra night but add quite a bit of conference time. Would suck though for people having e.g. a 3h+ drive home (common in Sweden)... Maybe end at say 17:00 or 18:00 and skip the extra dinner as part of the conference... I need to think about this a bit more.

The other option: Shorter, means you can cut costs (e.g. no nights) and make it simpler for people to "spare the time". We did this in Linköping (city with ~140k population) and it work out nice. I don't think shorter is an option when attendees have more than say 30mins to the conference but worked great if you want to introduce the concept of peer conferences to a more local group.
   

Facility and food

Facility and food are actually quite important since attendees will like sit for long durations and be exposed to a lot of information. When selecting a place to host the peer conference, take into consideration: costs, food, quality of the conference area, how much you have to fix yourself and facilities to use in the evenings.

For the evenings it seems like you want one of two things: An inspiring area to sit in (beautiful, unusual/creative, enough space etc.) or a relaxing pool area; e.g. a large outdoor jacuzzi... some alcohol (like a beer or two) also helps.

Being in an area that's great for taking a relaxed walk or jog also helps in my experience as people need some air after more or less a full day in a conference room. For this, choosing a location that's somewhat remote seems to help; this also has the benefit of helping people to fully commit to the conference as there are fewer distractions.

A conference center will greatly up the costs but also significantly lower the amount of work for the organizers. I've attended peer conferences hosted both at conference centers and at someones company, both work equally good to me as long as the organizers, in the latter case, have a good plan for e.g. food, evening location, some energy refillers (=candy/sweets) for the breaks etc.

Schedule

1. Make sure there's a schedule
2. Make sure the conference center, food catering etc. agree to your schedule
3. Be flexible; not interrupting a good discussion is more important than sticking to the plan
4. Schedule regular breaks but remember to not interrupt good discussions (see previous)

Evenings

During the evening(s) a lot of important processing, bonding and follow up discussions take place. Make sure there are good facilities for these, that attendees stay (except for the need of sleep or handle social overload) and that there are some "conversation/activity help".

"A good place" and "making attendees stay" were described in the "Facility and food" chapter above so let's focus on "conversation/activity help". A lesson learned (for me) during SWET 8 was I think the group benefits from being split up a bit in the evening. One simple example is having one or more tables devoted to e.g. the dice game, coin game, Test Sphere or Set as this will split up the group. If there's a pool area the size of the jacuzzi and the fact not everyone like spending time in a pool will automatically split up the group (not necessarily in an optimal way though, but hopefully good enough).

Other examples could be to actually schedule activities in the evening. One way would be to split into smaller groups doing some task, challenge or activity and then, in a simple and informal way, let the groups debrief their results to the rest of the attendees (either in the evening or the next morning). Another would be to set specific topics/tasks at different tables so people can rotate and discuss/do different things with different attendees. Be careful about ambitious plans though; it seems like as long as you provide a somewhat quiet area where people can easily split up into smaller groups themselves; you're basically set... but some help rarely hurts.
   

Summary

I think we can take the concept of peer conferences even further if we dare to challenge the current common setup by e.g. trying new formats, longer/shorter conferences, tinkering with the group we invite, try new locations etc. For instance my view of the "minimum viable product" for a peer conference (location, setup etc.) was significantly altered after I had attended PEST 6 which was the first peer conference I attended that wasn't hosted in a fancy conference center. PEST 6 then became important inspiration for how we arranged EASTish here in Linköping.

What's next

Part 2 will be a checklist for organizing a peer conference including an example of an invitation etc. The goal with this post was to help people learn new ways of organizing a peer conference, part 2 will hopefully inspire new people to organize them as they learn it's not that complicated.

Part 3 will deal with stuff related to the actual execution of the conference e.g. facilitation, check ins/check outs etc.