The Testing Show: Conferences and Conferring

The Testing Show: Conferences and Conferring

Have you been to a testing conference? Wanted to go? Wondered which ones you should attend? Matt Heusser, Jessica Ingrassellino, and Michael Larsen have been to more than a few as participants and presenters. We discuss our favorites, the pros and cons of various conferences, and what makes each of them worthy destinations to consider.

Also, putting a different spin on the News Segment this time around, Michael shares his enthusiasm for and about “The Privacy Paradox”.







Any questions or suggested topics?
Let us know!


MICHAEL LARSEN: Welcome to The Testing Show. I’m Michael Larsen, the show producer. Today we are joined by Jess Ingrassellino.


MICHAEL LARSEN: And, Matt Heusser.

MATTHEW HEUSSER: Good morning, or good time of day when you are listening to this. We’ve only got three people today. We wanted to do something a little bit different. We want to talk about conferences and professional development and learning, and what you are doing to grow as a tester. Before that, we have the news segment. Michael had a… podcast challenge thing about privacy that he wanted to talk about.

MICHAEL LARSEN: This is a little different from our regular news segments. Manoush Zomorodi and the “Note to Self” podcast recently did a series of programs and an initiative called the “Privacy Paradox”. The Privacy Paradox is how much we give up for the sake of convenience, for being able to use numerous services, and how much of our data is being exploited. It’s a process that allows us to take back a little bit of control over our digital lives. There are five different podcasts. Day one is focused on “What Your Phone Knows” What is the metadata that is being tracked and why does it matter? Day two is focused on “The Search for Your Identity”. In other words, how algorithms see us, sell us, and then sell to us. Day three is “Something to Hide”, which is focused on reclaiming your private parts and why Google needed an in-house philosopher. Day four is “Fifteen Minutes of Anonymity”; the digital gaze, our psyche and what happens when we know we’re being watched. Day five, and this is more of a concept than it is something you can actually do right now, but it’s “Your Personal Terms of Service”: defining your own acceptable conditions for living the good life online. I downloaded Signal and I’m playing around with that. It’ an encrypted chat tool. If you want to use a browser that never stores your search data, shift away from using Google and use DuckDuckGo. Check out the Tor browser. You can install a plugin called HTTPS Everywhere which minimizes what data gets sent out without encryption. One thing you can do if you want to get really hardcore is you can make a faraday pouch.

MATTHEW HEUSSER: Wow. That’s like, literally, a tin-foil hat.

MICHAEL LARSEN: Totally! I understand.

MATTHEW HEUSSER: Did you know I tried to build one of those on a project once?  We wanted to test what happened when you lost signal of your mobile device, so you needed to go in and out of an area with poor coverage. The company we were at, an $11 Billion in sales wholesaler, they kept installing more routers. We’re trying to get service in the elevator… started to work.

MICHAEL LARSEN: [laughter]

MATTHEW HEUSSER: [laughter] Having internet in the basement… started to work. Having internet in that one funky corner you would go to… started to work. We couldn’t simulate failures. I tried to build a faraday cage. You can buy them online; they start round $5,000, that a human can actually fit inside of, and they have a kind of wire mesh that prevents signals. We weren’t going to spend $5,000 plus shipping.

MICHAEL LARSEN: Right! [laughter]

MATTHEW HEUSSER: So I tried to build one out of aluminum foil and a shoe box, s you could put your phone in there and it would be completely encased in aluminum. Did not work. Course, I had no idea what I was doing, but it was kind of a fun way to spend the morning.

JESSICA INGRASSELLINO: Wow! There’s an app called Red Phone as like an ease of entry into private calling. I mean, you have to have both people calling using some kind of encrypted app for the encryption to matter. They some other ones, like TrustCall, which has kind of a higher barrier to entry, and it’s a little pricy. There are some other security apps, but RedPhone is one that comes recommended for people who are not especially tech savvy, but would like to reclaim some of their privacy over the phone.

MICHAEL LARSEN: A specific tool that they mentioned, called Panopticlick, is something that you can use so that you can determine what your browser’s fingerprint is. If you are looking to erase your fingerprints so that you’re not being tracked, or you want to determine how you are being tracked, Panopticlick is a tool that you can use to help you figure that out.

MATTHEW HEUSSER: Maybe we should get a security expert on here and go through their thoughts.

MICHAEL LARSEN: that would definitely be a great show for this, and again, I wasn’t looking to make this into a whole big thing or its own show. I just happened to listen to it last week and I thought “next time I’m on the show, I’d kind of like to talk about this.”

MATTHEW HEUSSER: and if you’re interested in more, I hope everyone knows, our show notes are complete, On the Qualitest web page, everything we say is going to be transcribed, and we’re going to have links to anything we talk about. So let’s talk about conferences. That was our theme for today; “What conferences should you be attending? What are the strengths and weaknesses of every conference?” I’m not sure if this is going to get me in trouble or not, because most conference organizer wants me to say “this particular one is the best! My favorite conference is ‘X’”. I think they all have strengths and weaknesses, so we should talk about them.

MICHAEL LARSEN: Conferences, at least testing conferences, are relatively new to me. I went to my very first software testing conference in 2010. That was the Pacific Northwest Software Quality Conference. I’d been to industry conferences. I went to Interop while I was at Cisco Systems. I don’t know what the scene was like before 2010. If there’s things prior to that, I’ll have to defer to others because I wasn’t part of it.

MATTHEW HEUSSER: My first testing conference, O’Reilly’s Open Source Conference, in 2003. I gave a talk on Test Driven Development (which was, like, new and hot and scary). 2004, I gave a lightning talk at the Open Source Conference, it was five minutes. In 2004, I gave a talk at STAREast on Test Driven Development. It was… thirteen years ago. I was a new speaker and I talked about the economic benefits of TDD and I talked about the process, but I had unit tests. It was Test Driven Development with unit tests, and one of the classic problems is, when you explain TDD, people think acceptance tests. I don’t know why; they’re so clearly distinct, but we’re just bad at describing it. I wish I had said “Test Driven Development is writing unit tests like the code you saw before where you actually have a test harness that automates and runs the methods to make sure that your methods implement what you thought they implemented.” That doesn’t mean everything’s going to work, but it does mean that the program at least has some confidence that what they wrote does what they thought it would, which improves code quality out of the gate. There was such shallow agreement that the audience thought they understood what I was saying, and they completely missed it. So I like STAREast; it’s the biggest show. If you want the big show where people’s eyes open and they go “Wow!” there is more to testing than just take the requirements and turn them sideways and create test cases and then run them”. Testing is a thing, that people can study and try to get good at, and you want some entry level training on it, I think STAREast is the first place to go. Agree or disagree, and once you’ve done that, where do you go next?

MICHAEL LARSEN: I think that’s a pretty fair assessment. I spoke at STAREast in 2012 and that was my reaction as well. I think of STAREast as a good example of, if you’re relatively new to the industry, and you want to get exposed to a lot of different ideas from a lot of different speakers, I think STAREast is good for that purpose. After you have been a few times, unless you are one of the speakers, you are going to see “hmmm, you know, I think I’ve covered these before”, and that something you run the risk of at every conference you go to. Unless you deliberately looking for those completely out of the way talks that are genuinely pushing the boundary are something you don’t understand at all, chances are you are going to be going to talks that are discussing things you are already doing. There’s nothing wrong with that; I think it’s good to go to participate in discussions where you already have a vested interest; “ hey, we are talking about this, I want to see what they are doing, in comparison to what we are doing, and maybe I can bring something home from that”. I’ve noticed that we are getting more targeted conferences. You take a TestBash, for example, that happens in just one day, or you take a conference that has a single track, and you have five or six speakers that are on a very specific topic. And, so many people who talk about conference say this… I think a lot of the best things that you will get from a conference don’t happen in sessions. They happen in between the sessions, or sitting down at the table talking to somebody that you just met and sharing your own experiences.

JESSICA INGRASSELLINO:  I have definitely had the same experience, and I am pretty new to conferences. My first to go to was CAST 2014. My most positive experiences there were talking with people after sessions. And then, having spoken at CAST and STPCon and then TestBash and some of the more local conferences here in the [New York] city, you know that’s been… a different experience, I guess?  The thing that I’ve noticed, and the thing that I’ve really started looking into is the topical interest and the topics, because Michael, like you said, I noticed this starting last year. I started to really take a qualitative and quantitative look. “Gee, it seems like so many things are repeating themselves.” Sure enough, I found like seventy talk titles and was able to group them into six or seven categories. It begs the question; what are we doing for testers who’ve been around for awhile? What is the offer? How are we addressing that?

MATTHEW HEUSSER: I would say I notice that big shows, there’s more and more talks about automation, but they’re still al entry level, and I think that’s just because there’s demand at the base of the pyramid. STPCon for instance has a kind of middle of the road, experienced tester vibe, and you lose something when you get away from entry level. When you’re at the entry level they say “just do it like this and you’ll be fine. Here are the steps.” The higher up the ladder you go, the more the entry level person is going to be confused. “You’re not telling me what to do. You’re asking me questions. What’s going on?” [laughter]. So I woud say that STP is in the middle for that. It’s also a smaller conference, so if you’re the kind of person that makes connections, and networks, it can be more valuable.

MICHAEL LARSEN: What I like about smaller conferences is the fact that the talks that are covered d tend to be more targeted, focusing on implementations. There’s a fine line between discussing concepts that you can do actionable things with, and showering people with code examples that explain what you are doing, and maybe even demonstrate what you are doing, but if you are not really in the position to download that framework and start using it, at least I’ve got an idea for what they are doing with X, Y and Z, and I can apply it using what we have.

MATTHEW HEUSSER: Yeah, if you’re talking about a technical thing; “How to Use Selenium”, you’re not going to figure out how to do it in a one hour talk.


MATTHEW HEUSSER: But the one hour talks will frequently give you a flavor of the process, or give you a feel for the style and the risks. You’re not going to learn the hard stuff of how to use it, and the one day tutorials are rarely much better unless they’re really focused. If you go to a one day session on Test Driven Development, you’ll probably learn it, but I know people who have been to a one day session hands on ‘How to use Jenkins” and they walked away with maybe, sort of, they could configure the simplest of build scripts for a Java app. I guess what I’m saying is, if you’re going to invest in training, go to a day’s training, don’t go to a one hour session. You will not learn how to do the thing.

JESSICA INGRASSELLINO:  I did an introduction, very clearly introduction for beginners, to making a Selenium framework, anything to a hard thing that happens is that attendees don’t necessarily understand, even when you say, “this is what is meant by a beginner, and these are the things that you need to have installed”. 10 people show up, and eight of them really actually got something out of it. They built the beginnings of a framework that they could actually use, but two people didn’t have the computer configured, and didn’t understand how to do basic things on their computer, like they needed to install terminal, they needed to install an IDE (all of which was given in the instructions, mind you). They hadn’t taken the time to contact. So think that’s something, I guess, I could’ve done better, but that conferences can also do better in advertising of the thing, is to really take speakers at their word. You don’t want to deny somebody entrance, but at the same time, when one person doesn’t have those skills, there is a teacher who is trying to get to everybody in the class, and you have that one person who’s like “yeah, so how do I make my computer work?”

MATTHEW HEUSSER: A couple of things there, and I’ve made that mistake, too. I went to a conference in Ohio, CodeMash, and I did the newbie thing, where I just picked my choices; I’ll go to the Hadoop class. I’ll go to the Docker class. I’ll go to this and that, they’re all day tutorials. It’s going to be great.  Of course, I spent the first three hours installing software off of a key fob that I had to wait my turn for because we had saturated the network bandwidth, when we should have all just read the description. So my conclusion after that was that nobody reads the descriptions, and just make your title really clear as a speaker and expect nothing.


MATTHEW HEUSSER: But what I’d like to do is have an install fest the night before where we put Ruby on your box and get it up and running, or…

JESSICA INGRASSELLINO: Yes. I say yes! [laugher]

MATTHEW HEUSSER: So, if you want to do a tutorial, you want to do a talk, and you want people to actually be hands on, which is awesome and we need more of that… please, run an install-fest the night before or the breakfast of, and anybody can come. You’re still going to get people that skip the install-fest, but maybe that will help; prerequisite install-fest for the thing that we’re about to do.

JESSICA INGRASSELLINO:  If you have an install-fest, they can maybe help together, although it is also not their responsibility. They didn’t pay to help somebody that didn’t get prepared, and I can be very frustrating, I think, as a conference teacher because I know that. I’m supposed to serve everybody.

MICHAEL LARSEN:  I want to give some props. In 2015, I was at Agile Testing Days. One of the classes that I was a participant in was doing a whole thing with Jenkins and Docker and such. Having been to a few of these conferences over the years, alright, it’s going to take time to install, it’s going to take time to configure this. What was great was, we walked into the tutorial session, and as you walked in, the instructor handed everybody a key fob. They made keys for everyone, and said “here is all the software that you need, I’ve giving you a README file on this. We are not even going to start this class until everybody has got the install in.”  They walked around, and it only took them about 20 minutes to do that. “oh my gosh, they actually took the time to give everybody a key. They worked that in, and I guess they said they wanted to make sure that they didn’t spend want to spend the round robin time having to install everything. That was impressive. Everybody got a chance to actually hands-on run it and work with it. If there is one takeaway, take the time to do something like that.

JESSICA INGRASSELLINO: Yeah, that’s awesome.

MICHAEL LARSEN: Or at the very least, have a handful of fobs so that, if you can’t do one for everybody, at least you can do two or three at a time.

MATTHEW HEUSSER: So that brings us to Agile Testing Days, which is all over Europe. The big one is in Germany. There is little one in Sweden. There’s a little one in the Netherlands, and I think that Agile Testing Days is kind of that intersection of Agile and empirical investigation into software at all stages. They run the Software Testing World Cup every other year out of there… which, by the way, if you can’t get to a conference, the Software Testing World Cup is like $50, and it is actually testing software for three hours and then being evaluated on how you interact with the product owner and the value of the information that you provide. Have either of you guys been in the World Cup?

MICHAEL LARSEN: I’ve been in the World Cup, and I helped moderate the World Cup at Agile Testing Days in 2015.

JESSICA INGRASSELLINO: I have not but it sounds super fun. I didn’t even know it was a thing until right now!

MATTHEW HEUSSER: SoftwareTestingWorldCup dot com!

JESSICA INGRASSELLINO: JessicaNeedsToGo dot com!

MATTHEW HEUSSER: If you have no conference budget, it’s like $50 and you do it from home, and then the winners of each continent historically have had their travel expenses paid, they get a free conference ticket, they get to go to Germany for the Worlds.

JESSICA INGRASSELLINO:  That’s very cool because it allows participation on a broader scale for people who might otherwise be unable to attend.

MATTHEW HEUSSER: If you’re in Colorado, there’s SQuAD Denver, which I’m going to be at this year. There’s GLSEC in Michigan. Agile and Beyond in Dearborn. CodeMash in Sandusky, Ohio. QA or the Highway in Columbus, Ohio. TestBash is in New York. Test Bash is in Philadelphia now. CAST is all over the place. CAST is in Nashville this year. Is everybody here going to CAST? We’re going to CAST, right?

MICHAEL LARSEN: I will definitely be at CAST. In fact, I signed on with Claire Moss to be part of the social media team, which means that Claire is going to be handling much of the Twitter feed, and I’m going to be handling the live blogging, and any other type of social media stuff that we can possibly do for the course of the conference. That brings me to a point, if you don’t have a budget for getting to conference, or participating in conference, and that’s volunteering. If you have the opportunity to volunteer for conference, they oftentimes have many different needs. It could be helping set up a room. It could be moderating tracks. It could be introducing speakers. It could be just running around with a microphone.  Whatever it may be, if you step up and say “hey, I’d like to do this”, they will often give you that opportunity. The more frequently you do it, and the more the conference organizers get to know that you’re willing to do that, you might very well be on the shortlist the next time a conference comes around. There’s another conference that might suggest that does quite a lot of volunteers, and that’s PNSQC. I’ve actually had a chance to participate with this in the last few years, even though I haven’t had the chance to actually attend it. The last time I was at PNSQC was 2012,  but I have been in a peripheral position as a PNSQC volunteer just about every year since then. One of the things that I really like about PNSQC; the Pacific Northwest Software Quality Conference, it takes place in Portland, Oregon, second/third week of October every year, and one of the things that I really like about their model is that, if you’re going to present at PNSQC, you need to write a paper. Generally speaking. Now they do have some exceptions, and they have different kinds of papers that you can do. You can do a full presentation paper, Academic style paper with references and peer review, the whole bit. They also have what are called poster papers, were you can come in with just a poster, with all of your ideas set up, and unlike a standard track talk, where you are up for 45 minutes and you get a series of questions to round out the time you are up there, during the time they do poster papers, you get your poster, you get an easel and you stand next to your paper, and anyone who walks up to you, you give them your presentation.  They can ask you their questions, as long as they want, and you can talk to them as long as you want, and then you go to the next person and do the same thing. Both of them have great value, in my opinion, because you really get to dive into your topic deeply.

MATTHEW HEUSSER: I went to PNSQC with Michael in 2012. It was a lot of fun. I think it’s a great model for the regional conference. PNSQC is nice because it is tester-y. The thing that I noticed was the variety of skill levels there. There were presentations on creating test cases in Word and putting them in shared network folders and then how do you do regression testing. There were presentations on… my favorite in 2012 was one on “how do you be sensitive to international cultures without drawing stereotypes”. Little ways to be culturally sensitive, and they are stereotypes, and there’s some truth in it, and the big thing is to not be offended when something happens or try to meet expectations, instead of judging people. Not your typical software quality talk, but it was relevant in 2012 as outsourcing/offshoring and global integration was picking up steam. Other conferences?

MICHAEL LARSEN: I just realized that I was talking about my volunteering at CAST, but totally sidetracked at the fact that we didn’t actually talk about CAST, so let’s do something about that. CAST does have some elements about it that are unique.  Probably the most unique aspect of CAST… I guess other conferences have picked this up since, but I first experienced it in 2011 (that was my first CAST conference) and that’s the whole idea of the K-cards. For those who have never been to CAST before, one of the things you receive when you go to CAST, is you get free cards. You get a green card, a yellow card, and a red card. Each set of cards has a number on it, and that number is unique to you. The cards are used during what is called “open season”, so you have a certain amount of time that is designated for your talk, and at CAST, if you have an hour-long talk, you actually have a little bit less time do your presentation, because they want to give more time at the end which is what they refer to as “open season”. Open season is where you field questions, but you don’t just field questions where somebody asks you something and you answer and you go on. There approach to open season, anybody who has a question holds up their green card. A facilitator goes by, and takes down their numbers in order. “13” gets to ask their question [Editor’s note: I realized after the fact I didn’t explain that “13” was an example of a person who first held up their green card. –Michael]. And then, as “13” asks the question, the person that’s giving the presentation can then answer, and in addition to that, other people in the room can then hold up their yellow card, because they have something that they want to say about that question that was asked. That creates a thread, and then somebody else may hold up a yellow card as well. The idea is that you exhaust the threads for that particular question, and then once everybody is satisfied that they have covered that question, they move on to the next question. The facilitator is also there to help make sure it doesn’t get too far down the hole, so that other people get a chance to get their questions answered. A lot of the time, I find that during the open season for a lot of the talks that they have, that’s really where I get some great insights. We’ve broken away from “well, here’s what I’ve prepared. Here’s the best case scenarios, and here’s how we did this” and now you get to hear “well, what about in this situation?” The person might be perfectly able to talk to that, or they may not be able to talk to it, but somebody else in the room can.

JESSICA INGRASSELLINO: When I spoke at CAST in 2015, it was probably one of my favorite thing about speaking at CAST. Open season starts and people start asking these questions that start engaging me and turning me on and clicking me out of that just being focused on “give the talk, just get through the talk”. As I was participating, I was getting more animated and the audience was getting more animated, and I actually found the open season part to be more rewarding in terms of participating with everybody and creating a dialogue and an understanding of what it was that I wanted to say, and in clarifying some things that I meant, and also investigating some paths I hadn’t necessarily considered. As a speaker, it was a challenge to be aware of, but it was also… I thought it was fun. I actually found it a lot more fun than giving the talk itself [laughter].

MATTHEW HEUSSER: This year, CAST is going to be in Nashville in August, and there’s an open space conference afterwards called TestRetreat. I need to get the sign-up for that going, because we’ve just closed a deal with AST to have TestRetreat at CAST. Michael’s been to that… every year? Almost every year?

MICHAEL LARSEN: Almost every year. I wasn’t able to be at CAST last year.

MATTHEW HEUSSER: So TestRetreat is different in that it is OpenSpace. There is no preformed agenda, you pay a very modest fee for the day because there’s no pre-invited speakers that get in free and there’s not much commercial interest. You build a half day schedule, propose the topics, propose the topics that you are interested in. If there’s to many we can dot-vote to reduce the number. We haven’t had to do that lately. By running a session you are offering to host. Less likely that it’s a presentation, more likely that it’s a conversation, where you are actually trying to solve a problem. We go to lunch. Hosted somewhere where you ca go walk to lunch. Then we come back and then we do it all over again in the afternoon, rebuild the afternoon based on what we learned. Then there’d a closing session, where we talk about what we learned and get an opportunity to help each other, and that’s it. What do you think about TestRetreat, Michael?

MICHAEL LARSEN: I have enjoyed TestRetreat, and what I like about the idea behind it is the fact that it is a fully open format. The idea is, everybody should put something up; put these ideas up on the board. People start to group their ideas, so what you start to see is certain things look like they go together. Ten people want to talk about one particular topic but only two people want to talk about something else. What other conference are you going to be able to go to where you’re going to be able t have your own track session to discuss a topic that is burning inside of your head that maybe only two people might be interested in, but those two people, together, may have the key to solving some big problem that you’ve got. You get those serendipitous “a-ha!” moments.

MATTHEW HEUSSER: The other thing that I’d say is that, if you are in a town with more than 50,000 people, you can run a software development open space conference. They’re super easy to put on. They tend to create this kind of emergent coolness. If you have more than a quarter million people in your metro area, you can probably do one for testing. There’s a lot of sharing of information and ideas. The danger that people talk about with OpenSpace is that sort of lossy shallow agreement. We all nod our heads and smile and we come up with “fifteen things you can do at the workplace to solve, I don’t know, the problem with disengaged workers”, but do they actually work? They all sound great, but then, when you try them, what happens? There’s an Agile Open Northwest. I believe there’s an Agile Test Northwest. So in the final analysis, I guess the conference you go to depends on what you want to get out of it and your skill level. I think that sending new folks or folks you’re trying to get on the same page to STAREast. I think that sending more senior folks to CAST is a good choice. Just the variety at Agile TD is impressive.

MICHAEL LARSEN: I think, also, something that we need to be aware of is that we’ve been talking so far about going to testing conferences, and I want to also challenge and encourage people that, if they are interested in discussing these topics, that they don’t just limit themselves to testing conferences. I’ve presented at ALM Forum up in Seattle. I was invited to be part of Øredev, which is a development conference in Sweden. You may think “Oh, I’m a tester. Nobody’s going to want to listen to what the tester has to say.” You might be very surprised at just how engaged people are and how much they do want to hear what your insights are, because you’re coming at their problems from a different viewpoint and a different angle.

MATTHEW HEUSSER: At testing conferences, there’s this sort of “King of the Mountain”, “I’m the best tester nerd” kind of conversation/behavior that happens. “Feels good to be one of the big tester nerds”. Wouldn’t it be better if we were figuring out how to serve the other roles in software delivery? So I totally hear ya’, man.

JESSICA INGRASSELLINO: I definitely would third that. Just to round out the agreement here [laughter]. Having participated in the PyCon Education Summit, being the co-Chair and now being the Chair this year and then sticking around for the conference, there’s so much to learn from listening to how developers and people who are non-testers are sometimes talking about things like testing and also how they are talking about software delivery because it really informs the way that I go back to work. It’s inspiring.

MATTHEW HEUSSER: I think we’ve covered enough ground here to say that, if you live in a major metro area, you can probably find a test conference that you don’t need plane tickets for. My life is different since I started going to them. If your company doesn’t send people to conferences, start looking into it. Send yourself. If you’re a manager,  you want to start thinking about the skills your people have, where they develop and where to best put them, and where to invest scarce training dollars. Think about it. Let us know. Write to TheTestingShow at qualitestgroup dot com. Let us know your struggles, your challenges and questions. Who knows, maybe we can put you on the show, or read your question. Thanks Michael, thanks Jess, it’s been great. You of course can read the transcript at We’ll be back in two more weeks. Thanks, everybody.

MICHAEL LARSEN: All right, thank you.