Software Testing Engagement Models

The Testing Show: The Renaissance Tester

For centuries, the Liberal Arts education was the gold standard that all education endeavors were based. The idea of a ‘Renaissance Person” was someone who had skills and abilities in a variety of fields, grew out of the classic Liberal Arts education. Some say that it’s a bygone piece of history, but many feel that it is a vital part of working and interacting with people, and in many ways, it’s perhaps the most vital of underpinnings for success as a software tester. We welcome back Jess Ingrassellino to talk with us about the value of a classic Liberal Arts education ,how it can be applied effectively to a software testing career, and how those who are so focused on automating as the be all and end all might be missing a few things.

Also, we take a look at the growth of online testing conferences, the UK National Health Service Email bomb and Virtual Studio for Mac… wait, what?!!






Any questions or suggested topics?
Let us know!


MATTHEW HEUSSER:  Hello and welcome to The Testing Show. Thank you for joining us today. We have a wonderful group of guests, let’s first start off with our regulars. Perze Ababa.

PERZE ABABA: Good morning, everyone!



MICHAEL LARSEN: This is Michael Larsen, I am your show producer, and we’d like to welcome back Jessica Ingrassellino.


MICHAEL LARSEN: And, as usual, let’s get down to it. Let’s start up with our news and punditry segment. Matt, let’s take it away.

MATTHEW HEUSSER: Let’s start with the change in the testing conference scene that’s been happening over the past several years. We’ve seen a lot more small conferences, we’ve seen mobile only conferences, big data only conferences, and now these specialized test conferences. So there’s TestBash, which has grown all over the place. Agile Testing Days is exploding all over the place. Anna Royzman, a friend of the show, has her Test Masters Academy. Most recently, there are now online test conferences. Joe Colantonio has got his Test Automation conference, it’s 100-200 bucks. January 9 through 13. What I’m curious is, how are these things going to work going forward? Are they going to save all of the recordings and then you’re going to be able to watch videos? Are you going to be able to sign up after the event, and if so, what’s the benefit of going to the event? Is this the new, great way to get online training delivered? Or is it going to be a bunch of high level, abstract schmaltzy stuff that you can’t really apply? Truth is probably somewhere in the middle.

MICHAEL LARSEN: So Charles Max Wood, who runs the Ruby Rogues podcast and a few others on Charles also does the “fill in the blank” remote conferences. They do these remote conferences, so that you don’t necessarily have to say, “we’re tying up a venue and we’re going to bring a whole bunch of people together” -and there’s a lot of overhead to that- whereas the idea is that you pay $100, $200, you get into an online environment, the conversations are recorded, and then, either your full entry fee pays for full access, or if you are there for free, then you get a special track that you can follow along with it. STAR has done this, a few other places have, STP’s done this, too with their Online Summit’s that you can join as well. Oftentimes, when you go to these conferences, somebody is a tool sponsor, they’re bringing in their tool and they’re talking about their tool and what it does, and they’ll give me five minutes of up-front, interesting, semi-abstract concept that I can work with, and the rest of the time, I’m looking at a tool that I either have to buy, or I have to give them my information for taking on a demo, and then I’m spammed for the next eternity. I’ve generally been skeptical of online conferences. Not because I don’t think they can deliver value, but because I feel like they’re fronts for selling tools, or have been fronts for selling tools. If we can get out of that aspect of it, and yeah, it’s going to cost you a hundred buck, take a raw figure, a hundred dollars, but for a hundred dollars you get access to all the content, you are walled off from the vendor stuff (unless you really want to look at that) and then you know that every talk that you will be covering is going to be a real topic, it’s going to be covering real material, and will be somewhat valuable.

PERZE ABABA: Thanks for making me remember the STP Online Summit, Michael. I think this was in either 2010 or 2011, I remember attending two of those. One specifically with Jon Bach as well as Alan Page. Those have been pretty good for me. I don’t remember any follow-ups after that from a marketing standpoint, but overall, that has been very helpful. I do see one fundamental difference between the test conferences that you just mentioned, Matt. So you have online test conferences as well as Joe Colantonio’s Automation Guild. I think the primary difference between both is how they set the program up. With Online Test Conference, there’s a set time you’re supposed to show up, so if you think 4:00 Eastern (I think it’s 4:00 p.m. to 9:00 p.m.) but then with Joe’s Automation Guild, there are videos that they will show you, and then the speaker are available for any questions at all through the chat room of the conference as well as through social media. One is like a real conference that you go to, where you’re bound to a set time, while Joe’s, on the other hand, has its benefits as well because you can go at your own pace, but of course, if you want to engage with the speakers, then you’re going to have to do that within that time period.  Joe’s will give you full lifetime access to the video, I think, so these are things that you can go back into but, definitely, these are very interesting, and it’s a welcome addition to the conferences that we go to, but of course, I’m a traditional guy when it comes to conferences like these. I would love to meet people that I follow on Twitter or people that I talk to on Twitter and shake their hand. I think that’s the one thing that you will not have when you’re dealing with these online test conferences.

MATTHEW HEUSSER: There are new innovations in online delivery formats. We’re going to have links in the show notes. Check them out if you want to. Including podcasts. Pere, you wanted to talk about this new email bomb.

PERZE ABABA: This was a couple days ago, when the NHS in the UK apparently sent a test email that was accidentally sent to 1.2 million employees. So that’s really a reply all problem that’s sort of bound to happen. Towards the middle of the day, there were approximately 270 million unnecessary emails that were sent throughout that system, so there’s definitely some concerns within the people in the system. It kind of reminds me of the companies that I worked with for some reason in the past seventeen years or so, where every single company that I’ve worked with, always has had this happen. I’m not surprised that this actually happened, but the volume of that, which is up to 270 million that were sent through the 1.2 million members, it’s amazing [laughter].

MATTHEW HEUSSER: So those systems should have some sort of Denial of Service sensor as a feature, just like web servers do, but it’s like inadvertent Denial of Service or accidental Denial of Service.

PERZE ABABA: Right, but most DDoS sensors or triggers or fail-safes look at external attacks. It’s not something that they look at, something that they do internally, and that might have been one of the challenges. Can’t stop this mid-flight.

MATTHEW HEUSSER: Yeah, right. It’s a lot harder than the requirement sounds on paper. “Oh, just make a sensor”. No it’s actually going to be… that’s a system. News piece. Microsoft released Visual Studio for the Mac Preview, for free, yesterday. Blew my mind. What a huge change. Now I was expecting a tool that would allow me to create mac or Windows apps, and it would cross compile in .NET. But it’s not. It’s for iOS and Android. You can’t make Windows apps in Visual Studio for Mac. You can’t make Mac apps in Visual Studio for Mac. It’s all 100% mobile. Isn’t that amazing? That tells me that we’ve had a paradigm shift in the kind of apps that we’re making. Microsoft isn’t even bothering to try to get the Apple desktop.

MICHAEL LARSEN: That is interesting! When I saw that it was announced, I figure it would be like “OK, well this is a way to encourage software developers used to working with Macs, OK, well here, you’ve got a tool so that you can actually write your .NET stuff on a Mac and not have to open up a Virtual PC”, but it sounds like a total different scenario. I haven’t taken a close look at it, but now I’m curious.

MATTHEW HEUSSER: Yep, go ahead, the preview’s free. So, when you talk about responding to change, in my generation we moved from client server to web, we moved from web to responsive design to mobile. I’m curious the percentage of native mobile apps, how much that continues to increase over time. Is that what’s next? So, main segment of the show. I wanted to talk about the Renaissance Tester. When I was working at Socialtext, there was a couple of sort of themes that ran through the technical staff, and one of them was music. A lot of us were musicians to one extent or another. I’m not embarrassed to sing in public, and apparently in North America that makes me a musician [laughter]. Chris McMahon played all kinds of instruments; guitar and banjo-ey kind of stuff. Ukulele, which I guess is easy to travel with. Lots of different music… and writing! There’s a lot of writers there. So it was really… it was god for me.  We had this kind of Liberal Arts vibe. Now, Liberal Arts means something specific. It’s a list of academic subjects that are not specialized and tied to a job, but instead are good things to learn to make you a better person. Literature. Philosophy. Mathematics. Social and Physical Sciences, distinct from professional and technical. Accounting would be a technical subject. Computer Science, probably a technical subject. Steve Jobs was known for saying “Apple was at the intersection of technology and the liberal arts”. The passion for design, and integration of hardware and software, and when I think Liberal Arts, I think of the Renaissance. I think of Florence, Italy. I think of DaVinci, who did a whole bunch of things. He didn’t just draw pictures and didn’t just do sculpture. He was also a physicist. He invented airplanes that would have actually flown, if he could build them, or gliders at the very least. What I’m getting at is, in testing, we tend to lose this sense of the person that can actually think creatively, and we say that a tester is either a coder or a box checker. I have a line of discussion I want to pursue there, but first I want to get your reactions to that; have we lost the concept of the Liberal Arts Tester? Does it matter? Should we take it back?

MICHAEL LARSEN: OK, so this is a personal hot topic for me. Having Liberal Arts or having a good grounding in Liberal Arts education or being a polymath, whatever term you want to use… I think it is critical. I think it’s important, and it’s one of the times I will hear people say something to the effect of “Well, you know, you don’t have to go to college to be a software tester or to be a programmer. On the surface, I will say that’s true. You don’t have to do that, but I will also say there are benefits that you get from going to a university or attending a college, especially if you are studying in the Humanities or the Liberal Arts, and they’re not specific to job skills. I think in some ways we are missing the point when we talk about people needing to have a four year college degree or going to a university so that they can be trained in a job field. That’s really not the core point of getting a classic Liberal Arts education. A classic Liberal Arts education is exposure to enough areas in the world, with enough cross-sections of the world, that you actually develop methods for thinking, both inductively and deductively. That you develop critical thought. That you look to se what’s happened in the past. When you say that you stand on the shoulders of giants, as a phrase, that you actually understand who those giants are, and what the realities of their world were that helped shape their world view. Whenever I hear somebody talk about “well, what’s the real value of a university education”, a lot of people will say the real value is the social network you develop… and there’s a value to that, too. When you see people go away to a university, what is the purpose to that? Is it a glorified dorm room experience so that you can be away from Mom and Dad for the first time? Yeah, maybe that is a big part of it; you actually have to develop social capital. You have to interact with people you otherwise wouldn’t come in contact with. Whereas, in this meritocracy world of “well, you can work in your bedroom and you can get a GitHub account, or you can work on events and activities that show your technical prowess, but they don’t necessarily help you with the way that you think, they don’t necessarily help you with the way you interact with others, you don’t necessarily develop social capital, and the skill I believe is the most important when it comes to a Liberal Arts education, and again, let’s be clear, you don’t have to go to a university to do this, but it does help because it gives you a lot of the structure for that, is that you develop the ability to pivot. Whereas, if something changes, or you see a different reality than the one you planned for, you have some tools you can draw upon so you can go a different direction. If you don’t have that background, if you haven’t had that exposure, and you’re stuck in a situation that you don’t have familiarity with, you don’t have the same set of tools to cope.

JESSICA INGRASSELLINO: You guys know I have to jump in here [laughter]. I fully and 100% agree with you, Michael and matt, I‘ve definitely worked in a lot of places in my testing life with lots of musicians and artists. I think I’ve worked with two other people that have Doctorates in Music, which is just a strange happenstance. I see it in action, but I think that’s what’s lead me in the fact that I entered testing as a music teacher, and I had no training, I didn’t know anybody’s names. It was really a super accident that I fell in. the things that got me through were the problem solving skills, and those are the things that whenever I give talks about… Art and Science of Testing was my first CAST talk, and that was all about qualitative research, skills, social sciences. I’ve given talks about play and how play factors into the way that we solve problems and learn about our environments, and to me, I really resist labels on myself as a tester, because of my relationship with things like philosophy, music, social sciences. Those things really inform my approach. I just bubble over with agreement.

MATTHEW HEUSSER: The thing is, you can’t really put Liberal Arts skills on a check[list].  I don’t know how many times in my career I’ve seen “able to think creative, critical”, seen that on a check list or a job description, and even if it is, how can anyone looking at a resume evaluate that, other than person say “Bullet point: able to think. Bullet point: creative.” So we lose that in the interview process.

JESSICA INGRASSELLINO: I think it depends on how the interview process is conducted. I think that, more often than not, it’s lost in the interview process, but I don’t think that it has to be.

MATTHEW HEUSSER: Yeah, right. There are companies that do it better and there are companies that do it worse, and there are way that you could pick up on that, but generally speaking, when we’re looking at resumes it’s challenging. If you know someone by reputation, maybe you can sidestep that.

PERZE ABABA: I think that, from the last point you were talking about, one key point that I really realize from that is it’s easy to find out what you know. It’s even more difficult to find out how you came about knowing what you know. I think a lot of the focus on the Social Science and the Humanities focus on the latter, where the focus tends to be, and I think that’s why it’s very key to look at this, because now that we’re looking into big data, for anyone that’s playing buzzword bingo out there, we’re really looking into insights that we can derive out of how people use things, but why it works is that it’s focused on the people, and it’s not just focused on the technology and how you built the technology around it. That’s my primary takeaway from it.

MATTHEW HEUSSER: Right, right. Well, where I’m going with this is that we tend to do the checklist; “yes, I know C# and Selenium and Cucumber and whatever the BD framework is for .NET… SpecFlow. I have all of the things, so you should hire me.”  There’s this curious thing that happens. I mean this to be challenging in a friendly way, but there are people who say “yes, I know all of the automation things” and they will jump in and they will do the Cucumber or whatever the strategic vision the company has, and they will interview well and they will say “I know all of the coding things”. You know I talk to those same companies with our team and… are you sure that’s really what you want? We don’t pursue the opportunity. They hire somebody else. A year later you talk to the somebody else they did hire, and they say “yeah, I’m spending all my time doing exploratory testing”… and I’m like “What?!!” Drop the mic. To some extent, I think things have happened like that with Michael and Jess, and I wanted you to tell me your experiences about that, and maybe we could come to a synthesis. Is my assessment anywhere near right? What am I missing?

MICHAEL LARSEN: I would say you are absolutely correct. That has been the case more times than not. There is always the abstract view. Now, if somebody is coming in with a very clear image of what it is they are going to do –I’m going to be an automator, I’ll just pick that up- then what it really comes down to is you need to be somebody that is selling themselves on their programming and automation and framework skills, and really making an effort and emphasis on that being the primary focus. Which means you would actually push back on doing other type of test work. My guess is, especially for me, I’ll put it out here… I’ve made it very clear, I can program when I need to, I can get into the guts and work on something when I need to and I consider it a valuable skill to work on and modify scripts and make sure that they’re taking out the busywork of the things that I don’t want to do repetitively. Whereas, when you get a chance, at least with a real thorny testing problem or you’re looking at a new feature and get into the guts of it, for me, that is what I prefer doing. So I don’t think that it’s unusual that, if somebody genuinely enjoys doing a particular task, that they are going to find ways to emphasize what they enjoy doing, and so I think, for myself, and to clarify this, I think the reason why I’m brought in to do automation and do various system adminy type stuff that, I’m fairly good at, I’m willing to do it, but then it’s like “Why are you spending 75% of your time doing exploratory testing?” Well, because that’s what I like doing.

MATTHEW HEUSSER: Yeah, I think I’m on the other side of the fence of… there’s a lot of people that say “oh, you don’t have the skills to write code, that’s so sad. You’re trapped in a ghetto. You’re never going to get high pay, high value jobs”. No. I have two degrees in CS. I was a programmer for ten years. I wrote hundreds of thousands if not millions of lines of code. I could do it. It’s just, I look at the number of bugs that tooling finds is, like, not much, and the amount of coverage I can get as a human is, like, a lot higher. This is not interesting to me and I’m not finding many bugs that way so, what are we doing?

JESSICA INGRASSELLINO: Right now, where I currently am, I’m an exploratory tester, and I have the option to help out with the code as much or as little as I like for the automated testing, but I was not brought in to do that. I was actually brought in to do what I feel that I excel at, which is taking a look at the whole system, learning it as quickly as possible, and as thoroughly as I possibly can, and then making some architectural decisions about how “quality” is approached throughout the system. The two jobs prior to this one I was brought in solely to automate. Like 100%, we don’t want you to do anything. I caught some hell for actually doing exploratory testing before I automated, so I would understand what it was that I was automating [laughter], which just seems funny to me, but I really caught hell for it, because there were very different ideas even during the interview process. I always express I’m not fast at writing code because I’m self-taught, and at this point I’ve been doing it for three years, so it’s not speedy. I can do it, but I wouldn’t consider myself great at it. Because it’s slow. My preference is to be a holistic thinker, to be an analyst, to be an architect, to look at a whole system, to figure out all of the places that value could be added in different ways, using all different kinds of tools. So I actually wound up leaving those jobs, because I was boxed in.

MATTHEW HEUSSER: So when someone says “Oh, you’re exploring, don’t… just automate it”, have you ever been tempted to say “Oh, so you don’t understand my job? You don’t understand what I’m doing?

JESSICA INGRASSELLINO: Not only have I been tempted, there have been times where that may and/or may not have come out of my mouth in one way or another. And that’s actually over several weekly or every other week kind of meetings, and it came to a point in both circumstances, I’d say “it seems like we have different ideas about what this is supposed to be, based on what we talked about in the interview and what I’m building and what needs to happen for what I am building to actually serve a purpose. I think those are hard conversations to have, because there is definitely a fear of losing your job.

MATTHEW HEUSSER: Yeah, there’s a weaker and a stronger way to say that. I think that “we have different expectations for the role”… there’s less friction there, because then it’s just two different opinions. “You don’t understand what I do” is kind of a consultant “this is the offering, and you either want it or you don’t and that’s OK” The strong confrontational is “Oh, so you don’t understand testing”, and there are only a few people in our field who will say things like that out loud and mostly, that’s not appreciated.

JESSICA INGRASSELLINO: Yeah, there are definitely people who would say “you don’t understand”. It’s different when you are a tester that is embedded within a culture of an engineering team versus an outside consultant. The consultant, theoretically, can say “ok, well, this is’t working because you said you wanted one thing, you actually want another. You clearly don’t understand test, gotta’ go!” When you’re embedded in an engineering team, and this is again the Liberal Arts thing, going back into that Liberal Arts piece, and going into things like critical theory, critical literature, philosophy, and Socratic method, is it our job as testers to tach people what our job is? No, but yes. [laughter] I think that there is some… not responsibility, but certainly, it’s helpful if one can listen to an opposing viewpoint, or maybe a lesser educated about testing viewpoint, and then use those kinds of critical questioning and critical thinking skills, to help to educate people on the team about what it is that you’re trying to do. That can help to build a strong team and a case, which I’ve been able to experience, actually. It’s nice when you have other people start to come around to what you’re trying to do, why you’re trying to do it, and then that gives ore support for the position of well-rounded testing, which involves, kind o, all the things.

PERZE ABABA: If you focus more on automation, automation being “the judicious use of technology to be able to solve a given problem”, it’s even more important to hold onto non-technical skills, because when you are looking at testability, for example, you have to take a look at “how can this affect me as a tester?” Can we open this information just for machines, primarily for tooling, or can our testers at the Scrum level, will they be able to use this as well? The difference between that way of thinking, instead of just thinking of automation as something that we use to convert all of our existing text cases into, is definitely faulty at best. In my entire career dealing with automation, the way we look into solving these problems, how I used to test before as compared to how we tend to solve automation problems at work, there’s hardly any difference, because we need to be able to explore what avenues are available for us and then solve that problem there and then. You know, the law of noncontradiction doesn’t necessarily apply where you either automate this or not automate this. For some reason, it’s never an OR construct for me, but it’s always been an AND construct. I think those two definitely go hand in hand. Especially with how fast the technology turnover is.

MICHAEL LARSEN: Another thing that also fits into this. In many cases, when we talk about testing, there are a number of components that you can use, and many of them require a particular way of looking at the world. When you do an automated test, you set up your environment to meet a particular condition. You then use assertions or other methods to determine if your hypothesis is correct, and then you tear it down, and then you go on to the next test and you do another sequence of that. But there’s a lot of interaction that you lose; you’re getting a very limited view as to what your app is actually doing. Whereas, if you’re a real human being playing around with the app, you get interesting state conditions that you don’t fall into when you’re just setting a test up, testing it for a very specific slice of functionality, and then tearing it down and going on to the next one. Those are very important things; if you’re going to have a repeatable, reliable automation suite, you need to be doing that, but there’s so much you miss, because that’s not the way a real human being uses a product. I think it’s really important for us to realize that the critical thinking, the Socratic method, the view of design, the view of aesthetics, of understanding perspective, of having varying viewpoints, of being able to personalize and humanize and empathize… all of that comes together because, at the end of the day, software isn’t just an algorithm, it’s an experience. We need to be able to have the humanity; there’s a reason why we make software, it’ not to feed a machine to feed another machine, it’s so that people can accomplish something, generally speaking. If we miss the whole point of what it is that people are trying to accomplish, then our efforts tend to be a little bit myopic.

MATTHEW HEUSSER: Thank you, Michael. I think we’re about out of time for the show today, so does anyone have anything that they want to announce or talk about before we get going?

MICHAEL LARSEN: The Association for Software Testing is expanding its borders. It’s doing its first conference, I want to say its first conference internationally. That’s not true. We’ve done two of them in Canada, but we are now doing what is called a CASTx, and it’s going to be happening in Australia. So that’s a big jump, and I’m really excited to see that an organization that I’m actively involved in that, for many years people have said is only focused on North America… not true! We’re coming to Australia, and if you happen to be in that vicinity or you want to be in that vicinity, I will put a link in the show notes as to how to do it. Let’s get people on the other side of the world that want to see what a CAST experience is like to experience it down under.

JESSICA INGRASSELLINO: You guys know that I am really into the Ed scene and the Python Education Summit Call for Papers  is out now. Registration is out now, papers are due on January 3rd, and please, please submit. We’re really looking for a wide range of topics.

MATTHEW HEUSSER: I, in May, I’ll be at Agile and Beyond in Dearborn and just got asked to come out to Denver for Software QA Users Group Denver, and my year is starting to fill up, and I may be over by Detroit more, we’ll see in 2017. That’s it for the show today, folks, I hope you enjoy it. If you have comments questions, concerns, please shoot us an email at the testing show at qualitestgroup do com. Shoot us an email, we’d love to hear from you. What do you want to hear about? Do you want to talk about management? Do you want to talk about hiring? Let us know, we’d love to hear from you. Thanks for coming everybody, really appreciate your time.


PERZE ABABA: Thank you.


MATTHEW HEUSSER: Thanks for listening, and we’ll be back in two weeks.