The Testing Show: Digital Transformation

The Testing Show: Digital Transformation

Over the past several decades, we have witnessed many changes in the way that we do things. Catalogs are mostly things of the past, as are phone books, a crew of phone operators and a variety of other services that we may often take for granted today. Each of those transformations, however have gone through their share of growing pains to bring us where we are today and truth be told, there’s still plenty of room for improvement. For this episode, Matt Heusser and Michael Larsen welcome Shachar Schiff and Gerie Owen to talk about the benefits of digital transformations, where things have gone well, where things have gone not so well and a few surprises along the way. In addition, they discuss the continuous nature of digital transformation and the software testers unique role in that process:

itunesrss

Panelists:


References:


Any questions or suggested topics?
Let us know!


Transcript:

Michael Larsen: Hello everybody and welcome to The Testing Show. It is February 2019, happy new year. Due to changes and things after the holiday, we decided to get things started in February and here we are. We’d like to welcome Gerie Owen and special guest Shach Schiff and of course Matt Heusser. You’ll know me, I’m Michael Larsen and let’s just get going.

Matthew Heusser: Thanks Michael. This month we wanted to talk about digital transformation and this is something that comes up all the time. A couple years back at the Agile conference, companies were talking about their transformation services and I was like “Transforming to what? What does that mean?” And, aside from naming the name of a framework like SAFe, I did not get an answer.

Matthew Heusser: I thought we’d feel better and talk about it for testing. To start with the obvious, what is digital transformation?

Shachar Schiff: So, I don’t really know what digital transformation is, and I’m an expert. For example, it’s the use of technology to radically improve performance and reach enterprises, and that’s the hot topic of companies across the globe today.

Matthew Heusser: So, using technologies to change companies, right?

Shachar Schiff: Okay, yeah.

Matthew Heusser: Isn’t that just our job?

Shachar Schiff: Yeah. That’s what we do.

Gerie Owen: Well, I think it’s a little more than that. I think it’s rethinking not only of what we do but also how we do it. The Sloan School of Management study more or less defined it as a way of rethinking the customer experience, as well as the business models and the operations. Finding new ways to deliver value and generate revenue and improve efficiency. Companies usually use innovative technologies to do that. But, sometimes it’s more important to look at the process first, because it might not always be a technological solution.

Michael Larsen: There’s some interesting examples that we come up with on this. It sorta comes into the same old definition, and to borrow the old Supreme Court adage, “You know it when you see it”, is the way it often gets presented. Maybe it’s easier to start with an example that most people will readily understand.
Michael Larsen: Here’s a quick question for you. How many people still receive paper catalogs?
Matthew Heusser: I’ve noticed, it’s only the extremely high profitable ones like flowers, and it’s only for holidays and it seems to be aimed at a different generation.

Gerie Owen: Well, I still receive some from fashion outlets. Yeah. I think it may be aimed at a different generation, which is an interesting point. Because, that’s where I think you need to know your customer. Some organizations that are dealing with tech-savvy group of customers will not need to have paper. Whereas, ones that are dealing with, yeah maybe flowers or any kind of things that an older generation would use, would still need to have that available.

Shachar Schiff: For the catalogs, one of the things that I’ve noticed is, sometimes they use cross-marketing and they’ll include a promo code, where the user would have to go to the website to get a discount, a reduced price based on what is offered in the catalog. So, both analog and digital cross-marketing.

Matthew Heusser: Yeah. I think there’s an omni-channel strategy companies are using where they’re gonna reach every possible way. We’ve got a Facebook page, Instagram page. We’re on the web. We’ve got 800 number. We’re on TV. We’ve got catalog.

Matthew Heusser: When I think about digital transformation, I mostly think about eliminating expenses, little steps done by offers for the person. So, that’s eliminating dual entry, managing business process, right? Let’s get in from catalogs to the web. That’s eliminating the person that you call when you call 800 number and say, “I want to subscribe to Ink magazine.” Maybe there’s still a paper magazine, but now I’m just gonna go to a website and type in the credit card number.

Gerie Owen: And the thing that you’re talking about there is, that’s an example of transforming processes, which is one of the three pillars.

Matthew Heusser: Operational processes. Right.

Gerie Owen: Right.

Matthew Heusser: Business model might be using to a subscription model from a buy a product model maybe.

Gerie Owen: Yeah. Business model could. Enterprise integration. More of a new ERP system, that would also be an example of improving the operational process. Going to mobile apps and would be an example of customer experience, offering multiple channels for customers to transact with the organization. Offering self-service and those kinds of things.

Shachar Schiff: So, more like a customer experience, customized customer experience?

Gerie Owen: Yeah.

Matthew Heusser: One thing I’ve noticed the company, I’m gonna call them out ’cause it’s a good thing. It’s SmartBear. SmartBear has built a portal of mutual support for their users, and they have some pretty serious power users. And, they’ve done a really good job. It’s not financially motivated. But, there are some serious power users who spend all day answering questions. And, they can then reduce the number of customer support reps they have. They can charge less for their products, and they can provide better customer service, and maybe make more profit. And, they can do the whole thing over the web. And, I think that would be digital transformation on the customer experience level.

Matthew Heusser: Chat bots are another one. I don’t like chat bots. The only reason I’m contacting chat bots is to schedule a service with my dealership for my car. And, I get all the way down and they say, “You’ve gotta call this number.” I call the number, it’s after hours, there’s nobody there. And, that’s the reason why I started talking to a chat bot. I call that a fail. I mean …

Gerie Owen: Yeah.

Michael Larsen: Absolutely. There are certain things I can understand if a chat bot is trying to say based off on fairly simple punch points, we ca n get you to a piece of information. Okay, but if I’m about to go on a phone call with somebody, I’m usually dealing with something that goes way deeper than what … It may work for some people. I’ve had a very low success rate with chat bots to be honest.

Matthew Heusser: I think they’re very hard to do well. Which is kind of a good pivot maybe in that, how can testers add value? What’s the testing role in digital transformation?

Gerie Owen: It’s probably beyond testing. It’s being the champion for the customer, and in doing so, being the guardian of the organization’s reputation. Because, if any of these transformational ideas don’t satisfy the needs of the customer, they’re not gonna work.

Gerie Owen: I have an example. I went into a women’s clothing store to get a gift certificate for my niece for her birthday. And, there were two sales associates trying to wring up customers purchases on these little mobile devices, and neither of them were working. That company obviously weren’t satisfying their customers that were standing there trying to make their purchases, but they were also making it difficult for their associates.

Matthew Heusser: I think that’s a fantastic example though. At Apple stores they transformed the buying experience by enabling you to buy right there with a tablet with a person. You didn’t have to go to a little desk. They could just click-click-clickety-click and then print out a receipt, which just magically appeared out of the wall. Here’s your MacBook Pro. Take it home now, you’re done. But, if they screw that up, that’s really bad. And, how many ways can that go wrong? The wireless goes down. Boom, your store can’t sell anything.

Gerie Owen: I actually had that happen. I was actually in the airport in Detroit and I went into a store while I was waiting for my flight. I was gonna purchase something and their network was down. And, I didn’t have time before my flight to run to the ATM and get cash. So, they lost the sale.

Shachar Schiff: That’s a really good example, including with the network being down, of being at a store and having to fill out information. For example, the Apple store. If the form’s too long, a tester might say, “Hey, this is too long.” This ends up being a bad user experience or pretty poor user interaction. Might be better for the store to just gather that person’s email address and followup later on with another form.

Gerie Owen: I have a great example of that, and this actually is while ago. The company that I was working with was an insurance company, and they were first putting the annuity application onto a new order entry system. The agent would take their mobile device or their laptop and sit with a customer and put this in. And the product that I was testing on, it was this decedent IRA, which needed a date of death, and it had to be done within six months of the date of death.

Gerie Owen: You went through eight screens of information before you input the date of death. It checked right there. There was a check if the date of death was beyond the six months, you couldn’t do it. I wrote that up as a bug, and it came back as designed, because that was the easiest place to put it. But …

Matthew Heusser: Bad design.

Gerie Owen: Yeah. It was a real bad design that was not customer friendly. So, what I was doing was being the champion for the customer, and also, how would the organization’s reputation be if that situation happened? That would hurt the organization’s reputation. That’s kind of what I mean. It’s more beyond the actual testing. It’s more the whole quality assurance.

Shachar Schiff: I think with social media today, which is also a digital transformation, I actually tell some of the clients or projects I work on, to justify exactly what you’re talking about Gerie, about poor user experience or poor customer experience. I like to remind them, “There’s a good chance that somebody is gonna tweet this or share this on Facebook, and the last thing you want as a brand is someone sharing this information on social media.” Which is why it’s better to address this.

Matthew Heusser: I think enough years have passed that I can share this story. There was a consultant was meeting with the board of directors of a large hospital group. And, they had just switched to voice recognition systems. And, it was even voice recognition. “Press one if you want to make an appointment with the doctor. Press two if your leg is broken. Press three if you need to see ER.”

Matthew Heusser: They were getting some feedback that these systems were not working for people. And this is a large hospital group with lots of campuses. You’d have to call the switchboard and select which campus you wanted, and then figure out who the right department was, and then press the right buttons to get to schedule an appointment. It was ridiculous. It wanted you to schedule an appointment by pressing buttons. And, if you pressed the exact right buttons, maybe you could talk to a person. And they were getting a lot of complaints about this.

Matthew Heusser: So, the board was like, “We’re way ahead of this thing. We need to figure out if it’s even a problem. These are anecdotal stories.” So, the consultant picked up the phone and said, “Let’s try to make an appointment for a general practitioner visit. Do you know the name of your general practitioner?” “Yes.” “Do you know what campus he’s at or she? Your calendar is on your phone?” “Yes.” “Let’s try to make an appointment for you right now.” And 30 minutes later he hung up the phone in frustration, and they said, “Oh, we need to redo this system. We don’t need to study it. We need to fix it.”

Matthew Heusser: And that ability to cut through the skin to get to the bone and say, “There’s a problem here that needs to be fixed.” I think testers can do that in a way that a lot of other roles can’t. Project managers have this incentive to hit the deadline at all costs. They’re not going to go, “The thing that we’re fundamentally building is floored.” And, if you say, “The thing we’re fundamentally building is floored”, and people say, “Build it anyway”, well, we did our job. Maybe the organization will learn next time.

Gerie Owen: Well, that’s the thing. I mean, our real job is to advise of risk. So, that’s the risk. The risk to the company’s reputation. If they build it anyway, well …

Matthew Heusser: I guess my question is, can we pivot so that we as testers are impacting customer service operational process and business model? ‘Cause, I think there’s a lot of value to add by that injections. ‘Cause, a lot of people think of us as button pushers and bug reporters, and I think there’s more to it than that.

Shachar Schiff: Yeah, absolutely. We can pivot. And, actually, most testers might be doing that because of digital transformation. Because, due to digital transformation, we are now upfront and embedded, and no longer siloed into individual teams. Testers are no longer at the end of a waterfall project. It’s mostly Agile. So, due to probably being Agile, agility and digital transformation, I see testers as being critical to the success of projects.

Michael Larsen: This is one of the things I’ve been actually considering about this. This might be an example of digital transformation in a macro sense, that I think there’s still plenty of room for improvement. And, maybe we’ve even done this wrong, with the whole idea of digital transformation and especially with the internet fulfilling its promise.

Michael Larsen: One of the great ideas of the internet was the ability for people to work anywhere and to be able to get the best talent in the best location. And, Matt, you just wrote about this. Have you noticed that even though we have these great digital transformations and these networks and all these operations, and ability of being able to work and collaborate, yet somehow we still doggedly decide that we want to cluster people into one city, into one office building? It’s almost as though the internet never happened. Is that an experience of digital transformation gone wrong, or digital transformation not fully implemented or understood?

Matthew Heusser: I just saw a job advertisement for a Senior Social Media Manager for a media company, it was Forbes or something. “Must work out of San Francisco office.” Okay, you just limited the number of people that would be interested in that job from, I don’t know how many thousand throughout North America, qualified people, some of them available, some of them reasonably cheap, to 15 people. Why would you do that?

Michael Larsen: And the need to pay them a whole lot more money, because …

Matthew Heusser: Right.

Gerie Owen: Going back to the pillars, that worker enablement is under the operational processes, and I guess that’s an example of not-enablement. I think it’s interesting though that you’re seeing in this age of the internet and the ability to work remotely, you’re seeing so many companies now wanting to bring people back on site. It’s a whole other topic that I’ve been studying.

Gerie Owen: I read a book called Genius, and a lot of it is about real true innovation is not done in a vacuum. And, a lot of companies are finding innovation works better when people are actually face-to-face. More of the innovation takes place around the break room than at the desk.

Matthew Heusser: My main thought is that all remote can work. All onsite can work. When you’re mostly onsite because of historical reasons, and a couple of people work remote sometimes, those are the people that are not gonna be plugged into the information culture. It’s gonna be hard for them to innovate. It’s gonna hard for them to get things done.

Matthew Heusser: They’re the ones that are gonna get laid off, because it’s gonna be hard for them to develop relationships. So, you don’t have an emotional connection to that person. You’re taking a lot of risk when you say, “I’ll be that experimental person that’ll work remote.” It’s probably fine this year, but three years from now when we have a bad year, you’re not gonna be in a particularly good position.

Matthew Heusser: So, there’s real dichotomy there, and I think that when we’re talking about designing business processes, testers are in a place to say, “Oh yeah. If we do that we’re gonna have a foundations problem when we create new employees. And then if we do that then they’re gonna have to come in the office to get a laptop. Then we’re gonna require that they be in the office. We’re never gonna break out the cycle. So, this small tweak to a business process could enable us to make remote work more effective.” And, getting that voice heard I think is the challenge. Getting the presupposition of what the role is change, I think is often harder than we hope.

Matthew Heusser: Shach, let’s talk about your experience for a minute at digital marketing agencies, working with them to make sure that the website, which probably looks fine for one person, will look fine for anybody anywhere in the world looking at it in any device and any browser. So, they get that right. Is that a lot of what you do?

Shachar Schiff: Yeah. It’s a lot of what I would call, educating and reminding people what best practices are. I think I’m a little bit passive and I’m not so assertive with them. I just say, “Here’s what we can make better with this browser, and here’s why it could be better. By the way, here’s your data on analytics”, which fortunately due to digital transformation we have that data, and we can say these are the browsers that help to prioritize, and these are the browsers that help to not prioritize, based on that data. Cross-browser testing, cross-device testing.

Shachar Schiff: Some industries, it’s more important to work on all browsers. For example, any website that’s regulated by the FDA. However, another thing that I suggested is, if it’s not regulated by any government body, I’ll come around and say, “Hey, instead of fixing this browser-“, based on that analytics data, instead of spending money to fix a browser that is declining year over year in usage, and here’s the data to back that up both with the site analytics and via W3C or any other online, publicly available data. Then, I’ll say, might just be better to have the user agents string just produce a message to say, “Hey, we see you’re on an outdated browser. For better user experience, please view this on Chrome or Firefox or a modern browser.” So, we do use data and analytics data extensively to make decisions on what to fix and what not to fix.

Matthew Heusser: Yeah. I was on a retail app a while back that was high-end luxury retail. We looked at dollars floating through the system. Not users, not viewers, not unique views, but dollars floating through the system. And, it was all Apple devices on Chrome, particularly mobile. So, it was an iPhone on Chrome. People would buy coats and jackets and hats and things on their phone, which I would’ve never known without that data. And, I think it’s a very powerful, compelling argument.

Matthew Heusser: We’re adding a lot more value there when you talk about statistics and options than, “On IE11, in the checkout tab, if you apply a discount, the discount isn’t applied when you check the checkout button.” Different level of value.

Shachar Schiff: Yeah. And, my perspective with my clients is to say, “Is this cost effective? I don’t wanna waste your time. I don’t want your resources spending time on browsers that aren’t really relevant, even though your client might be on it.” And one of the first questions I do ask with new clients and brands is, if they’re in the corporate IT policy, and if they are then what is the browser within their network.

Shachar Schiff: There’s a really good story I have about a big brand and brand new website that we worked on. The CEO of this brand escalated to one of the people in charge of this ad agency, because everyone in their network was viewing this brand new website and it rendered like it was an IE 8. It got escalated, and they emailed me and they’re like, “Hey, didn’t you work on this? Did you test it in these browsers? It’s totally broken.” And, we figured out after back and forth for about a day or two, which by the way the brand actually decided at that point that the site was not going live. And, we actually told them that everybody outside of their network, including where we’re at, were not able to duplicate what they were seeing.

Shachar Schiff: At the end of the day, what we figured out was that they had IE 11, and it looked great on IE 11 outside of their network. However, inside their network, they had a setting for IE 11 to view it was IE 8. So, that’s basically how the site up … there’s a mode in IE 11 that’s basically, they had it set in their corporate IT policy to render web browsers, like older version of IE, even though they’re using IE 11. There’s things like that where we have to troubleshoot and remind clients.

Matthew Heusser: Wow. And that’s probably because they were using some software that never got upgraded that required IE 8 or something. Some level of corporate internet or something was using something.

Shachar Schiff: It happens often actually. It’s not uncommon, so I wasn’t really surprised when it happened. Once we figured out exactly what it was, then that was a little bit surprising.

Michael Larsen: That actually happens a lot more frequently than you might think. It’s interesting that you would mention that, because back when I was doing a large push for a product for accessibility, that was one of the big issues that we had to deal with. Because, the client that was working with it, they had a lot of customers on a particular browser, because many of their applications key to that particular browser. And, they just didn’t get upgraded very frequently.

Michael Larsen: A business necessity, if you really wanna determine what’s the most important work to do, determine where the money is actually coming from. Well, if you’re gonna be getting a lot of seats for software that you sell, and those lot of seats can pay for an awful lot of your revenue for the course of a year. And, it requires that you interact well with another browser to be able to be effective, you’re gonna make sure it interacts with that older browser, right?

Shachar Schiff: Yeah, absolutely. It needs to work where your clients are at, unfortunately or fortunately, depending on what it is.

Matthew Heusser: I remember a project going back a while, where we had a YouTube embedded widget in a virtual dashboard, and we started to get these complaints. There was this error popping up that YouTube wasn’t supported. And, we wanted to put an effort into it, and I said, “Have them go to YouTube.com. What pops up when they go to the YouTube website?” “Your browser is too old. YouTube is not supported.” “Okay, of the YouTube.com website is not supported, it’s not gonna work in our little fancy widgety thingy that is in our dashboard.” And the entire issue just sort of went away.

Matthew Heusser: And, I do think that aggregating content so that we can go to one sole place instead of lots of other little places, and Gerie mentioned integration. So, seamless integration, so it just all sort of works together, and you don’t have to go … Now for my business, I have to go to QuickBooks and I have to go to the bank, and I have to work them out, because right now the integration is down. At the end of the month I have to reconcile and it happens to work. But, when the integration was working, the transactions were literally automatically flow right through, and I could just click, approve, approve, approve, approve. Balanced out end of month, send, done. Huge difference for someone who would really rather spend their time testing.

Gerie Owen: Yeah. I have an example of that from yesterday. I took my mother for a doctor’s appointment, and this doctor didn’t have access to the medical records from the hospital that she had been in. And, I had to give them all her medications and all of her medical history, when they should’ve been able to pull it up.
Matthew Heusser: I think we really haven’t thought through electronic health records. That’s terrible. I mean, the ability to not bring it up is terrible. What I see more is the practitioner says, “So, tell me why you’re here again?” And you’re like, “Well, I made this appointment two months ago with a specialist to resolve a problem. It was well documented and described.” “Tell me why you’re here again. Let’s run some labs on you and see if the problem is that your iron is down.” “It’s not down. I had that test done four months ago.” Then we may have disappointment. “It’s not my iron” “Well, we’re just gonna run it and check.”

Matthew Heusser: It’s almost as if they’re getting you out of the office, on a bad day. People have good and bad days. But, I think that that’s a good example where no one sat through the, “Okay, what is the business process here?” I’m tested on, it comes back, we make the physician have needs to know everything he needs to know because that person walks in the office. What is that? ‘Cause, it was overwhelming. Even if we gave the doctor 10, 20 minutes to review the file, how does he know what’s relevant?

Matthew Heusser: So, there are people working in that space right now, with machine learning, to figure out what … One company I worked with was working at the welfare space, to sort of figure out, “You’ve got 100 welfare cases. What are the riskiest ones?” “Juvenile cases.” “what are the ones where you have to take action? And then what do you need to know? How do we summarize this huge file of materials you have on?” The kid’s parents were bad, and then he was in an orphanage facility for a while. Then he broke this law. “How do we summarize all that so that you can figure out what to do?”

Matthew Heusser: People are using machine learning for that, which I think is a great example of machine learning plus operational process. And then, how do you test it? We just did podcast on that, but I think we need to get some more work in that space. Most of the answers that I find are maybe a little naïve. Look at it like a human, see if the computer comes up with similar answers. Oh, okay. I mean, that’s better than nothing.

Gerie Owen: And, I think the other thing, the whole digital transformation, any kind of initiatives, it’s not just the initial initiative and the completion of that. But, I think the companies need to continuously reevaluate. Look at how digital technology, how it’s all working. How it’s increasing revenue. What can we do further? I think a digital transformation, it almost sometimes can sound like something that’s once and done, where it’s really an ongoing process.

Matthew Heusser: Oh yeah. That makes sense. How do we keep rationing it up? Thank you Gerie.

Gerie Owen: And, I think that’s something testers kinda keep a pulse on too.

Shachar Schiff: A good example of that could be webpages, company websites before smartphones got introduced. So, people’s attention spans were probably a little bit longer than today, and websites, due to SEO rules 10 years ago, six years ago, were longer and had more words and keywords. Due to digital transformation, everybody had to apparently changed to a mobile first design, with less real estate for your customers to read on a phone. Because, that website that was really good 10 years ago is no longer adequate enough, or it’s no longer good enough for people to read on a mobile device, because it’s just not comfortable and takes too long.

Michael Larsen: The whole responsive design and modifications to being able to make those things work, it does produce some interesting problems. Where I’m at right now, we’re currently going through that with a number of the companies that we work with. The idea is, they don’t wanna have, if you’re on a computer you go to a webpage, and if you’re using a phone you download an app. In many cases, they just know. “We don’t want to work that way.” Just go to one place.

Michael Larsen: So, instead of having to take care of three different code basis for whatever device you’re working on, it’s just one site. But, because you’re scaling it down or you’re moving things around so that it’s visible in a manner on a phone, you do have to take care of the fact that on a website where you have a whole lot of text you wanna deal with. Punch that into a phone, it’s gonna be a lot harder to actually interact with it.
Michael Larsen: So, I think that’s another good example of where a tester can be able to say, “Look this is just not meant to be displayed in this kind of an environment.”

Matthew Heusser: Yeah. What is our abandon rate? Lately I’ve been getting that with captures on my phone, where I’m like, “I’m pretty sure I typed in the right thing. I’m pretty sure I typed in the right thing. I’m pretty sure. Okay, I’m just not gonna buy this. I’m just not gonna use it. Nope. Because, I can’t figure out how to use your mobile app, and I’m not dumb.” That’s even worse is when a user blames themselves.
Matthew Heusser: So, I’ve really appreciated this time. We really have to balance our audiences time with the value that we provide here. We wanna give you the best. We wanna keep it tight. So, final thoughts. I guess I’ll go in a circle, starting with Michael.

Michael Larsen: Digital transformation is a great space for testers to get their hands dirty. A lot of it I think is going to come with time, and it’s going to come with the understanding of what people actually want. I think there’re a lot of opportunities happening in the future, that we need to be a little bit more alert to. The whole medical transparency. Being able to, on side protect privacy of people, but also to be able to share very pertinent information in ways that we haven’t been able to do before. I think that’s gonna be a very telling area, where we’re gonna have a lot of opportunities to work with.

Michael Larsen: Some of that stuff is just not quite ready for prime time. And everybody’s thinking, “We do this once and never again.” No, that’s not the case. We’re gonna have to always iterate on it.
Matthew Heusser: Yeah. Thanks Michael. Shah?

Shachar Schiff: Thanks man. I think digital transformation today, our customers and the users will be ultimately the deciders of what is right or wrong or acceptable. So, for example, as testers, we work often with requirements and design comps and functional requirements, business requirements. And, at the end of the day, your customers and your users don’t really care and don’t know and shouldn’t know what any requirements, business requirements, functional requirements, what they were. They’re not gonna be checking that. What they care about is, “Are you giving me a comfortable user experience? And, is this easy to use?” And if it’s now, then they’re just not gonna come back.

Shachar Schiff: Just like any business, you make money and you survive based on returning customers. And, if your customers don’t come back to your product, regardless of what the requirements were, you definitely have to adapt more for a better digital experience.

Matthew Heusser: Thanks Shach. And Gerie, your final words.

Gerie Owen: I think, to followup on what Shach said, what’s the customer experience gonna be? And, I think we need to start with the question more of, what are you trying to transform? What are you trying to make better for the customer? And then get to the how to do it, rather than deploying something that the customer may not want. And, I think we need to remember as testers that we need to realize that digital transformation is an ongoing process. It’s not a once and done.

Matthew Heusser: Thanks Gerie. Well, thanks everybody for being on the show. I look forward to keeping the conversation rolling.

Gerie Owen: Thank you Matt.

Michael Larsen: Thanks for having us.

Shachar Schiff: Thank you. Thanks.