The pharmaceutical space (Pharma) is an area where many people have ideas of a tightly regulated environment, where initiatives can range from in-house software tracking systems to literal life-saving devices. Testing in this sphere is often seen by many as an intense challenge but does it really (or should we say “fully”) deserve that reputation?
To help answer that, Gai Anbar joins Matthew Heusser and Michael Larsen to discuss the variations and different areas that make up the Pharma space and how to test in and around it.
References
- Pharma Testing Services
- Pharma’s 6 Crucial Digital Health Challenges (and How to Overcome Them)
- 6 Ways Pharma Companies Can Ease Compliance Burden with FDA’s New CSA Guidelines
Transcript
Michael Larsen (INTRO):
Hello and welcome to The Testing Show.
Episode 112
Testing For Pharma
This show was recorded on February 23rd, 2022.
Many people may think of the pharmaceutical space as an area of tightly regulated environments, focused on literal life-or-death initiatives. What’s it like to test in a space like this? Also, does it fully deserve that “intense” reputation?
To help answer that, we welcome Guy Anbar to discuss the realities of Pharma and how to test in and around it.
And with that, on with the show.
Matthew Heusser (00:00):
Thanks, Michael, for the intro to the show. This time we have Gai Anbar who heads up pharma testing for Qualitest. He was with an acquisition to jump into the space that specialized in computerized systems and digital health within life sciences. So leading quality and IT outsourced work, and now they’ve become part of Qualitest. I’m sure I butchered that. I’m sorry. Welcome to the show, Gai.
Gai Anbar (00:28):
Thank you very much. I’m happy to be here.
Matthew Heusser (00:31):
Can you tell us a little bit more about your background?
Gai Anbar (00:33):
Yeah, sure. So actually my background is in bioinformatics. I have a master’s degree in bioinformatics when they didn’t even call it bioinformatics. It was a new area of data and analysis, data management in the biotech and medical space. Actually, I wrote code to make a living. I worked in a startup for Y2K bug solving so you can understand how old I am. And later on, I managed IT for several biotech companies. The latest was managing IT for the Merck companies in Israel, started my business Comply, which was, like you said, acquired by Qualitest a year ago. So, you know, taking my personal experience and my team’s experience in the real world and providing a service to our customers.
Matthew Heusser (01:28):
Thanks, Gai. One thing I like when we do these interviews is when the founders get acquired and stick around because it’s really the same job only it’s maybe a little bit more access to resources that you get through Qualitest. That’s really neat. So you’ve seen both your background in the pharma space, your current customers, but also the sort of broader, more general IT, that Qualitest serves. How do you see that regulatory space as unique or more challenging?
Gai Anbar (01:57):
First of all, to say that I really like what I’m doing. This is the reason why I’m still around and I enjoy every morning. And I love meeting customers and being a partner to their challenges. Just today, I met with this very large manufacturer and distributor of a drug warehouse implementing new IT systems. So bringing my background, my experience, and also working with other customers with similar challenges, I think this brings a lot of value to our customers. It also makes it very interesting for us. We’re lucky that we’re doing something that we like and something that our customers appreciate and value. Joining Qualitest now, like you said, gives us so much more resources and also a much broader global reach and being able to provide professional services that we wouldn’t have been able to do previously. One of the things that actually in this/today’s morning meeting with this customer, and I know that they really appreciated it, is that when we talked about our approach to quality, is that we know how to integrate different quality approaches, different quality streams, which in many places, there are completely separate manual testing together with automated testing together with validation and verification, which is the official quality layer, which is inspected by health authorities like FDA and European community. So being able to leverage all of these streams with also training, providing support to the vendors of the project. So this brings a lot of value. It reduces the time to market or the go live date of an IT project because your quality streams, they can work in parallel. They can work in synergy to minimize overlaps as much as possible. And this is great because it brings better quality at a lower cost at the end of the day. And by the way, this also meets new FDA regulations, which are planned to be issued in 2022 this year, which are called computer software assurance guideline, which talks about this kind of synergies, this kind of effectiveness, bringing better quality with less efforts.
Matthew Heusser (04:26):
So I’m really interested in the actual details of the regulations. Believe it or not. I’m in west Michigan. And I talked to a local company a few years ago that believe that every test case had to be written on a piece of paper and every test case needed a physical signature. So they, every time they did testing, they had to print out all their test cases and sign them and store them in a folder for that build. And then stick that in a drawer somewhere. And I’m curious how much of that is real, how much of that is historical accident. But before we get to that, our show producer, Michael Larsen had a question or two he wanted to ask and I’ve been kind of dominating for a bit.
Michael Larsen (05:10):
Not a problem. So as somebody who is testing, not necessarily, we’re not talking about testing pharma apps, we’re talking about, you’re doing testing for various companies that do this work. Judging from what I’ve seen from when pharma talks about anything. I think they have to be probably the most statistic loving group of anybody that I have ever seen. I know that metrics has to be brought in here. Like, what are you testing? And when we’re talking pharmaceuticals, it’s a little different than say, “Oh, if you’ve got a glitch in a writing program, or if you have 10% of your bugs or something related to that, it’s an inconvenience, but it’s not gonna be something that’s gonna set the world on fire.” Whereas in pharma, if something is not tested or if something is not confirmed and goes wrong, you can harm or kill people. I hope I’m not putting too much hyperbole there. So I am curious about that. How much of the metrics do you actually address? And I would guess that many of those metrics come down to reducing risks for customers, whereas in the pharma space, risk can be catastrophic in a very real sense. Vague? Is something you can answer?
Gai Anbar (06:30):
Sure. That’s a great question. First of all, let me just explain it. We’re talking about when we look at the pharma space, let’s say two completely different categories of computerized systems. The first is maybe software, which the company creates. Okay, let’s say you are a medical device company. Then you, of course, have software, which is part of your solution that you sell in a pure pharma world, usually you don’t do much of that. You are a consumer of software, let’s say you implement an E R P system, or you implement a ton of different systems in your quality control or production floor labs. So you implement them internally. But when you implement some systems that you buy, let’s say you implement an E R P system like SAP, which is very popular in the pharma world. You need to do all kinds of adjustments. So the system will meet your processes. And there aren’t any two companies, which are exactly the same. So when you take a system or you take some system for your QC lab, some analytical equipment, let’s say, it could be used for all kinds of QC tests, but maybe you use it for one or two specific things with very specific definitions. So our quality processes, actually, they want to ensure the system that you implement, the way you implemented it, meets what we call the intended use. Why did you get it? What do you want the system to do? And then we need to qualify the system according to these specifications. And also from our experience, our expertise, we also know what the regulatory requirements are for such systems, if there are any. The guideline for all of these quality processes is first of all, patient safety, this is the most important part. And the way we do that is that we perform risk assessments. And this means that the whole process is what we call risk based, meaning that, and identify if the system does so many things, identify the places where it could somehow directly/indirectly affect the patient safety. And also, and this is something which is sometimes overlooked, a pharmaceutical company has a responsibility to the public. It’s not just a simple, straightforward commercial entity. It has a public responsibility for the public’s health. It means that if you make a drug and you have patients that need this drug, you need to do all you can to ensure that the supply chain is there, that it is robust. So this is not only money. First of all, the health of the public and the service that you provide to the public. And this is something which we, when we look at an IT system, we look at, so if you do some tests, let’s say for a quality control lab, the system that you’re testing, it could be critical because if some calculation is wrong, then maybe you approve a batch of a drug which should have been rejected. We need to be able to do that. Most of the time, we must do this together with what we call a subject matter expert, SME, from the client who knows the process, who understands the details. They know what’s going on, what is the meaning of everything, every parameter. And they provide the professional guidance on the business side of these systems. At the end of the day, there is zero tolerance for issues or bugs. And this means that if we do find any problem, we need to justify why we can release the system with this bug. Maybe it’s not very critical. Maybe there is some work around. So this needs to be written down, documented, explained, justified correctly. But basically we strive for zero bugs, which is very different from what we’re used to in the high tech industry, where we download some app to our phone. And five minutes later, we need to update it. In the pharma space. This is something which is not acceptable. If this happens, we need to investigate, need to understand why it happened and to improve for next time.
Matthew Heusser (11:01):
So I wanna make sure I got that right. So what you are saying is that you strive to have zero bugs released, or at least no meaningful bugs. When release happens, you don’t know of any defects that are open and the customers ideally don’t find any that matter. The conversation that you had with Michael seemed to me to be very outcomes based. We don’t delay the project. We make the project on time. We provide the information about what’s wrong. So people can fix those defects. That information is actionable, consistent, correct, and complete. A lot of people are hung up on process. How many test cases ran and what percentage passed and what percentage failed and graphs of the defect discovery rate. You seem very focused on outcomes for the customer, which I think is fantastic. Do you provide like a one page status report? How often do customers get updates about what we’re finding?
Gai Anbar (12:01):
Yeah. Our philosophy is that we are an integral part of the project. Of the development process, and we need to be involved early on. The sooner, the better. When we are involved early, we can give our input and, you know, doing things right, and doing things wrong. Sometimes it costs the same when you do it, but then you pay afterwards to fix. So it’s better to do it right. First time, this is what we strive for to do right first time, be there to provide the guidance, the process, to be integrated into the project plan. So if we have a project plan, our deliverables, our milestones are part of the GANT. We can provide these or work on them and build these deliverables in parallel to other activities. So as to minimize their impact on actually on the project schedule, and we can reduce by that, again from this morning’s meeting we can reduce sometimes testing scheduled by 50%, just by being efficient and well integrated in the project, because sometimes customers come to us when they finish and they say, “Okay, we implemented this system, come test it.” That’s too late. They should come to us when they just start. When they say, “Okay, we want to implement a system.” And this way we can contribute, we can bring our value early on with very little effort to it. And of course we report. Depends of course, on the project and the type of development or implementation. Sometimes we even have projects where everything is online in terms of reporting. So the whole team can see at any given time the exact status of how many scripts have been performed, what is the status of them? How many open issues? So everything is very transparent cause everybody knows, “Okay. What is the current quality level of the deliverables of the software that is being developed,” for example. It really depends on the type of system. If you’re implementing an E R P SAP or something, it’s different than, “Okay, we are building something new.”
Michael Larsen (14:05):
So no handoff and wait for testing, but we work together.
Gai Anbar (14:09):
Yeah. I wouldn’t say no handoff, no testing. Cause sometimes you have to say, “Okay, let’s stop. We don’t wanna make any more change. And let’s just do some final QA what we call final testing.” So there is a period for that, but we wanna minimize it. This is something that every customer doesn’t wanna see. So we wanna minimize this as much as we can before we know for this part.
Michael Larsen (14:32):
Okay. Makes perfect sense. All right. Just to make you pivot here. One of the things that I always consider whenever I think of pharma or when I think of what comes into it, we can just take the last couple of years of COVID vaccines and talking about them. And a lot of the communication back and forth is the level of regulation. When something is actually allowed to go across the line. By that virtue naturally, I just think, oh, pharma is loaded with regulations and everything that needs to be done has to be done to this crazy fine tooth comb spec. So I’m curious, how much of that is real or if real is not the right term, how much of it might be exaggerated or what is the meaningful regulations versus what your average, well, Michael, in this case, might think is, “Oh, we have to dot every ‘i’ and cross every ‘t’ and there are no exceptions.”
Gai Anbar (15:32):
Yeah. So I think the pharma industry is unique in that. It’s true that there are a lot of regulations, but I think the pharma industry is unique in that the regulatory authorities don’t detail too much on how to do things. They tell you what to do, but it’s the company’s responsibility to determine how to do it correctly. And of course this is later audited and needs to be justified, but it transfers the responsibility and the accountability to the pharma company itself, which means that, okay, when we test something, the depth of the testing, we need to determine how rigorous testing we need to do. And of course, when you have a new product, let’s say a vaccine that you need to approve, FDA or other health authorities will look at your reports in detail for these. But when you look at the ongoing work of the company later on and they improve the processes and they implement new IT systems, new systems in their labs and production floor. Then of course, you know, you don’t need to approve every kind of change like that. But when authorities come in for an audit every few years, they will look at your practices and you want to be able to show them to demonstrate that you are responsible, whatever you did you did after you did some risk assessments and thinking and understood the meaning of it. And at the end of the day, you want the public to know that when they consume a product that is regulated by FDA or other leading health authorities, you’re consuming a quality product.
Matthew Heusser (17:12):
Well, to get specific, I mentioned the people that thought they had to physically sign on every test case. I think there was some wild thing like printout screen captures and then initial and put the date on it for everything that they do. And I think that was driven by, “We did this 20 years ago and it pass the audit,” this particular implementation of how, passed the what, so don’t change it. And what I was talking to them about was, “Well, if you had a tool that could run through the automation that could take screen, captures it every step, throw it on a network, drive that the folder name was the date or the branch or the date, plus the branch plus the command or something like that. You could have the evidence you need, and this is how to get the signatures. This is well, the tool would know the user ID that ran it and could log that too.” So I think there’s a lot of room to innovate in the tooling and process space. And one way is to do that is to find a partner that says, “We’ve passed audits before we know how to do this.” Is there innovation in that space? You wanna talk about?
Gai Anbar (18:23):
Sure. First of all, the new guidance that CSA and computer software assurance, that’s exactly what they’re meant to enable is to say, okay, let’s innovate, let’s use automation, let’s focus less on documentation, more on the quality of your testing and less screenshots. Okay, maybe you should take a screenshot here and there, but you don’t need to take a screen for every next that you press or every change that you do in every screen. So, you know, more room for flexibility, more room for innovation and around that in order for that to be possible to do you need to define the right processes, put in place the right processes. For example, to be able to rely more, more on your vendor’s quality system. If we buy a product from a well known vendor or some vendor that we initially did some kind of vendor audit, and we know that they’re good and they have good practices and good testing and good documentation, you know, why not rely on their quality? Why do we need to do everything again? These new guidelines, they give room for that. This is something that we at Qualitest, you know, we know how to do with customers. We do that to enable them to improve their quality, actually, and reduce efforts that need to be done for that. What you said about printing all these piles of paper and, you know, killing the trees. It’s true. It’s still happening. In many places, we can work more electronic than we did in the past, but the concept is the same. And this is something that we need to get out of and focus on doing better testing, better quality and not creating piles of useless paper.
Michael Larsen (20:05):
If I can jump in here, I just have a question and I’m not even sure that this can be quantified, but this is just a lead in for what I’m hoping to ask. If we’re talking about the average life cycle of say a given pharma project, is that even something that can be quantified? Is there such a thing as an average life cycle, like we’re used to thinking about releases. Some companies do them, if they’re really aggressive and they’re doing DevOps type stuff, we, we can release a feature a day or maybe several features a day. In the pharma space, of course, that’s not really a reality unless it’s, “Hey, we’re changing up the format for something that you can look at.” If you’re doing something that actually has code and functionality purposes, my guess is, is that that testing might take a while, but to actually release something, does it go on average a quarter, a year, multiyear? And if you’re working on a project, when you have to deal with perhaps long timelines, what are some of the things that you would do to encourage along the way that you are actually mitigating risks that the companies fear? I hope that’s clear enough.
Gai Anbar (21:16):
Yeah. You asked a lot of questions there.
Michael Larsen (21:19):
Sorry.
Gai Anbar (21:20):
(Laughter) and we can talk about that like a week, but let me just say that we cannot really quantify because every project is different and every system is different. What I can say is that we look at it from the life cycle approach, what we call sometimes they say, you know, from cradle to grave of the system, throughout the setup, the implementation, the upgrades, changes, whatever, until you decommission it. And these quality processes have to be all along this life cycle. It’s not a one time activity. You need to maintain it and you need to maintain it efficiently. Maintenance of the validated state is a heavy duty activity. It’s time consuming, it’s resource consuming and being able to streamline all of these activities is really important and provide value. What we’re doing in the past couple of years, more and more is actually introduce automation into the validation space, which is fantastic because an automated script runs very quick with very little human intervention. We know how to define correctly with our customer what is the right balance of automation that could bring value, reducing a lot of the time and efforts required. Another thing is that when a company feels insecure for any kind of reason, they tend to overdo things. And I think that one of our roles is to become a partner, trusted advisor to our customers. It’s not that we just only come in, do a project and go, we build long term partnerships and this way we can provide confidence and this confidence that we provide allows them to breathe, do a little bit less and not be afraid to move forward and do innovative things. I hope I answered.
Michael Larsen (23:16):
Oh, definitely. Yes. So I guess another thing I can ask here or get some clarification, if I may, if you’re doing a pharma project or a program, a project is a one-off. A program, as you said, is a long term thing and you might be running for years, literally, testing that. Do you partner with some people to get these things out or do you work with it over time and develop the expertise in-house to be able to accomplish these? I guess the question I’m after is how do you stay up to date and current with what’s necessary to be able to test these and is that a process that evolves?
Gai Anbar (23:55):
Yeah, first of all, at Qualitest, we have a delivery center for pharma and medical and digital health quality. So the delivery center provides and does consulting for all kinds of companies in all kinds of scopes of projects. Usually short term projects. Short term could be quick consulting, could be for a couple of days, but it could be for a few months. An average project would be three to six months, but being able to run simultaneously several projects in all kinds of different disciplines, different industries, different activities, it creates a lot of knowledge in the company. This knowledge, we’re working hard to maintain it. We have a team just working on knowledge management. So the next customer comes in with a similar challenge. Then, we can pull from this base, you know, shorten the cycle. And sometimes we also rely on some of our partners and also on other Qualitest teams, like for example, in data privacy or information security. So, you know, these kinds of expertise, they exist in the company. And we can also always pull on this resources and this knowledge. Just a recent example I can give you. We have a company who is implementing an automated warehouse management system. We need to do performance testing. There is a performance testing team in Qualitest, which we’ll do that. We can pull on that expertise. So this is the value that we bring. This is how we maintain, you know, our knowledge, our expertise. We participate a lot in conferences, webinars. You need to keep up to date all the time, do a lot of reading, seeing on audit reports from different companies, which are published on the internet, talking to customers. It’s a challenge for us that we must do.
Matthew Heusser (25:49):
Yeah. I think it’s pretty clear. The difference between “we get your one piece software out the door we tested for you and we go away now” and then most projects move into a maintenance or support mode. And there’s a great deal of expertise that’s built up around it. Why not stick around? Because if it’s in maintenance mode still needs to get retested. We may have some expertise there. So it’s been really great talking to you, Gai. We’ve got a few minutes left. What questions should we be asking? What do you want to get out there about pharma testing?
Gai Anbar (26:23):
I would say to our partners, to our customers, my first message would be, let’s not be afraid to innovate. Let’s not be afraid to introduce new technologies into this very conservative space up until now, like AI, automation. That would be my biggest message. Let’s move forward and not be afraid and we will be there to help.
Matthew Heusser (26:47):
Wow, thanks. So I’m gonna give you the last word. We usually move to a section of the show called the last word. If anybody’s wondering about me, I’m gonna be at agile testing days, this June in Chicago USA, and I’m looking into Agile 2022 right now, which is gonna be in Tennessee, summer 2022. Michael, what are you up to?
Michael Larsen (27:09):
At this point? I’m primarily focused on getting ready to speak at InflectraCON. It’s currently scheduled for Washington DC. And at this point, unless something very strange happens or takes us out at the knees again, I plan on being there in person. So watch this space.
Matthew Heusser (27:26):
Fantastic. And Gai, this is the part of the show where we typically talk about what we’re up to, what we’re working on, where people can go to learn more. If there’s a website people can go to and the piece of software that AI, I just, I can’t wait until that starts to be more open sourced and generally available, but where can people go to learn more about you? What you’re up to and pharma testing and qu test.
Gai Anbar (27:49):
First of all, you know, the, the best place to go and look is in the Qualitest website. There is a section for pharma it’s relatively new. We’re adding content all the time to that section. So people can go in and look at it. So if we look under the Qualitest website, we have healthcare and medical devices and all kinds of other interesting sections. So in there we also have pharma, a lot of content, you can contact us through that on our activities and expertise.
Matthew Heusser (28:22):
Thank you so much for being on the show guy. And we’ll see you all again, sooner or later on the internet. Thank you.
Gai Anbar (28:29):
Thank you very much.
Michael Larsen (28:31):
Thanks very much. Thanks for having us. Bye.
Michael Larsen (OUTRO):
That concludes this episode of The Testing Show. We also want to encourage you, our listeners, to give us a rating and a review on Apple podcasts, Google Podcasts, and we are also available on Spotify. Those ratings and reviews, as well as word of mouth and sharing, help raise the visibility of the show and let more people find us. Also, we want to invite you to come join us on The Testing Show Slack channel, as a way to communicate about the show. Talk to us about what you like and what you’d like to hear, and also to help us shape future shows. Please email us at thetestingshow (at) qualitestgroup (dot) com and we will send you an invite to join group. The Testing Show is produced and edited by Michael Larsen, moderated by Matt Heusser, with frequent contributions from our many featured guests who bring the topics and expertise to make the show happen. Additionally, if you have questions you’d like to see addressed on The Testing Show, or if you would like to be a guest on the podcast, please email us at thetestingshow (at) qualitestgroup (dot) com.
