Recently, we sat down for an interesting discussion with James Bach of Satisfice, Inc. to pick his brain on upcoming testing trends. Bach started in the field of programming, but determined that his passion instead lies in software quality analysis and improvement, not creation. He does a lot of work with individuals and teams using the knowledge he gained working for Silicon Valley companies like Apple Computer and Borland to help them with risk analysis, test design, and the design and implementation of computer-supported testing. Here he discusses crowdsourcing, DevOps, and risk analysis, and the effects we could expect them to have on the testing industry.
“If a big segment of the industry loses sight of the value of testing, crowdsourcing may be a fig leaf that industry segments use to enable that move. Essentially, Google does that now with its bounty program, where they pay people who find security problems in their programs.”
What do you think of crowdsourcing companies and organizations?
They seem to be good at helping solve one particular kind of testing problem involving compatibility testing. They also provide a way for new testers to get their feet wet, although it seems that many new testers don’t get very far with them.
On a global scale, looking at the way most organizations test, we estimate that about 60-70% is done internally, about 30% is outsourced, and maybe only .05% is crowdsourced. Do you see that changing in a big way?
Potentially, it could. If a big segment of the industry loses sight of the value of testing, crowdsourcing may be a fig leaf that industry segments use to enable that move. Essentially, Google does that now with its bounty program, where they pay people who find security problems in their programs.
What other trends can you tell us about?
It’s hard to determine if any trend in testing is really a trend because it’s so hard to get data. I could say that intellectual testing is on the rise, based on comparing today with 10 years ago. But I don’t have a way to compare that with the rise of stupid testing.
How about DevOps? Where do you see that going in regards to the testing industry?
For certain kinds of applications, the idea of throwing up a site and then trying to make it better while it’s in the field seems to work often enough that lots of people want to try it. It is risky, but maybe not too risky; it depends on what kind of thing you are doing. As we see, it has not worked with Healthcare.gov. Of course, there were a lot of factors at work there, not least of which was that that was not a situation where they could put out a version 0.1 and then revise it; they needed the 1.0 from the start.
Are there any other big issues that you think testing could address?
Here’s the great big issue for the future: risk analysis, understanding and controlling your risk (whether or not testing is involved on a large scale). Testing is a classic approach to risk analysis, but the more mature your technology and your corresponding process of development is, the less need for super-fantastic testing; as such I don’t need to “test” my food, because I can trust that the system that provides it to me probably isn’t poisoning me.
What about challenges for people in your line of work specifically, as someone who works with project teams and test engineers instead of the programs themselves?
There are a couple of challenges for people like me, going forward. One is packaging what we do into a rhetorical framework that people in different communities can accept. For example, I teach testing skills, and those skills are important, but they might not be CALLED testing skills in the future. The same abilities and practices may be called “live-site risk management” or even just “software engineering.” That means to be heard, I need to keep updating my exercises and ways of speaking.
Another challenge is that my form of consulting and theorizing is grounded in experience, which means I need the nourishment of projects to keep growing my methodology. Since a big challenge to testing practice is coming from Continual Deployment projects, I am going to need to start one or join one in order to give my ideas the best foundation and properly integrate my thinking with that practice.
As this interview makes clear, the questions which plague the tech world in general – the best way to secure talent, the best methods for development – are also concerns within the testing industry itself. We are happy to have had the opportunity to discuss these topics with Mr. Bach. For more information, please visit Mr. Bach’s website, Satisfice.
