There is little doubt at this point in anyone’s mind that Selenium is seeing wide spread adoption, evident in the fact that even historically reluctant organizations are welcoming this open source tool into their kit. So what is Selenium anyway? Selenium is an open source tool for automating web browsers. That is really all you get and, as many organizations are discovering the hard way, this product tackles the problem of test automation in a radically different way than most other market leading tools of the past and present.
A large challenge with creating any test automation suite is ensuring simultaneous scalability, maintainability, and reliability. Engineers need to spin up new tests quickly and consistently. New builds might, and often do, break things in existing automation, but resolution should take minutes or hours, not days or weeks. Furthermore, automation that works 80% of the time is really not useful to anyone.
But this is only one part of the whole story. While the same challenges that we’ve always faced still exist, an automation suite around Selenium WebDriver needs to further address the measurement and reporting of test coverage and test success as well as build a strategy for change tolerance through object recognition and code maintainability. Such challenges are absolutely surmountable and worth tackling to unlock the power and accuracy that an automation suite built around Selenium can provide.
The keys to unlocking success with any Selenium implementation are good design and process. Testers may not be developers, but adopting tried and true development practices can go a long way in such an endeavor.
Keyword driven testing (KDT) enables an organization to separate the design and documentation of test cases from the data they consume and the mechanism by which they are executed. KDT, in its goals, is similar to the model-view-controller (MVC) pattern of software design. Much as MVC addresses the need to separate presentation from data modeling from functionality, KDT allows us to separate tests from data from execution.
KDT is quite different from a lot of other test automation design approaches in that it has a strong focus on design and business logic, rather than focusing on the technical aspects of an application under test. The idea is to model application behavior with your tests while hiding the technical details of test implementation behind the scenes, exposing only that information which directly impacts a keyword’s usefulness to modeling business logic. So how do we do this?
Develop your framework. Unlike many other popular approaches to test automation, KDT is not as strongly supported by third party tools and frameworks. While the end game is to produce less technical test automation, this is accomplished with some technical magic behind the scenes in which you’ll need to invest up front.
A successful KDT framework implementation needs to do a handful of things very well:
- Provide a strategy for object recognition and enable the test designer to externalize object identifiers
- Make keyword development as light weight as possible. Provide a hierarchy of abstract keywords that provide common functionality to prevent re-implementation many times.
- Minimize the effort involved in deploying new keywords. This kind of functionality can be part of the framework and transparent to both the keyword developer and the end user.
- Allow configuration of your front end (excel spreadsheet, XML files, etc.) without making it too cumbersome to use.
- Support data driving of tests as strongly as makes sense. Automation should be both configurable and reusable.
- Generate easy to read reports of test execution.
- Provide concise and clear documentation of keyword behavior that does not require highly technical or assumed knowledge. Consider using tools like javadocs to generate this documentation from source documentation. Custom doclets can be created to generate convenient HTML files for distribution.
1.1 The Good
There are several things that Keyword Driven Testing truly excels at:
- Strong separation of the technical complexities of test automation from the business logic of an application
- Increased accessibility for consuming test automation by non-technical personnel
- Increased capacity for collaboration between different teams in the organization to achieve comprehensive test coverage
- Managing maintenance costs normally associated with test automation
- Abstracting technical details away from test automation design
- Enabling data driving of test automation
Successful KDT implementations are driven by strong design practices. To leverage the benefits of this approach, establish practices and process around your keyword design process. Create test cases at a high level that provide good test coverage and follow up with a detailed design of all of your keywords before writing any code.
1.2 The Bad
There’s no “one size fits all” test automation approach and as such, it is important to know the weaker points of any design pattern.
- KDT does not inherently provide an object recognition strategy like some of its counterparts. One must be designed and implemented as part of the framework
- In some organizations, the design overhead that KDT creates does not provide enough value to be worth the time, especially in those organizations with especially fluid development practices that position test and development very close to one another.
- KDT can take more time to spin up due to the high cost of developing a framework than some of its counterparts.
Page Object Model
Page Object Model is a design pattern which, in several key ways, is juxtaposed with KDT. While KDT models application behavior and business logic, we do the exact opposite with Page Object Model. This approach seeks to create page objects which model the UI components of our application in order to accomplish two things in particular. First, we expose the services of a page to the test developer as a sort of API. Second, we abstract the deep knowledge of the page structure away from the automated test itself.
A successful page object implementation reduces maintenance costs by creating a clear separation between what our interface does or enables our user to do and what a page looks like. If business logic remains relatively stable the presentation layer of the application can change significantly without creating much rework.
1.3 The Good
Successfully implementing the Page Object Model will:
- Generate extremely high tolerance of changes in the UI of an application
- Provide an intuitive model for automating workflow driven applications
- Leverage parallelism with other design patterns like Model-View-Controller (MVC)
- Reduce code duplication
- Provide a built in approach to tackling the challenge of object recognition
1.4 The Bad
There are some disadvantages to the Page Object Model approach, but most involve dealing with edge cases or uncommon scenarios. The real disadvantage of this approach comes from the high degree of technical knowledge required to implement and maintain it. Page objects make no attempt to close the gap that exists between business logic and test automation. Test creation and execution are both technical tasks under this approach.
BDD combines the principles of test driven development with ideas from domain-driven design to enable collaborative software development. This is accomplished primarily by establishing a domain specific language (DSL) that allows teams to express the behavior and expected outcomes of application functionality through natural language.
This means that tests can express conditions, actions, and outcomes in a contextual and understandable way with the technical details of their implementation hidden in the background.
1.5 The Good
BDD does an excellent job of closing the gap between test automation and business logic, but the true power comes in how flexible it is on the back end. The most amazing part is how seamless it can be to bring these pieces together in a technology stack that makes sense in your organization.
An implementation of BDD will:
- Ensure that requirements and application behavior are clearly communicated
- Enable easy communication of requirements and behavior across diverse groups
- Allow for easier debugging and maintenance of test automation
- Strongly enable testing from the user’s perspective
1.6 The Bad
While the concept of BDD might sounds attractive to nearly everyone, it necessitates a huge paradigm shift in the way we build software. This gap is smaller for organizations actively leveraging a test driven development methodology, but for those that have completely different models for developing and deploying software, the change is quite considerable. Similarly, BDD requires that we change our culture and mentality around creating software in a pretty significant way.
Beyond the huge paradigm shift that some organizations would experience, BDD also requires that we introduce additional tools into our technology stack, and in some cases this may be undesirable. For more information, look into tools like Cucumber or RSpec as options that strongly enable the creation of DSL and the implementation of logic behind it.
Properly evaluate your options and settle on the approach that best suits your organization. The first step is properly select a tool or set of tools to enable the automation of your application functionality, then carefully gather your requirements.
Make sure that you do not fall into the trap of creating one-off test automation scripts that address only specific application requirements. Sometimes, organizations mistakenly conclude that they need to reuse existing manual test suites that were not designed with modularity and reusability in mind, or that their applications are too unique or complex for generic design approaches to work. The long term cost of creating bulky and inefficient test automation scripts will outweigh the cost of immediately designing good test automation. Strong design is your number one key to success.