It’s easy to say that testing is transformed in a virtualized environment. That’s a bit vague, however. An illustration can paint a clearer picture. A typical Value Added Service (VAS) such as a Multimedia Service (MMS) will differ between virtualized and legacy networks environments.
The first big difference involves frequent flyer miles and hotels. In the legacy environment, the vendor of the MMS platform arrives at the service provider’s facility to install and integrate the hardware and software with the local test environment.
The test team checks to see that all the specs are correct, that the platform can be appropriately integrated into the network, that it can be scaled up and down easily and that the applications actually works. These tests are written and carried out in incremental fashion.
At some point, the VAS’ integration into the network, how it behaves and the performance of the services are deemed stable enough and the MMS goes live – perhaps with stops in beta and friendly user environments before full scale commercialization.
In a virtualized environment, there are no racks of servers arriving at the test facility (and – in an important side benefit — no airplane fares and hotel and restaurant tabs for the people who brought them). The virtualized elements are delivered via the cloud. The physical location of the technicians is unimportant.
The two worlds are completely different. Legacy, hardware-based environments are highly constrained. Testers have to pick their spots, so to speak. In software-based testing, test routines are highly automated. There essentially is no limit on the number of test scenarios that can be performed. Many can be done in parallel.
These are significant changes. However, they aren’t the biggest transition in the switch from legacy to NFV test environments.
A “waterfall” approach now is standard. Tests of the VAS platform roll out in a well-planned, incremental manner. These tests include elements such as static testing (a review of documentation including requirements specification and design), tests of scope (what the virtualized network functions will do in the network and what resources it will need to accomplish the tasks set out for it) and others.
This approach is a quaint notion in the virtualized world. NFV uses an agile, DevOps approach. This is characterized by a continuous feedback loop in which testing and development never really end. Indeed, they are not separate. All parameters of the VAS and its impact on the network are continually under scrutiny, and changes in one area may lead to unexpected changes in another. The tests themselves evolve as the feedback loop evolves.
There is one last change. The freedom from hardware constraints enables a new and valuable dimension to be added: Non-functional testing becomes more viable. In the legacy world in which tests are limited, the focus naturally is tests that are deterministic: They focus on whether things are working or not and, if they aren’t, where the problem lies. Non-functional tests focus on elements that are vital but don’t immediately impact services.
