DevOps: tools for the Gartner "Verify" toolchain step

Today we will cover the "Verify" tooling from the Gartner Toolchain. Gartner defines "Verify" as follows:

Verify includes all the activities required to ensure the quality of the release. These activities can occur simultaneously with creation activities. It is important to describe specific activities designed to deliver high-quality code prior to deploying the release into production. This will include tests for both functional and nonfunctional (performance, security, usability, system environment configuration and compliance) validation. Create and verify are generally connected together via continuous integration facilities and the use of automated tools to evaluate the created components.

In another publication Gartner says the following:

Verify includes all the activities required to ensure the quality of the release. As with Create, in some DevOps environments, Verify activity is included in software creation. However, it is important to describe specific activities designed to deliver high-quality code prior to deploying the release into production.

Here's a summary of verify activities:

• Acceptance test
• Regression test
• Static analysis (quality and compliance)
• Security analysis (vulnerability)
• Performance test
• Defect status
• Configuration test
• Release test

Here's a list of tools for the "Verify" step, subdivided in four categories:

Test Automation

• HPE
• IBM
• Micro Focus
• Microsoft
• Sauce Labs
• Thoughtworks
• Tricentis

Static Analysis

• Cast
• Microsoft
• Optimyth
• Parasoft
• Semmie
• SonarSource

Test Lab

• Delphix
• Microsoft
• Perfecto
• Quali
• Qualsys
• Skytap
• Sauce Labs

Security

• HPE
• Trend Micro
• IBM
• Trustwave
• Veracode
• Whitehat Security

General

The profile (skills needed and role) of a tester are changing. They have to be involved from the start and be part of the planning process. Having insight in the code and knowing how to analyze it becomes also necessary. There is a shift from “the lonely tester” to a team effort, where also developers are part of in order to make sure that quality is part of the software development process. In other words testing is everyone's responsibility.

Importance of testing

Today testing is seen as one of the main components of a DevOps toolchain. This because today it is still labor intensive and as such costly.

Changed nature

To put it a bit simple, initially testing was about functionality: does it do what it is supposed to do?

After that performance testing came and what you could see as the “new kid in town” we have security testing, not to forget the GDPR (General Data Protection Regulation) from the EU that also will require additional testing.

Even Static Analysis is seen as part of a testing suite. I agree it is part of what software quality should be, but I wouldn’t put it into the testing tools. Static Analysis should be part of the Planning cycle.

During Planning, when we talk about legacy systems that need to be adapted, we should investigate what the Technical Debt is.

Ward Cunningham describes Technical Debt in 1992 as follows:

"Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise."

Who does what?

Initially a developer was supposed to do his own testing. But you can’t be hunter and forester at the same time. That is why testing became a separate discipline. It is fully understandable that testing is everybody’s responsibility.

But we should be careful and not concentrate everything around developers that in some approaches are also responsible for testing. Yes their code should have the desired quality and they should pay attention to what is called technical debt. If they do that, the testing effort will become automatically less time consuming and complex.

Take away

Make sure your DevOps process is well defined, that “Testing” is taken into account from the planning stage, that every stakeholder does his part. Make sure you automate the provisioning of your test environment is automated and that the quality assurance people can start testing right away without having to organize the test environment themselves.

And that you and they know what version they are testing as development goes on and new builds may be made available.

In the next post

Next we will cover the PREPROD from the Gartner toolchain.

About the author

Hello, my name is René De Vleeschauwer.

Throughout my career I've been responsible for the development of enterprise software. Since 12 years I've been leading the development of IKAN ALM, an open DevOps framework.

Your E-mail address

Your question

Optional email address

Send my question

Form by ChronoForms - ChronoEngine.comDo you have any questions about this post? Just ask me!