Security testing must have reliable and productive procedures; hence, it is generally worth highlighting their importance. With more than a decade of experience in this domain, Seema Kalwani has an in-depth understanding of how automation can work to simplify testing in security products. The journey of Kalwani from 2005 while working at CA Technologies (now with Broadcom) displays how various programming languages like C, C++, and Python can be incorporated into enhancing the testing process in complex security products.
At Control Minder (now Symantec CA PAM), Kalwani first began testing security products. From 2005 to 2015, she tested many versions of the product, including versions 5.3, 8.0, 8.0 SP1, 12.0, 12.0 SP1, and 14.0. The core of her work was the realization of problems encountered during the testing of large-scale security solutions deployed in different environments.
Security products of that time were mainly set up on UNIX or Windows platforms, and testing would involve testing on multiple flavors of UNIX. Due to such platform diversity, installations were a major challenge because a build might run perfectly on one flavor of UNIX and won't on another due to subtle differences between the platforms.
Kalwani soon realized that testing on such a huge scale required tremendous product knowledge and, of course, an able team to keep pace with the work in hand. A working force of over 20 professionals endowed with diverse skill sets was what was demanded and that was monumental of the testing demands; yet even after that, the workload was a constant bugbear.
Testing different versions of Control Minder, with its array of releases, was no small feat. Each major release required testing of 700+ test cases, an overwhelming task for a team of 30, especially as agile testing processes began to gain prominence.
Agile testing, which involved daily checks to ensure that new code did not disrupt existing functionality, presented an additional challenge. It became apparent that manual testing could not keep pace with the continuous delivery cycles. This is where automation became an invaluable tool. In response, Kalwani and her team developed a homegrown automation tool that incorporated programming languages such as TCL, TK, Python, C, and C++. This tool became the cornerstone of their testing strategy, enabling them to efficiently track the results of daily tests and identify issues early in the testing cycle.
The automation tool worked by running installation tests across various UNIX platforms, providing immediate understanding into which platforms passed or failed. If one platform failed, the team could quickly pinpoint the problem without having to test all platforms individually. This drastically reduced the time spent on testing and increased efficiency. Kalwani’s responsibility was to analyze the results from the automation tool, investigate any failures, and create detailed bug reports. Catching bugs early in the testing cycle was important, as it allowed for quicker fixes with fewer resources. In contrast, bugs caught later in the cycle could take more time and effort to resolve, impacting the project timeline.
As the product matured and the team became more familiar with its functionality, the importance of transitioning from manual testing to automation became clear. Kalwani emphasized the advantages that automation brought to the table. In addition to improving efficiency, automation decreased reliance on particular people. Once the tests were established and running, any team member could analyze the results and open bug reports. This process dependence, rather than individual expertise, ensured that testing could continue seamlessly even as team members shifted roles or left the project.
A fundamental truth in product testing is brought to light by Kalwani's approach: although manual testing is useful, particularly for novices learning the product or when it is still in its early stages, switching to automation as the product develops offers real advantages. Automation significantly maximizes efficiency, improves testing coverage, and facilitates faster identification of issues, all of which are crucial for meeting the demands of modern security product development.
Through adopting tools built with C, C++, and Python, Kalwani and her team set a new standard for how security product testing could be automated, resulting in a more streamlined, efficient, and less resource-intensive process. As companies depend more and more on security products to safeguard their assets, Seema Kalwani's experience demonstrates how important automation is to making sure that these products are fully tested and prepared for use.