Certified Software Testing Professional Learning Resources Automated testing

Learning Resources
 

Automated testing

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.


The  principle of automated testing is  that there is a program (which could  be a job stream) that runs the program being tested, feeding it the  proper input, and checking the output against the output that was expected.  Once  the  test suite is written,  no human intervention is needed,  either to run the program or to look to see if it worked; the test  suite  does  all  that,  and somehow indicates  (say, by a :TELL message  and  a  results  file)  whether  the program's  output was as
expected.  We, for instance, have over two hundred test suites, all of which  can  be  run  overnight by executing  one job stream submission command;  after  they run, another command  can show which test suites succeeded and which failed.


Open Source GUI testing tools

  • Abbot
  • AutoHotkey
  • Cobra - WinLDTP Windows version of Linux Desktop Testing Project by VMware
  • CubicTest is a graphical Eclipse plug-in for writing Selenium and Watir tests.
  • Dogtail by Red Hat
  • Dojo Objective Harness is a Web UI testing framework from Dojo, similar to Selenium.
  • Linux Desktop Testing Project by freedesktop.org
  • Maveryx is an automated functional testing, regression testing, GUI testing and data-driven testing tool.
  • QAliber by QAlibers, free open source for testing applications and web over windows OS
  • Selenium for Web UI testing
  • Sikuli for testing any GUI based upon screenshots
  • SWTBot functional testing of SWT and Eclipse based applications
  • Tellurium Automated Testing Framework (runs on top of Selenium).
  • Robot Framework via Selenium, Sikuli and other Libraries
  • Watir browser driver for web UI testing
  • Xnee Recorder and replayer for X Window System



Proprietary GUI testing tools

  • AutoIt Microsoft Windows GUI Testing. Free of charge but not open source.
  • eggPlant by TestPlant Ltd
  • froglogic's Squish, object-based cross-platform, cross-GUI-technology testing
  • GUIdancer by Bredex, for Java (Swing, RCP/SWT, GEF) and HTML
  • HP QuickTest Professional by Hewlett-Packard (formerly by Mercury Interactive)
  • HP WinRunner by Hewlett-Packard (formerly by Mercury Interactive)
  • IBM Rational Functional Tester by IBM
  • IcuTest GUI unit testing for WPF
  • Infragistics TestAdvantage
  • iMacros
  • QF-Test by Quality First Software, for Java/Swing, Eclipse/SWT and HTML only
  • Ranorex
  • RIATest for Flex
  • SilkTest by Micro Focus International (formerly by Segue Software then Borland)
  • Soatest (absorbed WebKing starting in version 6.0) by Parasoft
  • Test Automation FX Windows UI testing with Visual Studio
  • TestComplete by SmartBear Software
  • Testing Anywhere
  • TestPartner by Micro Focus
  • TOSCA Testsuite by TRICENTIS Technology & Consulting
  • Telerik Test Studio by Telerik
     

 

ADVANTAGES
Automated Testing
Manual Testing

• If you have to run a set of tests repeatedly automation is a huge gain

 • If Test Cases have to be run a small number of times it's more likely to perform manual testing
• Helps performing "compatibility testing" - testing the software on different configurations • It allows the tester to perform more ad-hoc (random testing)
• It gives you the ability to run automation scenariosto perform regressions in a shorter time • Short term costs are reduced
• It gives you the ability to run regressions on a code that is continously changing • The more time tester spends testing a module the grater the odds to find real user bugs
• Can be run simultaneously on different machines thus decreasing testing time  
• Long term costs are reduced  
DISADVANTAGES
Automated Testing
Manual Testing
• It's more expensive to automate. Initial investments are bigger than manual testing • Manual tests can be very time consuming
• You cannot automate everything, some tests still have to be done manually • For every release you must rerun the same set of tests which can be tiresome
OTHER FACTORS
• The performance of yout test tools
• The knowledge level of your testing team
• The continous growth of software to be tested
• Number of necessary regressions
 

 

 For Support