Getting Started with Automated Testing

June 4, 2020

Automated testing elevates quality, helps teams deliver software faster, and reduces risk. In this post, I’ll share my thoughts on test automation and how best to proceed if you’re new to it all. If you’re new to QA in general, check out my seven quality assurance principles blog post. You already understand the value of QA, you know that it helps you build robust, resilient, and long-lasting products. Now let’s explore how to begin using automated testing to help you build those products efficiently and successfully.

You may have googled recommendations and found an array of manual and automated options. I’ll discuss some things to consider first and then share several tools we employ here at SingleStone.

Note: the automated testing described here refers to functional test automation. Where the test’s objectives are ensuring that the features being built are correct from a business and functionality perspective. There are other types of automated testing (unit testing, integration tests, infrastructure, and security tests) that are also important and are topics for future discussions.

A few words about manual vs. automated testing

This doesn’t have to be an either/or decision. There are certain types of tests that lend themselves to manual testing, and others where it is best for computers to perform the work. As application features continue to be developed, more and more testing may be needed to ensure quality and to prevent any regressions. Think about a scenario where you are 6 or 8 sprints in and your regression test suite may take hours to perform. You may be tempted to cut it back (introducing risk into the equation). This is a perfect scenario to automate portions of your regression tests and allow your manual testing to focus on newer features and riskier or nuanced areas.

Good candidates for test automation are:

  • Portions of the regression test suite. Automate portions that take time to manually test or are error-prone to test. The amount of automated testing will grow over time, providing that safety net and reducing risk. It’s best to keep the manual portion timeboxed and focused on recently developed features and areas where the ROI for automated tests is low (for example: visual and layout tests).
  • Mission-critical areas of the application. Ask yourself the following questions: What areas of the application, if malfunctioning, would render the application unusable? What defect scenarios would damage our brand or result in bad press. The answer to these questions will point you to areas where test automation is most likely necessary.
  • Tests that take a QA resource a long time, are nuanced or are error-prone to perform.

Adding test automation to these areas will save your valuable QA resources and allow them to focus on areas where they have the most impact, such as newly developed features, places where conversations are needed or finding critical nuanced issues.

In summary, a well-rounded and robust QA strategy includes both manual and automated testing—each working and collaborating to focus on areas and activities where they provide the most ROI.

Automated testing tool example
An example of an automated testing tool

Our approach

At SingleStone, we believe in a collaborative Agile QA process because it helps efficiently build high-quality, high-functioning, and well-rounded products. Since Agile may fail over time without automation, we like to employ that form of testing where possible…here are three key reasons why:

  • Test automation brings early and often feedback on the state of the system and frees up other resources to do their best work. Automated tests are easily repeatable and run frequently and can alert teams to any issues that arise.
  • Automated tests also serve as documentation of system functionality. Since automated tests are updated when the system changes, they serve as living documentation. This eliminates the need for static documents, which can get out of date quickly.
  • Finally, automated testing provides a safety net and a feeling of confidence; especially when approaching a production release. In the age of DevOps and continuous deployments, automated tests are not only a good idea, but are mandatory in order to be successful.

Choosing the right automated testing tools

A common question asked is: which automation tool(s) should I be using? There are a bunch out there. They range from free/open-source coding solutions to big commercial products that claim to solve every problem for you.

My short answer to this question is this: you should choose the tool(s) that best enable you to meet your quality goals and fit well with your organization’s culture and team’s skillsets. To further elaborate on this, check out some questions you can ask yourself as you begin to explore what is right for your team:

  • What are your quality goals? What problems are you trying to solve and challenges are you facing?
    • Will a given tool help you meet these goals and challenges?
  • What are your organization’s needs and constraints?
    • Is it budget? Do you need buy-in?
  • Test automation takes some effort to get started and implement successfully. Are you ready?
    • Work test automation into your sprints. Create user stories to get going. Create research and proof of concept spikes. Look for the right opportunities for quick wins.
  • What are your group’s skillsets—both technical and non-technical.
    • Are your resources highly technical? Which technologies and development skillsets do you possess? Use tools geared towards a range of skillsets.
  • Does this tool match your organization’s culture?
    • Are you Agile? Waterfall or somewhere in between? There are some good collaborative Agile automation tools out there (for example: Cucumber). Some tools are not as well suited to an Agile environment.
  • Does it fit well into your team’s technology infrastructure?
    • Can it integrate into your development environments and deployment pipeline?

After answering these questions, decide as a team what tools your group and organization will commit to. Make sure you have the time, resources, and support to successfully begin. Create automation stories and work them through the agile process. Start small, get some quick wins, and then show others what you have learned to build and create momentum!

The automated testing tools we’re using today

Below are a couple of test automation tools we like. Remember, the right tool(s) for you may be different. We like these tools because they give us the flexibility to create robust and easy to maintain tests. They fit in well with our development culture, skillsets, and the applications we typically build.

Cucumber – Cucumber is an automated testing framework that uses Gherkin syntax as the test specification. The tests are version controlled along with the application code. Tests can be implemented in a variety of programming languages (including Javascript, Ruby, JAVA, and C#) using an ATDD (Acceptance Test Driven Development) approach. Cucumber can provide automated testing coverage for web applications written in almost any language and supports front end (UI) tests, integration, and API layer tests. Cucumber is open source and free.

Cypress – All-in-one testing framework, assertion library, with mocking and stubbing. Tests are implemented with JavaScript. Works well for end to end testing. Geared towards today’s progressive web development frameworks (React, Angular, Vue). Open source and mostly free (you can upgrade to gain access to additional support and features)

Here at SingleStone, we deliberately implement automated testing where its benefits can be maximized.

What tools are you or your team using to improve your QA process and increase ROI for your stakeholders? Drop a comment or send a message, I’d love to chat with you about all things QA.

George Hatzikotelis

Test Automation Lead
George Hatzikotelis is SingleStone's QA genius, with 18+ years of experience in quickly learning new technologies, communicating effectively, and meshing with team members to deliver robust, high-quality solutions.