Transcript for:
Goals and Strategies for Test Automation

in this chapter we'll take a moment to consider what is your goal for starting a test automation initiative and what is it that you want to accomplish who do you envision participating in your test automation initiative and in what capacity and how do you plan for the execution of this strategy so what is your goal is it to reduce the time that it takes to execute regression testing perhaps your team needs to release features faster so you want to reduce the manual effort it takes to run the regression test and allow your testers to focus on testing the new features or perhaps your goal is to reduce technical debt by closing your Sprint's with test automation in place for the new features as your developers are creating new features and your testers are verifying them there's not much time for full-on manual regression testing and closing the sprint without automating the newly identified test only as to that problem so maybe you want to take care of this early on or perhaps your goal is to enable continuous testing as part of your build process after checking in new features your developers may want the confidence that what they've added isn't compromising the overall quality of the application adding automated tests to your continuous integration pipeline can certainly help with this the key here is to understand what it is you're trying to accomplish before you begin any planning take a moment to decide why it is that you need test automation make sure this goal is at the forefront of all of your decisions because your subsequent actions in regards to test automation should be aimed toward this goal who you involved the tools you use which tests you automate the execution strategy all of this should be aligned with your goal communicate the goal to the entire team make sure everyone is aware of why the team is doing test automation and what it is you all wish to accomplish by it I stress is because many teams jump into the implementation of test automation without taking a moment so consider why they are doing it this leads to failed automation projects where teams are unprepared for the issues that present themselves and are unsatisfied with the return on their investment let's have a look at considerations that should be made when defining your strategy holes are you envision participating in your test automation initiative and in what capacity before answering this let's talk about the implications of automating your regression test you need someone to write the test automation scripts you need someone to monitor the results every time the tests are run you need someone to update the test when the behavior of the application changes these tasks can all be done by the same person or shared amongst the team but let's explore this a bit ok so who's going to write your tests let's say you choose the developers because hey they already know how to code well your developers are working on new features will they realistically have the time to work on scripting regression tests ok so let's say you choose the manual testers as the ones to write the test scripts because hey they're better at testing anyway let's remember the problem we're solving you're investing in test automation as a way to reduce the time and effort required for a regression testing can you now spare the testers to not only test new features but also to spend time automating old tests here's the hard truth unless you're able to pull your developers or testers off of their current task of working on new features or you're able to provide them with the time and resources needed to do these tasks these may not be your best options make no mistake test automation is a software development project in and of itself it takes a considerable amount of time and skill to do it if you treat it as a side task that people only contribute to whenever they have time it will fail even if you're able to find the time to allow your developer or tester to write the automated scripts there's still the task so monitoring them and updating them as needed so what do you do well you have a couple of options here some companies assign someone to temporarily work on automating their regression backlog however this only works if you have another plan and strategy for future tests otherwise the regression backlog will continuously grow as new features are developed and you'll find yourself back at square one another option is to assign someone to permanently work on your test automation this becomes their a full-time job and yes in most cases it is indeed a full-time job you can assign a developer or a tester to this role or have them work together as a pair but be sure to account for the fact that either these would have to invest in learning test automation yes your developer knows how to code but how familiar issue with testing and test automation your tester understands testing but how familiar is he with development and test automation don't get me wrong this strategy can certainly work but you must be realistic about the learning curve in time investment and it also really helps if the person actually wants to do this work another great alternative is to employ a test automation engineer this person is ideally skilled in testing as well as automation development they can leave your test automation efforts and include the developers in the test and a more reasonable capacity if you go this route is critical that you make this person a standard part of your team having them interact and collaborate with the rest of the team members will strengthen your test automation initiative now if none of these options seem possible ask yourself if you're viewing this as a side project are you willing and capable of investing what is required for this to succeed let's say you define your strategy and you have an idea of who will lead the effort as well as how others will contribute the final thing we'll consider in this chapter is how do you plan for the execution of your defined strategy let me offer a question to ponder when your team was executing these regression tests manually do they run all of them every time or do they make a strategic decision based on other factors such as risk and value you may think that since you're automating these tests you might as well automate all of them but there is a cost that comes with automation the time and the maintenance required is not something to be taken lightly so automated tests you need a clear understanding of the scenarios are the actions and expected results clearly defined this may be covered BIA acceptance tests gherkin scenarios or test cases and while verbal communication is wonderful a written summary of expectations even if light is extremely helpful when writing test automation the test script authors mine is swapping from creation mode to verification mode as they're writing this test code so having a guide to consult with to ensure nothing is missed is key you need to choose tooling for your automation project such as a programming language or a coalesce offering just as with your development projects you'll also want to develop standards and conventions especially if you envision multiple people contributing to the automation project will explore considerations around these in Chapter four on Tooley explore your thoughts on winning how these tests will be executed is it okay to have someone kicked them off when needed and report back with the results or do you want them executed multiple times a day within their own CI job where everyone has visibility into the results or do you want them integrated into your development CI job where they're executed anytime someone checks in new code each one of these requires a certain level of trust and focused attention let's look at each one running the automated tests locally and reporting the results the team requires manual intervention and therefore will indeed be slower but the result reported to the team has already been triaged and can be trusted to be accurate running the test multiple times a day in a CI jobs separate from development gives faster feedback but the reported results have not yet been triaged so the tests themselves have to be written with a greater level of proficiency in care to ensure that they're not flaky and that they can be trusted and running the tests as part of development CI process requires fast and reliable tests that only fail when the application under test has provoked it too I recommend gradually moving to these phases starting out with running locally or as part of a private CI bill that is not exposed to others this will allow you to learn more about what causes your test to be flaky including quirks about your application once you find that you can trust your tests yourself move to a separate CI job that is visible to everyone but doesn't affect the we're checking this strengthens your trust and what you've written and also allows others to begin to trust it as well once you are very confident in your test only then should you get your check-ins with them by adding them to the development CI process it's best to realize early on that your test failures will need to be triaged and maintained and plan for this and your strategy again an automation project is a software development project and as the development of your product changes so much your automation code I cannot stress this enough it's not an optional task your automated tests will need to be triaged in maintains so plan accordingly the good news is that this is a great opportunity for others on the team to assist as your developers run tests to verify that their new features didn't break any of the existing ones they can be responsible for triage and any failures ideally any tests that fail will be due to their changes and therefore they can fix their code or update the test to reflect the new expectations of the application however also be prepared for unrelated tests to fail as well this will give developers insights into things that cause flaky tests and ways that they can improve the tests automate ability of the application manual testers are also a valuable option for triage and automation failures especially ones who have an interest in contributing to the automation code base reviewing test failures debugging the automation code and updating the code when needed are great introductory opportunities into test automation while all developers and testers can certainly assist in the triage and updating of test automation scripts because it is not their primary responsibility there's still a need for someone to monitor this consistently unmonitored tests are untrustworthy tests as your team continues to lose trust in the automated test they tend to rely less on them and the automation project eventually dies