Agile Test Automation – Practically

In the last post I mentioned that our automation team had been able to achieve in-Sprint automation. We achieved most of what I wanted, which was a huge step forward for this client. In this post I want to share some practical tips that will help you achieve an Agile Test Automation program.

Test Automation is a partnership – Developers and test developers need to work together to create testable code. By the time we reached our goal, the UI developers were creating the visual framework first with all object selectors in place and giving this to the test developer. This set of selectors was considered a contract, and was minimally dependent on the xpath. UI developers didn’t change those selectors without telling the test developer the what and the why. UI developers also minimized the impact on the automation developers where possible. The result was reduced churn in test development leading to faster, more reliable automated tests.

Test Automation is software development – Software developers have learned a lot over the years and test developers need to apply those lessons. Externalize configurations into files so tests can be moved from environment to environment. Put reusable components into libraries so a test bug can be fixed once to impact every test using that component. Create small modules to enhance readability and maintainability, especially since you can’t test your test until the developer completes the code. Use or create frameworks to simplify development. Make components as stand-alone as possible so test artifacts don’t create false errors. Finally, implement quality development behaviors such as code standards and peer reviews. The result is much higher quality tests that stand the test of time.

Automated Tests are a tool for developers – Make tests runnable in any environment so developers can use them for Test Driven Development. By the end of our project the developers were running the automated tests and helping the test developer resolve test issues before giving a build to QA. This helped the developer with a shorter feedback loop and all the debugging tools he or she needed to analyze a problem. The result was much higher quality software on first release to QA.

Be Agile in Test Development – All the agile values and principles work as well for test development as for software development. Create critical tests first and iterate to a more complete suite. Communicate with the software developer. Pick tools to allow fast failure and continuous integration. (We waited much too long to try to get our automation suite into the CI tool due to tool issues.) Focus on end user value. In the case of test automation that’s the developer who needs to use the test results to fix bugs. Keep things simple and focus on technical excellence. The result will be a better functioning team that truly values your contribution

If you’re a test developer, learn what these things mean and how to apply them to your tests. Other team members make sure these are applied in your test automation program. The results are well worth the effort.

Posted in Quality Assurance | Tagged , , , , , | Leave a comment

Test Automation

I’ve been telling QA professionals for at least 5 years that they need to learn test automation. I don’t mean a record and playback tool. I mean writing test code with the skill of a software developer.

Why? The most obvious reason is regression.

In an Agile project the development team is adding functionality in increments as the project progresses. As each increment is completed it needs to pass new and regression tests. It doesn’t take a genius to see that regression testing will eventually overwhelm the manual testing team. At that point the team can try to find additional testers or reduce the number of tests. But reducing the number of tests introduces risk.

The solution is automation. The automation tool can run regression on each build, ideally as part of the build cycle. This frees the test team to focus on the new functionality, the best return on the human investment.

The other reason is less obvious. People are creative and most tend to like expressing their creativity. Coming up with new tests is creative. Executing them isn’t. If your test plan is in the form of an automation script you’re focused on the satisfying creative work.

Conventional wisdom is that automation scripts can’t be written until the code is stable. In our most recent project we proved conventional wisdom wrong. In my next post I’ll go into some of the specific behaviors that made early automation possible. For now I’m asserting that it can be done, making a more satisfying and successful test effort.

Why not have separate manual and automation testers? Soon it will come down to economics. When a team can do all its testing with automation the manual testers will be a luxury. If you are a QA professional, you don’t want to wait for that to happen.

Posted in Quality Assurance | Tagged , , , , , | Leave a comment

The Agile Plan

It’s a common notion that Agile methods like Scrum mean no planning, just doing. That’s myth. The reality is that Scrum provides and organic way to handle surprises that force a plan to change. This means you don’t have to be afraid of change, unwilling to move forward until everything is understood. But making room for surprise is very different from not planning.

When we started my last project we didn’t know enough to identify the complete data model, so we modeled the data we knew about. It was a given that there would be model changes so we developed processes and found tools that would allow us to adjust as necessary. The data model we could design, and the process for changing it, were key parts of our plan. This kind of planning is required if you want to succeed with Scrum.

You might think planning for change meant we didn’t have to plan the model. But you’d be wrong. We’d minimized change’s pain but we hadn’t eliminated it. We planned the data model as completely as we could with what we knew. We let the design of our code, tools, and process handle the surprises.

Dealing with surprise is Scrum’s strength, but only with good planning. As anyone would expect, those Stories without surprises were developed faster and with fewer bugs than the Stories that contained surprises.

So don’t fall for the myth. Make good plans and deal with surprises when you have to. That’s true Agility.

Posted in Management | Tagged , , , , , , , | Leave a comment

Process vs Passion

When working an Agile project I spend a lot of time trying to get past the “when is it due?” question. Whether it’s from team members or from other teams, people still have a tendency to focus on the process of developing software and where they fit in that process.

Agile isn’t about “when is it due?” Agile focuses on “when can it be done?” And, while that can be a little frightening, it’s freeing as well.

The fear comes from being concerned others won’t think you’re good enough.  When everything is “as soon as possible” people think that means “fast.” At Scrum meetings I hear explanations for why it will take a long time or why it’s taking so long instead of answering the three basic questions (what did I accomplish, what will I accomplish, what is blocking me). It takes a while for people to understand that we’re not trying to find a scapegoat, and we’re not asking them to commit to the impossible. We are just asking what they can do, and then asking them to do it.

Once they understand, then the freedom comes in. When you don’t have to worry about how you’ll be judged you’re free to work the way you think you should. People who have accepted this lesson stop worrying about making excuses and start asking why they can’t do more? If the team says we can’t do the User Interface until we’ve created the Application Programming Interface the UI designer asks why he can’t mock the API until it’s ready. If the tester is told he can’t validate the results until the UI is in place the tester asks whether there are test tools he can use to validate under the hood. The team starts to find new ways to get things done sooner, because sooner means they can do more.

I titled this process vs passion because process can sometimes kill passion. But I have to caution that all process can’t be thrown to the winds. Scrum specifies a Definition of Done and a Working Agreement because some process is the glue that holds a team together. Too much process, however, smothers the team. So we need to create enough without creating too much.

I’ve written before about making your work place one without fear. This is another reason why it’s so important. When people are afraid they hide behind process. But when they are free to explore their passion, you will see amazing results.

Posted in Joy, Success, Uncategorized | Tagged , , , , , , , , , | Leave a comment


I want to start by saying that I’m not a professional recruiter. I’m certain they have challenges I know nothing about. But I know the way recruiting works has to change if we’re going to see Agile methods take hold.

A key element of Agile work is a small team that develops a good working relationship. The team’s ability to interact replaces the heavyweight process that comes with traditional management. Changes in any team’s membership has an adverse effect on its performance. But this is compounded when the team is using established working relationships to keep the project flowing smoothly. This means we have to be able to find good replacements quickly. Current recruiting practice doesn’t seem to lend itself to that.

We found great difficulty getting past what people can talk about but can’t actually do. I believe our project worked on the leading edge of what the latest technologies offered. Since what we were doing was so new, we didn’t expect to find people who had already done it.  We were looking for people who understood the basics of the frameworks we were using. Then we could teach them what they needed to know to move forward.

I don’t know whether we were not finding people who really knew what they could speak to or we were not finding people with whom we could communicate. What I do know is we had to work through many new team members who required more time in instruction than they gave back in completed work. We finally wound up with a team that was able to succeed, but we wasted a lot of time getting there.

I’d be interested in your suggestions for improving the process of finding people. Is it better testing? Do people need trial exercises on-site, such as hack-a-thons? Should we bring people on part-way and only start paying them when they start delivering? In that case, what about the cost of on-boarding someone?

I’d love for this to be a post about what I know. Unfortunately, this is a post about what I know needs improvement. But I don’t know how to improve it. Feel free to make your suggestions in comments.

Posted in People | Tagged , , , , , | Leave a comment

The Right People

An Agile project needs good people. But what makes good people for an Agile project? They need skill in the work at hand, of course. But you need more than just skills. There are some other competencies that are critical to helping an Agile team work well. Here’s a short list of key competencies.

Courage – Agile developers need to have the courage it takes to try things. They also need to have the courage to commit. Both of these imply the courage to make mistakes and to admit them openly. Without courage your team will be too tentative. They will tend to over-estimate effort (whether you’re using Story Points or something else). They’ll be afraid to set a target and work toward completing it. Worst of all, they will make excuses rather than face up to their problem areas. Courage to face your weaknesses gives you the opportunity to improve or help the team compensate. If a team member makes excuses rather than addressing their weaknesses they eventually have to leave the team. (Of course, you need to give them room to be courageous. But that’s the subject of another post.)

Transparency – This goes along with courage. Agile developers need to be willing to share exactly what’s going on, the good and the bad, in the work they’re doing. They need to be quick to ask questions and ask for help. They need to share their blockers so the team can work to unblock them. It’s can be frightening to be so transparent, especially if your experience is punishing bad news. People will try to hide the bad and hope for the best. That kind of delay in admitting a problem makes dealing with it all the harder. An Agile team needs to deal with the truth if it’s going to succeed.

Ownership – Your team members need to feel that team success is their success. You don’t want people who are complacent about delays or problems with their work. And you really don’t want people who don’t accept responsibility for how the team is doing. Obviously, things happen that team members can’t be responsible for, but team members need to take enough pride in their work to want to produce good results on time. You want people who will find ways to meet their commitments, not shrug when they miss them. Team members need to own their targets and do what they can to achieve them.

Ability to Plan – Many people think Agile methods don’t require planning. But nothing is accomplished without a careful look at how it can get done. Agile team members need to be able to break their work into steps and create a plan for accomplishing them. Having a plan allows the team to assess how it’s being executed. The more granular the plan the more effective the team will be at spotting or even avoiding potential problems. A good Agile team member can make and work a plan, and make noise when the plan runs into trouble.

These are some of the competencies, beyond skill in the role, that an Agile team member must possess. If you’re just focused on skilled people, you’re not going to succeed with your Agile team.

Posted in Management | Tagged , , , , | Leave a comment

An Agile Firm Fixed-Price Contract

On February 1, 2017, I started as the Scrum Master on a Firm Fixed Price contract. We had a specific Scope of Work and a target end date. The planning team had reviewed the Scope and assessed the number of Scrum Teams we would need to complete the scope on or about January 7, 2018.

It’s now December of 2017 and we are on track to complete the project when we said we would while staying within our budget. We have already released some of the highest quality software I’ve had the pleasure of overseeing. The project goal is moving systems from a mainframe computer to a modern n-tier application. The end-users will have to get used to the new interface, but we all believe they will appreciate how much easier their jobs will be after they learn the new interface.

While leading this project I’ve learned some very important lessons.Over time I’m going to be sharing these lessons. Some are no brainers, although it’s been nice to have them reinforced. Some surprised me. I’ve learned lessons about people, agility, the customer, scope, quality, and tools. I’ll be sharing on all of these topics.

I’m still convinced that Scrum is the process that any organization should use to manage software development, and probably the rest of its projects. But Scrum is a framework that’s designed to flex to fit the needs of individual teams and specific projects. I’m not going to recommend a specific way to run Scrum. But I will tell you where and why we made adjustments, and I’ll share what I liked and what I didn’t about the choices we made.

All that I’ve learned will help me do a better job as I help more organizations embrace Agile and Scrum. I hope you’ll read along and share your own experiences.

Posted in Success | Tagged , , , , , | Leave a comment