Additionally, somewhat arbitrarily setting up to automate testing can be self-defeating when you don’t have solid requirements/design docs, because then it’s tough to figure out if the errors are being caused by poor reqs/design or because the tool itself (or the test environment or the data being used) wasn’t properly set-up.

If your team embarks on test-driven development at the unit level (“test as you go”), chances are you’re going to get higher quality code than using more traditional dev methods, so that when the app reaches a higher level of testing (integration or system test), you’ll be more likely to get good results using a tool, because higher quality code is being feed into it.

Again, if you work with a tight, high quality SDLC methodology and use approaches designed to ensure high quality output code – from the very beginning of the project – you’ll have far fewer quality issues to deal with at higher levels of application integration (especially if your app has to interface with external or 3rd party apps) and as you get closer to production.

Then, after a first release when the application is stable and whatever defects are present are known, transitioning to more robust test automation (especially when working with a highly complex application, such as a mainframe mortgage banking or insurance app) at that point makes a great deal of sense (it lowers the cost of regression testing as the app is enhanced or updated in various ways).

Interesting thread – what I’ve learned (the hard way) working with Agile teams is that if the “right” balance is not found between streamlining documentation for agility and not having sufficient / adequate documentation, large rocks lie ahead when your project ship puts out to sea and flips over into production.

With my own particular specialization on the link between testing and requirements, I’ve found major problems occuring when the app breaks or testing indicates faulty design or code – which then can’t be traced back to original requirements, because of a lack of documentation.

No need to go nuts on documentation, but having at least well-formed use cases and robustness diagrams (and solid design documents) can help drive better results, so that time-for-testing (which has been squeezed on 100% of the projects I’ve ever worked on) is used to validate that the app meets requirements (the purpose of testing), rather than spending time you don’t have trying to find and report defects (which is not the purpose of testing).

This is particularly frustrating after an app goes into production, when things break and then you can’t figure out what happened or why – because you don’t have sufficient documentation / requirements to go back to review.

Better to spend time up-front doing proper documentation than spending time and lots of money chasing things much farther down the road that should have been done right the first time. Good luck with the team.

I noticed your post on LinkedIn, so I thought I would check this out. I did not see anything mentioned about security, so I wanted to offer some advice. My colleagues and I have come across many applications that met their software requirements, but they did not have any formal security processes integrated into their software development lifecycle. This often leads to high rick vulnerabilities that are found in the architecture or design flaws just before or after the product is in production. Many times these issues could have been resolved at the start and during the development if security had been integrated into the development process. Here are some good starter documents, if you have not already become familiar with a Security Development Lifecycle:

* Microsoft, Microsoft SDL, http://www.microsoft.com/downloads/details.aspx?FamilyID=d045a05a-c1fc-48c3-b4d5-b20353f97122&displaylang=en

* B. Sullivan, Agile SDL: Streamline Security Practices For Agile Development, MSDN Magazine, http://msdn.microsoft.com/en-us/magazine/dd153756.aspx

* D. Wichers, Agile Security Breaking the Waterfall Mindset, AppSecEU08, http://www.owasp.org/images/b/b8/AppSecEU08-Agile_and_Secure.ppt