Cytoscape Wiki
Cytoscape 3.0 Functional and Acceptance Testing
For 3.0 we would like to develop a suite of automated acceptance tests that check the basic functionality of Cytoscape.
- The purpose of the tests is twofold:
- Provide a simple way to validate that expected functionality is present.
- Serve as regression test to verify that no functionality has been lost or broken.
- The tests would be delivered as one or more separate, standalone modules. These would not be unit tests verifying small parts of the system, but rather integration tests than span many modules and require the GUI to run.
- Ideally the tests should be specifiable in English-like prose, so that non-technical users can write the specifications.
Decision:
Which selection of tools should we use to accomplish this goal.
Options
JUnit Only
We could simply use the JUnit framework and express our specifications as a suite of tests. We can use tools like the fest toolkit to execute GUI widgets.
Benefits - Most developers are familiar with JUnit and it would therefore be relatively easy for them to write tests.
Costs - The tests would require knowledge of Java to understand.
JBehave
JBehave is a Behavior Driven Development framework that allows test cases (a.k.a. "stories") to be written in English text, which are then mapped onto JUnit-like test cases.
Benefits - Tests are written in English, so anyone can write a test case, which developers can then implement. Tooling appears fairly robust and support for Maven is good.
Costs - Mapping the English text to test cases could certainly become time consuming.
JDave
JDave is another BDD framework that uses a "fluent" Java API for expressing test cases. You might see code like this: specify(stack, must.be.full());.
Benefits - Fluent APIs can be easier to use that plain Java. Specifications are easier to write since they don't require separation of spec and code that runs the test.
Costs - Writing the test specifications is more complicated JBehave since it involves a Java-like syntax.