posted on Apr, 11 2014 @ 03:10 PM
I spend the vast majority of my time as a software tester. Something I was thinking about earlier today would be very useful as a tool for test
automation, but seems to be a really difficult thing to search for. I'll give you a bit of a rundown on the situation that prompts the need, then a
description of the tool I think could exist... but is tricky to find.
We have an in-house developed automation tool to deal with a development platform that shuns traditional Windows objects. This tool creates
text-based scripts that can interact with a built-in communication service to pull out the attributes of objects that are created, and works very well
for it. What it lacks is logic, meaning for every potential path through an interface, we would need to script a long set of instructions, then set
starting system state exactly, so that nothing breaks along the path to the successful result, or the desired unsuccessful result. Each one of these
scripts would likely be exactly the same as another script that tests a different outcome. That means we'd end up with thousands of scripts, and
about 75% of the actual content of those scripts would be shared with at least one other script. (There could be no error handling, beyond "Assert
there is no error")
The Potential Solution
Since our test tool can be run from the command line, it would be useful if we could easily build a command-line script generator. (This part, I
could do in Excel, if I were to get fancy... but I want something deeper). Ideally, I want this tool to read the contents of a directory, and show
each file as a card. Once the whole list of cards is present, it would then allow me to arrange them and create associations between them (This one
precedes that, these three are possible outcomes of this one, etc.) From there, I could create a new "Scenario", at which point I see the entire
path of available actions. Selecting the scripts in the order they should be run then shows a single path through the application, and allows me to
save it to a master file which could essentially be a list of command-line calls to a testing tool (I figured it would be more useful to leave out the
specific tool, since this could be used for any command-line based tool.)
Master file contents would either look kinda like this:
(Single path on one line - The first argument being the tool used to run the scripts, and the remainder being IDs for the scripts being run. Each of
this scripts contains its own setup and teardown, and its own pass and fail criteria, and this system would have to control running the next script if
no error condition fails on a preceding script)
or like this:
(Single path on multiple lines - The commands are queued, and will just go through in order... making the master list just a large shell script.)
Now, I know there are parts of this that sound like a workflow design system... but I'm looking for something that will 1) give a graphical view of
all covered paths through a system (at least to the level of the test cases that have been scripted) and 2) either create an executable list to test
all of the defined paths, or allow the running of these scenarios within its own interface.
Now. Anyone have any idea what I'm talking about? Anyone have any suggestions beyond "Workflow design"?
I'd love to hear it!