Searching for a Very Specific Type of Tool...

page: 1
2
<<   2 >>

log in

join

posted on Apr, 11 2014 @ 03:10 PM
link   
Hi there,

I spend the vast majority of my time as a software tester. Something I was thinking about earlier today would be very useful as a tool for test automation, but seems to be a really difficult thing to search for. I'll give you a bit of a rundown on the situation that prompts the need, then a description of the tool I think could exist... but is tricky to find.

The Situation

We have an in-house developed automation tool to deal with a development platform that shuns traditional Windows objects. This tool creates text-based scripts that can interact with a built-in communication service to pull out the attributes of objects that are created, and works very well for it. What it lacks is logic, meaning for every potential path through an interface, we would need to script a long set of instructions, then set starting system state exactly, so that nothing breaks along the path to the successful result, or the desired unsuccessful result. Each one of these scripts would likely be exactly the same as another script that tests a different outcome. That means we'd end up with thousands of scripts, and about 75% of the actual content of those scripts would be shared with at least one other script. (There could be no error handling, beyond "Assert there is no error")

The Potential Solution

Since our test tool can be run from the command line, it would be useful if we could easily build a command-line script generator. (This part, I could do in Excel, if I were to get fancy... but I want something deeper). Ideally, I want this tool to read the contents of a directory, and show each file as a card. Once the whole list of cards is present, it would then allow me to arrange them and create associations between them (This one precedes that, these three are possible outcomes of this one, etc.) From there, I could create a new "Scenario", at which point I see the entire path of available actions. Selecting the scripts in the order they should be run then shows a single path through the application, and allows me to save it to a master file which could essentially be a list of command-line calls to a testing tool (I figured it would be more useful to leave out the specific tool, since this could be used for any command-line based tool.)

Master file contents would either look kinda like this:

TestingTool.exe,1,5,4,7,15,34

(Single path on one line - The first argument being the tool used to run the scripts, and the remainder being IDs for the scripts being run. Each of this scripts contains its own setup and teardown, and its own pass and fail criteria, and this system would have to control running the next script if no error condition fails on a preceding script)

or like this:

TestingTool.exe script1.agt
TestingTool.exe script5.agt
...

(Single path on multiple lines - The commands are queued, and will just go through in order... making the master list just a large shell script.)

Now, I know there are parts of this that sound like a workflow design system... but I'm looking for something that will 1) give a graphical view of all covered paths through a system (at least to the level of the test cases that have been scripted) and 2) either create an executable list to test all of the defined paths, or allow the running of these scenarios within its own interface.

Now. Anyone have any idea what I'm talking about? Anyone have any suggestions beyond "Workflow design"?

I'd love to hear it!




posted on Apr, 11 2014 @ 03:25 PM
link   
reply to post by CrikeyMagnet
 


I thought, tools I know tools...you went the complete opposite way with this thread...



posted on Apr, 11 2014 @ 03:30 PM
link   
reply to post by chrismarco
 


My thought exactly!



posted on Apr, 11 2014 @ 03:36 PM
link   
Sounds like you guys don't have the chops to do the job. Hire me and I'll design it for you. Don't expect people on ATS to design it for you free.



posted on Apr, 11 2014 @ 03:50 PM
link   
reply to post by schuyler
 


Not expecting design. Hell, I'll design it and learn to build it, because learning how stuff works is what I do. I'm looking for ideas on how to find an existing tool for this (since it's slippery), or if anyone knows of something like this that is already out there...

You know... so I don't have to go down the road of designing and building, or of securing development resources for it, and waiting 6 months for their allocation.



posted on Apr, 11 2014 @ 04:34 PM
link   
So you want a Vsual Programming Environment? Is that correct?

There are quite a few eamples out there of what you're suggesting, none that I know of are quite the "Holy Grail" that it sounds like you are looking for, but many suffice.

Have you considered using a Game Engine?

They uses a similar logic in their programming approach that you seem to be describing.

We use a few in our company to achieve a variety of logic tasks.



posted on Apr, 11 2014 @ 04:40 PM
link   
Sounds simple enough. I love building scripts like that , i could knock the core of that out in visual basic in a very short time. But i realize that doesnt help you of course.

Designing the graphical output could be tricky though



edit on 11-4-2014 by PhoenixOD because: (no reason given)



posted on Apr, 11 2014 @ 04:45 PM
link   

puzzlesphere
So you want a Vsual Programming Environment? Is that correct?

There are quite a few eamples out there of what you're suggesting, none that I know of are quite the "Holy Grail" that it sounds like you are looking for, but many suffice.

Have you considered using a Game Engine?

They uses a similar logic in their programming approach that you seem to be describing.

We use a few in our company to achieve a variety of logic tasks.


I had not considered a game engine, but the idea of co-opting a tool that is intended to serve a completely different purpose sounds like exactly the sort of thing I'd want to be involved in.
I'll see what I can easily get into. Did you have any particular ones in mind?



posted on Apr, 11 2014 @ 04:47 PM
link   


I thought, tools I know tools...you went the complete opposite way with this thread...
reply to post by chrismarco
 


I thought the same thing,my suggestion... try a hammer...and since I am a sheet metal worker, if that doesn't work, try a bigger hammer.

Sorry brother but you lost me somewhere around the first paragraph when you said something about IDK.... something



posted on Apr, 11 2014 @ 04:48 PM
link   

PhoenixOD
Sounds simple enough. I love building scripts like that , i could knock the core of that out in visual basic in a very short time. But i realize that doesnt help you of course.

Designing the graphical output could be tricky though



edit on 11-4-2014 by PhoenixOD because: (no reason given)


Right. The graphical presentation and manipulation is what I think would really give this legs, though. I would settle for something that can build the relationships and create a scripted path, as at least that would make the planning and creation of paths a lot easier, and enable us to use smaller scripts as "components" of a path, but the graphical interface would make it something that I could pass on to other carbon units to use. And I really dig not doing it all myself.



posted on Apr, 11 2014 @ 04:49 PM
link   
reply to post by tinner07
 


Perhaps not too oddly... I get that a lot.



posted on Apr, 11 2014 @ 05:03 PM
link   
reply to post by CrikeyMagnet
 


You could try Unity 3D, which is quite lightweight and good at 2D graphic interfaces, or if you want something with a bit more graphical grunt there's something like Unreal Engine 4, which has a lot more power ut also makes more heavyweight environments.

You'll see that most of the immediate examples are mainly 3D (which may be a useful approach for what you are trying to achieve), but they can both do 2D as well.

There are many others, but after repurposing video game technology for close to a decade, these seem to be the most complete current options, in my opinion.

Love to hear how you go!
edit on 11-4-2014 by puzzlesphere because: (no reason given)



posted on Apr, 11 2014 @ 05:04 PM
link   

CrikeyMagnet

PhoenixOD
Sounds simple enough. I love building scripts like that , i could knock the core of that out in visual basic in a very short time. But i realize that doesnt help you of course.

Designing the graphical output could be tricky though



edit on 11-4-2014 by PhoenixOD because: (no reason given)


Right. The graphical presentation and manipulation is what I think would really give this legs, though. I would settle for something that can build the relationships and create a scripted path, as at least that would make the planning and creation of paths a lot easier, and enable us to use smaller scripts as "components" of a path, but the graphical interface would make it something that I could pass on to other carbon units to use. And I really dig not doing it all myself.


Well as for work flow i would split it up into 3 tasks

1 - first write the core to produce command lines you want
2 - make something to gather the results and store them in a way that they can be interpreted by another program
3- make something that coverts the results into graphics.

Its one step of a time with this kind of programming. The important thing is working out how the results are going to be stored so they can be easily converted into gfx at a later time. That kind of design takes a bit of experience to get right first time.

By the time you get to step 2 then you have to know if you are going to code your own gfx display program or use someone else's. its pointless jumping head first into step to with out a clear idea of what you are going to do for step 3.

Thou to be honest i usually just jump in and then abandon it if i cant manage the last step. for me its more about the challenge of the design rather than the end product sometimes lol.



posted on Apr, 11 2014 @ 05:06 PM
link   
reply to post by puzzlesphere
 


I was heavily into coding and mapping for the unreal engine 3.5 for about 5 years. Ive got a ton of scripts at the unreal wiki


Unreal engine 4 already has a graphical programing environment rather like the output that the OP is looking for. Its all flow charts and boxes, also a bit like the new texture editor in Autodesk 3D Max 2014.



edit on 11-4-2014 by PhoenixOD because: (no reason given)



posted on Apr, 11 2014 @ 05:09 PM
link   
reply to post by PhoenixOD
 


Yes... ;-j

That's why I suggested a game engine.

Many of them do much of what he is suggesting, even some other paramtric modelling software might be a good tool, depending on what the final outcome he is looking for.

Yahoo Pipes may even be an interesting option.

;-j



posted on Apr, 11 2014 @ 05:10 PM
link   
reply to post by CrikeyMagnet
 


You may have to research deeper but try this list: Wikipedia - List of GUI Testing Tools

I personally used to use the Rational/IBM Tools ages ago but dev has moved on and they are probably different now.



posted on Apr, 11 2014 @ 05:12 PM
link   
Okay. For now, I'll start by checking out Unity and Unreal engines. (I like to jump in too.)

Then I'll likely get a spreadsheet going as a proof of concept, where it builds simple commands out of a selected file and its input parameters. (partly because I like doing spreadsheets with real functionality in them)

Er... I'll update further once I decide whether I can get to that point.

Thanks folks!



posted on Apr, 11 2014 @ 05:13 PM
link   
reply to post by puzzlesphere
 


The yahoo pipes look interesting.

For me i prefer the beauty in solid code, i never got on with these new flow chart style programming languages.



posted on Apr, 11 2014 @ 05:14 PM
link   
reply to post by CrikeyMagnet
 


Good luck, keep us updated



posted on Apr, 11 2014 @ 05:14 PM
link   
reply to post by PhoenixOD
 


Awesome!... so you were using UDK?

What's nice about UE4 is that they've completely reworked it so it's all C++ now... much better for compiling, plugins, external libraries, etc. ;-j

Were you making games?





new topics
top topics
 
2
<<   2 >>

log in

join