Source
r/softwaretesting•
Soan

Recommendation for test case management tool

Hey all, I’ve recently moved to an organisation that has no defined QA practices. Teams are agile and adding their test cases to Jira ticket as comments. I’m looking at different tools to find a more efficient way to maintain the test library. I’ll be using this to define regression tests and automation strategy as well. I’ve looked at Zephyr for Jira and wasn’t too impressed. I’ve used practitest in the past and am leaning towards it. I figured I’ll post here in case there are any new tools in the market that I should be looking at. So hit me with your recommendations on test management tools that play well with Jira.

Posted Sep 30, 2020, 2:53 AM UTC

735
Annanondra•

We use testrail which integrates easily with jira. We chose it because it’s an independent platform so if the company decided to change process tools our tests wouldn’t be lost. It’s rather cost effective as well.

It supports test case structures with detailed steps like quality center, a loose test deign method, and exploratory charters.

It is customizable so you can add fields if needed and has a fairly easy to use reporting suite including scheduled email reports.

8
JoeDeluxe•

Another good thing about TestRail is their API. One project I was working on we were running automated tests from our IDE and the test case status was updated to pass/fail in TestRail at runtime.

3
iliestani•

My favorite tool is Test Rail too. I also use ALM and Zephyr at my current work place and I'm not too happy about that.

2
degeneratepr•

Just curious, why aren't you happy with those tools?

1
iliestani•

They are harder to use and not that user friendly. Especially ALM. The version on ALM I need to use is only accesibile with Internet Explorer and we don't have Jira integration.

2
sonofabullet•

I'll sound like a broken record, but what problem are you trying to solve?

In order of importance the changes go like this

  1. people
  2. processes
  3. tools

Tools are an emergent property of processes. You pick the tool that best suits your process.

And Processes are an emergent property of your people. You develop the processes given the abilities your people have.

Unfortunately, a lot of interventions default to fiddling with tools instead of addressing people or processes.

Are you sure that you'll fix the problem by replacing a tool (comments in Jira) with a different tool? Or will you just move the buckets around, force people to re-learn an new tool and not net any software quality improvements?

You mention you're looking for a more efficient way of maintaining a test library. Is it possible that the inefficiency you currently have actually results in better quality product?

Think about this like a tester, what are the second and third order effects of replacing a tool on a team of people?

7
SoanOP•

Fair enough. I’m already working on people and processes part of the things. My main issue with keeping things in JIRa is that once sprint is done, that work done to identify test cases is gone (or hidden along with 100 other tickets). We have worked with team, so people are now documenting what they are testing. We have made testing as part of done criteria, so it’a part of our development process. I’m now at a stage of streamlining it, so we can reuse regression set + look at making it more efficient. You effectively cannot start to improve your testing unless you know how much testing is being done. With everyone doing their own thing within the tickets, we are most likely creating redundant test cases and missing out on scenarios that normally would be captured by regression.

3
mandibt•

You can also give Xray a shot (a Jira plugin)
In my experience, in terms of simplicity and covering almost everything a testing team could need - TestRail is the best test management tool that offers that.
The only drawback that somewhat irritates me is that when you (if you) move to automation, you can't fail test steps but the whole test case which we overcame with Xray.
On the other hand, as u/Annanondra mentioned, if you ever want to migrate or change other tools in the process, you can easily import/export tests from/to TestRail or keep it as an independent test case management tool.

Edit: link removal for Xray as per suggestion

3
ocnarf•

Thanks for your contribution. Could you remove the link to the commercial testing tool as this is not allowed in this community.

1
noobul•

Just my 2 cents. I used to work in a place where we had no test case creation. We just defined scenarios and variables that we're going to test and we had a excell file with the regression tests. Was it a 'best practice' scenario? No. But for our specific use case it was great. The application was pretty complex but it relied on ad-ons. And we basically tested most of the software's functionality in different scenarios and environments. If you're in a similar situation you're probably looking to resolving a problem that doesn't exist and would just bring extra work for the QA team.

2
SoanOP•

We don't have a dedicated QA team, just different team members helping out with testing. That's why I think having a dedicated test case library would help everyone.

1
SebastianSolidwork•

I'm using this approach in Confluence/Wiki, mostly for regression:

  • write frequently used instructions (I don't call them test cases) of actions and checks at a common space
  • make a page per release and make lists (similar to mind maps) and/tables for everything to test. Here should be all observations and decisions noted (e.g. "failed but accepted because...")

For new features I'm fine with noting this at tickets in a bug tracker. The communication with the developers should be short as possible. But just deltas in the comments, a full coverage/plam/status in the description. Extracte here what is useful for regression and take it to your Wiki.

Over all I love mind maps. Testing is a iterative process, priorities and content changes easily with new information.

2
blueboybob•

SynapseRT is a Jira plugin

1
feliciasnh•

We use TM4J to document our manual testing. It integrates really well with Jira issues, and because it is a repository based test management tool, it lets us easily use, re-use and maintain the tests. Our agile teams are pretty happy with it, it does everything we need.

1
vladamir_the_impaler•

You should have 99% automated integration and E2E tests existing in SpecFlow within a solution/project. When they run, they should report back to a db which PowerBI points to for reporting. You can also run SpecFlow reports to see all of your scenarios/tests in case people want that.

There should be no "test case tracking tool" at all as it would just be a duplicate of the automated tests you have in the code.

If you think there are manual tests which are valuable outside of automation....then create automated tests to do those actions, and use SpecFlow and PowerBI to report on them as above.

-2†
paulmwhite•

Whilst I agree that test effort focus should be on automation, suggesting a c# specific framework and a specific reporting tool is quite naive.

I do agree with the sentiment though, IMO manual test effort is still valuable, but the time should be spent on exploration, not writing manual test scripts.

5
vladamir_the_impaler•

Why is it naïve?

I provided specific solutions instead of something overly vague. With those specific solutions you can extrapolate substitutions if needed.

Also, your tests don't need to be written in the same language as your applications.

Regardless, there are SpecFlow-like alternatives such as Jasmine for the js side etc. If you google SpecFlow you can arrive at these.

Don't be a fucking dick.

-3
simonsglcfc•

Chill with the reply my dude, there's no need to be that aggressive. Many companies cannot have 99% automated tests, and some people prefer to not use a BDD approach.

5
vladamir_the_impaler•

Dude has nothing to add to the conversation and calls someone naïve when he's actually the one naïve by assuming that tests need to be written in the same stack as the applications - if he knew anything about automated testing he'd know that is not the case.

So he added nothing and wrongly criticized someone...he's a douche.

It's fine if someone doesn't want to do BDD or automation, but that's not what he said. And if someone doesn't like my suggestion they can move on rather than insulting me for actually adding the conversation.

-2
ocnarf•

Please keep all conversations professional. We are here to argue about opinion, not people!

1
vladamir_the_impaler•

Tell that to the fool that insulted me for no reason.

You know, you guys can have this thread, I'm dropping it - not that you'd care.

The key problem is a bunch of QA people sitting around and hypothesizing on the principles of testing like it's some classical philosophy or something.

Automation. That's it and that's all. Exploratory testing? Define the scenarios and automate them.

You guys keep asking questions that were relevant ten years ago, like what test case tracking tool to use etc. Ensuring quality isn't about manual testing anymore or updating spreadsheets or tools with Pass/Fail statuses.

Coded automated testing is THE ONLY way to do it now and in the future and any company not on board with that is falling behind in the world of continuous delivery and deployment.

You guys ask questions and then don't want to hear the answers. This is probably why you are still QA's and not feature developers yet. Your jobs will be gone in 5 years and you can see the decreased number of postings now along with the explosion of SDET reqs.

But yeah, keep asking questions about test case tracking tools and ignoring the reality of the situation. Good luck.

0