Take control of your test automation with Delta Reporter
Monitor your test results with this open-source tool
Automated testing is commonly seen as a solution to all quality issues in product development. Obviously, it’s impossible to click through the entire application manually every time there is a new release. Sounds simple: just automate the repetitive tasks and free up your time. But when you start writing automated tests, you realise — they’re easy to write, but the maintenance is a huge task, that can exceed the manual testing time. The tests keep failing, no matter how hard you try 😭. (Spoiler: it’s not that bad, we’ve got a solution!)
How did we come up with the idea of Delta Reporter
Our team faced the problem I described above. So we started to brainstorm on what can be done to reduce the maintenance time 🤔 . We made a list of issues we were experiencing: flaky tests on our side, unstable environment, high volume of deployments, inability to coordinate efforts between QA team members, black-box regression without even knowing what features were released today. The list could go on forever. We knew we needed something smart. Something that could gather all of the test data on each release, analyse the failures, collect the metrics and show the high-level and low-level information on demand. We searched the web to see if such a tool existed, but everything was either not powerful enough or too heavy to be deployed on our environments. So, we just decided to build one ourselves! 🎉
Delta Reporter Overview
So here it is: our super-fast and super-shiny tool that we have been using for over a year now.
Some of Delta Reporter’s powerful features:
- Real-time updates. When the test run starts — Delta Reporter is immediately updated, so you can check the test results as the tests run. 🏄
- Flaky tests badge. There is this flashing red sign ❗️ letting you know, that this test failed more than 5/10 times. So you instantly know, there is a potential issue there.
- Screenshots. If you are running selenium tests, you can view the screenshots and video recordings for the failed tests . Very handy, as you no longer need to go and reproduce the test yourself to figure out the issue.
- Filters. For your convenience, you can filter out only Failed tests. Let’s be honest, who cares about the Passed ones 😃
- Test history. There is a tab, where you can view all the historical runs of this particular test. It gives you the option to quickly check how this test behaved before.
- Resolution. This is the yellow badge on the screenshot above. You can set it for each test, so when you or your colleagues scroll through the results and see, for instance, “Product defect” or “Environment issue”, they instantly know there is no need to spend time checking this failure now. And this is what we are trying to achieve — reducing the number of test failures we need to go through! And make the collaboration smooth and less painful 🐼
- Charts and graphs. We know how your manager loves a good chart. Ours does also! 📈 We are currently developing a charts page, where you’ll be able to filter by the project, resolution, date, test type and many-many more options. Apart from the shiny metrics for the tops, these charts will be invaluable to the QA team, as it’s easier to prove there is a constant environmental issue or the devs are pushing too many bugs, when you have a good proof. Use these stats to highlight the issues and start improving the quality!
- Another cool thing is that we can combine all the test results (unit, integration, e2e) under one “launch”. So that when there is a release, everything is there for you in one place.
- And the last but not least, the tool is open-source!
Frameworks supported:
- WebdriverIO — end to end selenium based framework
- TestNG — api test framework for Java applications
- Junit 4 and 5 — for Java apps unit testing
- Cypress [we are currently working on the implementation]
- Pytest [we are currently working on the implementation]
Where to start
First of all, please visit our Delta Demo Page! (it might be a bit slow to load).
If you are happy to try it out locally, good news! You just need to clone the repo and run “docker-compose up”! 😝
You might also want to deploy the tool on your environment. We promise, it should be easy enough! Please check out our Getting Started Guide for the details.
