I like to think of Cucumber as a philosophy, much like Agile or even the MVC design pattern. And just like Agile or MVC, you get the best out of Cucumber when you execute its related processes as they are applicable to your team, product & project.
As a QA Automation Engineer, I've been involved in the testing of various aspects of application quality; performance, load, UI & APIs. I have experience with tools like JMeter, Saucelabs, Selenium, Cypress, TestRail , NightwatchJS & Webdriverio.
Recently, I worked on a project with Cucumber-based test automation and throughout this experience, I was able to identify situations where Cucumber was beneficial and others where it only caused more problems. I'll be sharing the main takeaways from this experience in hopes that it will help you to make an informed decision about whether Cucumber is the right fit for you.
Cucumber was originally created as, 'a collaboration tool that aims to bring a common understanding to software teams - across roles.' It seeks to accomplish this by encouraging project teams (business analysts, tester, developers etc.) to discuss features to be implemented, the expected behavior and how to test it, thereby employing BDD. It goes without saying that this should all happen before any application code is written.
The way in which these system behaviors are documented are in a format known as Gherkin and the overall intention should be to automate the testing of the application features where the Gherkin steps are translated into executable steps that assert the behavior of the system. The most common mistake I have seen however is that teams try to integrate Cucumber into their product development as a tool because they think it will improve their testing in terms of transparency, readability & overall understanding of results.
This brings me to my first point.
1. Don't use Cucumber as only an automation tool.
Let me clarify this point. Don't use Cucumber as just an automation tool and then blame Cucumber for not solving the problems with your automated testing.
Let's use Agile as an example. If a team decides to become Agile, but they only ended up doing Standup meetings, they would be missing out on all the other valuable ceremonies and important processes involved in Agile. They would most likely give up on it believing the methodology doesn't really solve any problems.
Cucumber is so much more than a testing tool; it involves having project meetings to discuss specifications and then documenting these in Gherkin so all stakeholders are on the same page.
2. Use Cucumber at the beginning of the project.
Don't try to inject Cucumber into a project hoping it will solve the problems you are having with system specifications. The most valuable activities of Cucumber come from having those initial project meetings where Gherkin steps are clearly defined (This, in itself, is another skill). This documentation based on mutual understanding is then used to drive testing and development.
So trying out Cucumber during a project that wasn't initially equipped for it is much like building the foundation for a house and then deciding to build a skyscraper on the house foundation. You are probably going to run into more challenges than usual, increasing both the length and complexity of your project.
3. Don't use Cucumber if you don't have the resources to support it.
Don't underestimate the required resources to start and maintain a project that is using Cucumber, especially if it's the team's first encounter with it. Plan for additional resources in the areas of:
- Training time & material.
- Human resources for training.
- Extended project timelines, especially in planning and requirements documenting phases.
- Increased resources to maintain automated tests.
Expect some growing pains. Once again, much like Agile, some teams embark on the journey and underestimate the amount of time and energy to shift how development is done (When I say development I mean product development from idea conception to delivery to the end user). Additional key resources to consider are a team that is excited/interested to try it & a champion for Cucumber; someone to spearhead Cucumber who has experienced it before.
4. Use Cucumber if the team will benefit from it.
One of the key benefits of Cucumber is its claim to improve collaboration. My personal advice would be not to implement Cucumber if your team can already communicate requirements well. From observation, this usually happens in smaller companies where there aren't clearly defined roles of business analysts, developers, testers, project managers etc. Usually these roles are split among a small team who, more-or-less, share the same skill set and communicate using the same jargon and no details are lost since no translation is needed.
In this scenario, this team may not see any true benefits from Cucumber and may even become frustrated with what they see as unnecessary steps to getting their product out the door.
5. Don't use Cucumber to reflect implementation.
This is closely tied to point number 2, but it's important to distinguish this one. Cucumber should drive the implementation of application code and automated tests, it should not rely on the existence of either of these.
If this is the case, it will result in the team trying to play catch up in the area of documentation and then trying to fix requirements to match the system specifications. Furthermore, it could lead to an inconsistent product being built out that doesn't quite match original plans or scope creep where new requirements have to be defined all together mid-project.
Choosing to begin your project journey with Cucumber will require adjustments and consideration of your team, your resources and the project itself. So is the juice worth the squeeze? For my team, it was. It is up to you to decide if Cucumber is the best fit. My advice: Go all in. Use it on a new project to see its pitfalls and advantages first hand.
Cucumber is no silver bullet, but really, which framework is? So I think it is up to you to try different approaches, try to mix and match the best features of the ones you like that would best suit your team, your resources and your project.
Try being a chef instead of a cook…
In future posts I will go over A suggested approach to automation with Cucumber & WebdriverIO & Integrating WebdriverIO with Cucumber & TestRail.
Other articles written by creators and practitioners of Cucumber.
The world's most misunderstood collaboration tool