Skip to main content

Empowering QA - A Modern Testing Strategy with Feature Flags and Bucketeer

· 5 min read
Yoshifumi Kakinoki
Yoshifumi Kakinoki
Software Engineer

Are you a Quality Assurance (QA) engineer struggling with these daily challenges?

It works in development, but you can't test it in staging right away because there's no data.

The combination patterns for each feature are too complex to test everything.

In today's world, where development cycles are sped up by Generative AI, the role of QA is more important than ever. Yet, preparing test environments and testing complex feature combinations takes a lot of time and can slow down development.

What if you could freely switch specific features ON or OFF without changing any application code or environments?

This article will show you concrete ways to use Bucketeer, a feature flag and A/B testing platform, to solve the problems QA engineers face and drastically improve test quality and efficiency.

What are feature flags for QA engineers?​

Simply put, a feature flag is "a mechanism to remotely toggle a feature's ON/OFF switch from outside the source code."

For a QA engineer, this is like getting a "magic switch" that can dynamically change the application's behavior. By operating this switch from the Bucketeer dashboard, you can instantly create the necessary test scenarios—without having to redeploy or ask developers to make configuration changes.

Three Use Cases for Bucketeer to Dramatically Change the QA Process​

Let's take a look at some specific usage cases.

Case 1: Limiting test subjects and conducting safe "production environment testing"​

When testing pre-release features, it's often difficult to create an environment that is perfectly identical to production. However, testing directly in production is risky.

[The Challenge]

  • A feature is fully developed and merged into the main branch, but it shouldn't be visible to all users yet.
  • You need to test this feature in a way that only members of the QA team can access it.

[Solution with Bucketeer]

You can solve this challenge smartly by using Bucketeer's User Segmentation.

  1. Create a new segment named "QA Team" in the Bucketeer dashboard.
  2. Add identifiers to this segment, such as the account IDs of your QA team members or your office IP address.
  3. For the new feature flag you want to test, add a rule to turn the feature ON only for users who belong to the "QA Team" segment.

This setup allows general users to continue seeing the existing functionality, while only QA team members can verify the new feature's behavior in the production environment. This enables safer, more reliable testing using real production data and infrastructure.

Case 2: Collaborating with E2E testing to improve test coverage​

Automated E2E (End-to-End) tests are a cornerstone of quality assurance. However, trying to cover all ON/OFF patterns for a feature can complicate the test code and make the setup for specific states difficult.

[The Challenge]

  • You want automated tests to cover scenarios for when the new feature is both "enabled" and "disabled."
  • You want to avoid changing environment variables or modifying the database state for each test run.

[Solution with Bucketeer]

The solution is to incorporate a scenario in your E2E tests where you log in as different test users.

  1. Prepare test accounts in advance, such as "Test User A (new feature ON)" and "Test User B (new feature OFF)."
  2. In the Bucketeer dashboard, set up rules to turn the feature flag ON for User A and OFF for User B.
  3. In your E2E test code, simply log in as each respective user and verify that the application behaves as expected.

In this way, the test code can switch the function on and off simply by changing the logged-in user. This simplifies test preparation and makes it easy to improve test coverage.

Case 3: Making complex combination tests surprisingly simple​

In modern applications, it's common for multiple features to interact with each other. The application might behave completely differently when "Feature A is ON and Feature B is OFF" compared to when "both Feature A and B are ON."

[The Challenge]

  • You need to test various combination patterns of multiple feature flags.
  • When testing manually, asking developers for configuration changes or waiting for deployments for each setting change is time-consuming.

[Solution with Bucketeer]

The answer is simple: just click the ON/OFF switches for the flags in the Bucketeer dashboard.

A QA engineer can have the application open in one browser window and the Bucketeer dashboard in another.

After verifying the "Feature A ON, Feature B OFF" state, they can simply flip the switch for Feature B in the dashboard, reload the browser, and immediately test the "Both A and B ON" state.

There's no need to ask a developer or wait for a new deployment. This allows QA engineers to conduct combination tests autonomously and quickly, at their own pace. As a result, the effort required for regression testing is significantly reduced, freeing up more time to design critical test scenarios.

Conclusion: Shifting the QA Process from "Reactive" to "Proactive"​

Integrating a feature flag platform like Bucketeer into your QA process brings more value than just a new tool. It leads to:

  • Improved safety and reliability of tests.
  • Increased coverage and efficiency for test automation.
  • Reduced effort for manual testing (especially with combination tests).

The testing process, which was often "reactive" due to environmental or code constraints, evolves into a "proactive" one where QA engineers can control the application's behavior themselves.

To contribute to accelerating the entire development cycle while ensuring quality, why not consider leveraging Bucketeer as the first step toward realizing a more "offensive" QA strategy?