OpenSTF/DeviceFarmer Alternative: Modern

If you’ve been running OpenSTF (Open Smartphone Test Farm) for your mobile testing infrastructure, you’ve probably hit a wall. Your Android 14 and 15 devices aren’t working. Updates stopped years ago. And you’re stuck wondering: what now?

You’re not alone. Thousands of QA teams and developers built their device labs on OpenSTF, and now they’re scrambling for alternatives that actually work with modern devices.

In this guide, I’ll explain exactly what happened to OpenSTF, why the “official” fork isn’t cutting it, and why we built DeviceLab as a modern replacement that solves the problems OpenSTF never could.

What Happened to OpenSTF?

OpenSTF (Smartphone Test Farm) was originally developed at CyberAgent in Japan to manage their growing collection of 160+ test devices. It became the go-to open-source solution for teams wanting to build their own device labs instead of paying cloud testing prices.

But in July 2020, the core team announced they were abandoning the project:

“This project along with other ones in OpenSTF organisation is provided as is for community, without active development.”

The last official OpenSTF release was v3.4.1, which only supports up to Android 9. If you’re trying to test on Android 10, 11, 12, 13, 14, or 15 devices—you know, the versions your actual users have—you’re out of luck.

The DeviceFarmer Fork: Not the Answer

Development supposedly moved to a community fork called DeviceFarmer. But here’s the reality:

• 156+ open issues sitting on GitHub
• Volunteer-maintained with no funding or dedicated team
• Slow Android version support—Android 14/15 support is incomplete
• Still no iOS support—same limitation as original OpenSTF
• Same ancient tech stack—RethinkDB, Node.js 8.x, ZeroMQ

The project openly acknowledges: “Development is still largely funded by individual team members and their unpaid free time, leading to slow progress.”

One developer on Hacker News captured the sentiment perfectly: “They dropped support on it and moved to another product (DeviceFarmer) which never seems to have materialised at all. We just kept running OpenSTF until it stopped working.”

Why OpenSTF’s Architecture Was Always Flawed

Beyond the abandonment, OpenSTF had fundamental architectural problems that made it unsuitable for modern testing needs:

1. The 12-Container Problem

To run OpenSTF in production, you needed to orchestrate:

• RethinkDB (database)
• nginx (reverse proxy)
• stf-app (web interface)
• stf-auth (authentication)
• stf-processor (device processing)
• stf-triproxy-app (message routing)
• stf-triproxy-dev (device routing)
• stf-reaper (cleanup)
• stf-storage-plugin-apk
• stf-storage-plugin-image
• stf-storage-temp
• stf-websocket
• stf-api
• stf-provider (one per host machine)
• adbd (ADB daemon)

Setting this up took days. Maintaining it was a part-time job. The documentation even warns: “stf local was developed for development purpose. Users are not supposed to use it in production.”

2. No Data Privacy Architecture

The OpenSTF team acknowledged this in their own docs:

“We have made certain assumptions about the trustworthiness of our users. As such, there is little to no security or encryption between the different processes.”

And:

“Devices do not get completely reset between uses, potentially leaving accounts logged in or exposing other sensitive data.”

For teams handling sensitive apps—finance, healthcare, enterprise—this was always a dealbreaker. (Learn more about why enterprises build private device labs.)

3. iOS Support: Never Happening

iOS support exists through a separate project (stf_ios_support), but it’s barely functional:

• Only 1 iOS device per Mac machine (video streaming limitation)
• 1 FPS hardcoded framerate
• Clicking is slow due to Apple API limitations
• Devices frequently get stuck in “preparing” state
• WebDriverAgent has memory leaks requiring restarts every 4 hours

From the stf_ios_support FAQ: “Q: Why can’t I use/provide multiple iOS devices simultaneously from a MacOS machine? A: This is a technical limitation within the video streaming mechanism.”

4. Test Automation Gap

Here’s what surprises many teams: OpenSTF is not a test automation platform.

OpenSTF provides remote device access—you can view the screen, tap, type, and install apps. But it doesn’t run your Appium tests, Espresso suites, XCUITest cases, or Maestro flows.

To run automated tests, you need to set up a separate Appium server, build integration logic, and handle device acquisition/release yourself. This is why solutions like Zebrunner MCloud exist—but now you’re managing even more infrastructure.

The Cloud Alternative: Wrong Trade-offs

Many teams fleeing OpenSTF looked at cloud device farms. But the real issue isn’t just cost—it’s data privacy.

When you use cloud device farms, your app binaries, test data, user credentials, and potentially sensitive business logic all flow through third-party servers. They promise they won’t look at your data—but they technically could. For regulated industries or companies with strict security requirements, this is a non-starter.

Why We Built DeviceLab: A Modern OpenSTF Replacement

After running RobusTest (our enterprise device lab solution) for 12+ years serving companies like Disney+Hotstar, Airtel, Swiggy, and Jio, we saw the gap in the market clearly:

• Small teams have the same security concerns as enterprises but can’t afford complex on-premise solutions
• Cloud device farms require trusting third parties with sensitive data (see our BrowserStack alternative guide)
• OpenSTF is dead and DeviceFarmer isn’t a real solution

So we built DeviceLab with a radically different architecture.

The Zero-Trust Architecture Difference

Here’s what makes DeviceLab fundamentally different:

OpenSTF Architecture:

Your Device → OpenSTF Server → Your Browser
           (all data flows through central server)

Cloud Device Farm Architecture:

Your Test Data → Cloud Provider Servers → Test Results
              (they promise not to look)

DeviceLab Architecture:

Your Device ↔ WebRTC P2P ↔ Your Browser
        (data flows directly, never touches our servers)

With DeviceLab, your test data, app binaries, and device interactions flow directly between your devices and your browser via WebRTC peer-to-peer connections. Our servers handle authentication and connection coordination—but your actual data never touches them.

This isn’t “we promise not to look at your data.” This is “we architecturally cannot access your data.”

What DeviceLab Solves That OpenSTF Couldn’t

Full Test Framework Support

Unlike OpenSTF which only provides remote access, DeviceLab is a complete testing platform with built-in support for all major frameworks:

Run your existing test suites without building custom integration layers. Your Appium scripts, Espresso tests, XCUITest cases, and Maestro flows work out of the box.

DeviceLab is also the only device lab platform with native Maestro support on iOS physical devices—a capability no other solution offers.

DeviceLab vs. DeviceFarmer (OpenSTF Fork)

Verdict: DeviceFarmer is “free” but costs you in DevOps time, limited device support, and no iOS. DeviceLab pays for itself in reduced maintenance burden alone.

How DeviceLab Works

Step 1: Connect Your Devices

Run one command on the machine with your devices. No RethinkDB, no ZeroMQ, no complex infrastructure.

# That's it. Devices auto-connect.
curl -fsSL https://app.devicelab.dev/device-node/KEY | sh

Your Android and iOS devices automatically appear in your DeviceLab dashboard.

Step 2: Start Testing

Access your devices from any browser, anywhere. Run manual tests or execute automated test suites:

# Run tests on your devices
curl -fsSL https://app.devicelab.dev/test-node/KEY | sh -s -- \
  --framework appium \
  --app ./app.apk \
  --tests ./tests/

All device interactions flow peer-to-peer via WebRTC. Your test data stays yours.

Migrating from OpenSTF to DeviceLab

If you’re currently running OpenSTF or DeviceFarmer, migration is straightforward:

What You’ll Keep

• Your physical devices
• Your Appium test suites (fully compatible)
• Your Espresso and XCUITest projects
• Your CI/CD integrations (we have APIs)
• Your USB hubs and host machines

What You’ll Gain

• iOS device support
• Android 14/15 support (and future versions)
• All test frameworks built-in (Appium, Espresso, XCUITest, Maestro)
• Zero-trust data privacy
• No more RethinkDB maintenance
• No more minicap/minitouch debugging

What You’ll Leave Behind

• RethinkDB administration
• Node.js 8.x security vulnerabilities
• The anxiety of running abandoned software
• Android-only limitations
• 12+ Docker containers
• Manual Appium server setup

Migration Timeline

Most teams complete migration in minutes:

1. Run device-node command—devices auto-connect (5 minutes)
2. Run your first test (5 minutes)

That’s it. Compare that to the days or weeks it took to set up OpenSTF originally.

When OpenSTF Still Makes Sense

To be fair, OpenSTF/DeviceFarmer isn’t wrong for everyone:

OpenSTF might fit if you:

• Have dedicated DevOps resources with STF expertise
• Only need Android devices (older versions)
• Don’t need integrated test automation
• Have unlimited time for setup and maintenance
• Run in an isolated network with no security requirements

DeviceLab is better if you:

• Want to focus on testing, not infrastructure
• Need iOS and Android support
• Run automated tests with Appium, Espresso, XCUITest, or Maestro
• Value your DevOps team’s time
• Need enterprise-grade security

Frequently Asked Questions

Is OpenSTF still maintained?

No. The original OpenSTF project was abandoned in July 2020. The last official release (v3.4.1) only supports Android 9. Development moved to a community fork called DeviceFarmer, which is volunteer-maintained with limited resources and slow to support new Android versions.

Can I use my existing devices with DeviceLab?

Yes. DeviceLab works with any Android or iOS device you already own. Connect them to any Mac, Windows, or Linux machine running the DeviceLab agent.

What testing frameworks does DeviceLab support?

DeviceLab has built-in support for Appium, Espresso, XCUITest, and Maestro. Your existing test suites work without modification—no separate server setup required.

Does DeviceLab support iOS?

Yes. Unlike OpenSTF/DeviceFarmer which never properly supported iOS, DeviceLab works with both iOS and Android devices natively. You can run XCUITest and Maestro on iOS physical devices.

How is data privacy handled?

DeviceLab uses WebRTC peer-to-peer connections. Your device screen, inputs, and test data flow directly between your device and your browser—never through our servers. This is architectural privacy, not policy privacy.

How does DeviceLab handle new Android versions?

DeviceLab maintains compatibility with new Android releases as they launch. Unlike community-maintained projects, there’s no waiting period for volunteer contributors to update native components.

Can I migrate gradually from OpenSTF?

Absolutely. You can run DeviceLab alongside your existing OpenSTF setup during evaluation. Many teams migrate device-by-device over a few weeks.

The Bottom Line

OpenSTF was revolutionary for its time. It proved that teams could build their own device labs instead of paying cloud testing premiums. But it’s 2025, and OpenSTF is dead.

The DeviceFarmer fork is on life support. Cloud alternatives require trusting third parties with your data.

DeviceLab is what OpenSTF should have evolved into:

• Modern architecture (WebRTC P2P, not 2015-era message queues)
• Zero-trust privacy (your data never touches our servers)
• iOS and Android support
• All frameworks built-in (Appium, Espresso, XCUITest, Maestro)
• Actual maintenance and support

If you’re still running OpenSTF or DeviceFarmer, it’s time to upgrade.

DeviceLab is built by the team behind RobusTest, trusted by Disney+Hotstar, Airtel, Swiggy, Jio, and other enterprises for 12+ years.