TL;DR: AWS Device Farm warns you not to enter sensitive data. BrowserStack has been breached twice. Bank of America and AT&T built their own labs. Here’s why enterprises are abandoning shared cloud infrastructure for private device testing.
You’re testing a banking app. The test script enters a customer’s account number, simulates a login, processes a transaction. The test passes. You move on.
But here’s the question: Where did that data go?
On BrowserStack or AWS Device Farm, your test ran on a device that was used by another company an hour ago—and will be used by someone else an hour from now. The vendor promises they “clean” the device between sessions. But what does “clean” actually mean?
AWS answers that question with uncomfortable honesty.
AWS Device Farm: “Data May Persist”
In their official documentation, AWS Device Farm includes this warning:
“It is possible for data to persist between sessions in some cases, especially if you make use of the device system outside the context of your app.”
— AWS Device Farm Data Protection
And their FAQ is even more direct:
“We recommend that you avoid providing or entering sensitive information such as account info (e.g., Google Account, Apple ID), personal information, and other security-sensitive details during your automated test and remote access sessions.”
— AWS Device Farm FAQ
Let that sink in. AWS is telling you not to use real data on their shared devices.
Why? Because the cleanup process between sessions is best-effort, not guaranteed. Mobile devices are complex systems with multiple storage layers—app data, system caches, keychain entries, clipboard history, browser storage. A script can clear some of these, but not all of them, not reliably, not every time.
For a startup testing a todo app, this is fine. For a bank testing a mobile payment flow with real account numbers, this is a compliance violation waiting to happen.
The Breach History: Theory Meets Reality
If the theoretical risk of data persistence isn’t convincing enough, the actual breach history of cloud testing providers should be.
BrowserStack 2014: The ShellShock Incident
In November 2014, BrowserStack was breached through an unpatched server vulnerable to ShellShock. The attacker:
- Accessed AWS credentials stored on the server
- Created a new IAM user and generated access keys
- Mounted a backup disk and copied production data
- Exfiltrated email addresses, hashed passwords, and last tested URLs
- Used AWS SES credentials to send a fake “BrowserStack is shutting down” email to 5,000 customers
BrowserStack’s admission in their post-mortem was telling:
“All our servers, running or not, should have been patched with the latest security upgrades and updates.”
— SecurityWeek: BrowserStack Says Hackers Exploited ShellShock Vulnerability
BrowserStack 2025: AWS Credentials in Logs
More recently, a security researcher discovered that BrowserStack was printing AWS access keys and secrets directly into logs stored in S3 buckets. The impact:
- 300,000+ customer data entries exposed
- Private test logs became publicly accessible
- S3 buckets lacked presigned URL requirements
- Attackers could enumerate and retrieve sensitive files
- PUT permissions potentially allowed data modification
“This critical vulnerability at BrowserStack vividly demonstrates how a combination of insecure logging practices and misconfigured S3 access controls can lead to a massive data breach.”
— Medium: Critical Vulnerability at BrowserStack
Two breaches, a decade apart. Different attack vectors, same fundamental problem: when you test on shared infrastructure, you’re trusting someone else’s security practices with your data.
Why Bank of America Built Their Own Lab
Enterprise security teams understand this risk. That’s why the largest financial institutions don’t use public device clouds—they build private labs.
Bank of America partnered with Kobiton for an on-premises device lab deployment:
“Bank of America is able to scale access to devices to 30+ lines of business & 1,000+ users while maintaining security compliance requirements.”
— Bank of America QA Manager, Kobiton
AT&T chose a hybrid model combining on-premises and private cloud:
“AT&T uses a combination of Kobiton’s on-premises and private-cloud solution in order to meet their unique needs for each of their teams.”
— AT&T Quality Assurance Manager, Kobiton
TIAA, a Fortune 100 financial services company, “doubled their commitment” to private infrastructure after experiencing Kobiton’s on-premises support.
Nike uses private device labs for performance testing, reducing test time from 160 hours to 12 hours.
These aren’t companies that can’t afford BrowserStack’s enterprise tier. They’re companies that can’t afford the risk of shared infrastructure.
The Cost of Getting It Wrong
For enterprises in regulated industries, the stakes go beyond reputational damage.
HIPAA (Healthcare)
Healthcare organizations handling Protected Health Information (PHI) face penalties up to $50,000 per violation, capped at $1.5 million annually. If your mobile app test includes patient data and that data persists on a shared device, you’ve just created a compliance incident.
A 2018 survey found that 84% of FDA-approved medical health applications had security threats that might expose sensitive data. The risk isn’t theoretical.
PCI-DSS (Payment Cards)
If your app processes payment card data, PCI-DSS compliance requires strict control over where that data is processed and stored. Non-compliance can result in fines of $5,000 to $100,000 per month from payment card brands.
Testing on shared infrastructure with real card numbers—even test card numbers that follow real patterns—introduces compliance questions that security auditors will flag.
Barclays: A Cautionary Tale
In early 2025, Barclays experienced a major mobile banking outage. The estimated compensation costs ranged from £5 million to £7.5 million. While not directly related to testing infrastructure, it illustrates the financial stakes: a single mobile failure can cost millions.
Organizations at this scale can’t leave testing reliability to shared queues and best-effort cleanup scripts.
HeadSpin: Air-Gapped for Banking
HeadSpin, a device testing platform, explicitly markets air-gapped deployment for financial services:
“Isolated Testing Environment: HeadSpin offers an air-gapped deployment model, ensuring complete isolation of the testing environment. This safeguards sensitive testing data, which is crucial for applications like banking apps.”
— HeadSpin: Air-Gapped Testing Labs
Their on-premises PBox appliance enables:
- Secure, internal device labs within the bank’s network
- Testing on real smartphones, tablets, and payment terminals
- Temperature-controlled, RF-enabled environments
- Zero data leaving the organization
HeadSpin offers two deployment options for BFSI (Banking, Financial Services, and Insurance):
- Cloud-Connected (VPC): Controller hosted in HeadSpin’s cloud, but devices remain private
- Air-Gapped: Controller deployed on physical servers within customer premises, no internet connectivity
The air-gapped option is specifically designed for “banks and insurers with the highest levels of security and compliance requirements—where data must never leave the organization.”
Kobiton: On-Premises or Nothing
Kobiton positions their on-premises offering as a hard requirement for regulated industries:
“Install Kobiton fully on-premises, even in air-gapped environments. Your entire solution, safely behind your firewall.”
— Kobiton Device Lab Management
Their deployment options span the spectrum:
- Public cloud: For teams without compliance constraints
- Private cloud: Dedicated devices in Kobiton’s data center
- On-premises: Full installation behind the customer’s firewall
- Hybrid: Combination of local and cloud devices with unified management
The fact that Kobiton leads with on-premises in their enterprise messaging tells you what their largest customers are asking for.
BrowserStack’s Private Devices: The Upsell That Proves the Point
Here’s the most telling signal of all: in March 2025, BrowserStack launched Private Devices—a premium offering that addresses the exact concerns we’ve been discussing.
“With Private Devices, we’re addressing the critical needs of enterprise customers who require both advanced security and testing flexibility.”
— Nakul Aggarwal, CTO of BrowserStack
The Private Devices offering includes:
- Guaranteed availability (no queuing)
- Persistent device setups (apps, accounts, settings retained)
- Fixed SIM cards and phone numbers
- Isolated device access for compliance
- UDID access for consistent testing
Who is this for? BrowserStack is explicit:
“Ideal for BFSI, healthcare, telecom, and other industries with strict compliance needs.”
Read between the lines: BrowserStack is acknowledging that their public cloud isn’t suitable for regulated industries. They’re selling the solution to their own problem.
The Hybrid Model: Where the Industry Is Heading
The enterprise market is settling on a hybrid device infrastructure model:
| Tier | Use Case | Infrastructure |
|---|---|---|
| Breadth | “Does the app crash on a 5-year-old Android phone?” | Public cloud (BrowserStack, AWS Device Farm) |
| Velocity | CI/CD regression testing on top 10 devices | Private cloud (dedicated devices, zero queue) |
| Depth | Performance profiling, complex authentication flows | Managed private (admin access, SIM cards) |
| Sovereign | Banking apps, healthcare PHI, air-gapped requirements | On-premises (your devices, your network) |
The binary choice between “build your own” and “use the cloud” has dissolved. But for enterprises in regulated industries, the question isn’t whether to use private infrastructure—it’s how much.
DeviceLab: Private Testing Without the Infrastructure Burden
DeviceLab lets you build a private device lab using devices you already own—without the operational overhead of Kobiton’s on-premises software or HeadSpin’s PBox appliances.
How it works:
- deviceNode runs on a machine where devices are physically connected (your office, data center, or home)
- testNode runs on your laptop or CI runner
- P2P WebRTC connects them directly—no data touches our servers
# On the machine with devices (deviceNode)
curl -fsSL https://app.devicelab.dev/device-node/KEY | sh
# On your laptop or CI runner (testNode)
curl -fsSL https://app.devicelab.dev/test-node/KEY | sh -s -- --framework appium --app ./YourApp.apk
What this means for security:
- Your devices stay in your network — We never touch them
- No data persistence risk — You control the device lifecycle
- No shared infrastructure — Single-tenant by design
- Air-gapped compatible — Devices don’t need internet access to run tests
- Compliance-friendly — All test data stays within your perimeter
For organizations that need the security of an internal lab but don’t want to build and maintain the infrastructure, DeviceLab provides the control without the complexity.
The Bottom Line
AWS tells you not to enter sensitive data. BrowserStack has been breached twice. The largest banks and telecoms have built their own labs. And BrowserStack itself now sells “Private Devices” for enterprises who need security.
The pattern is clear: public device clouds are fine for compatibility testing, but they’re not designed for sensitive workloads.
If you’re testing apps that handle customer data, financial transactions, or healthcare information, you have three options:
- Build an internal lab (expensive, complex)
- Pay for vendor-managed private infrastructure (HeadSpin, Kobiton, BrowserStack Private Devices)
- Use your own devices with DeviceLab (private by design, minimal infrastructure)
Platforms like Perfecto offer private cloud options, though at significant cost premiums.
The one option you don’t have is pretending the risk doesn’t exist.
Try DeviceLab
Connect your existing devices. Keep your data private. No shared infrastructure.