Discover why Google’s massive 96% failure rate could be your key to success.

 

In a 2011 U.S. Senate testimony, former Google CEO Eric Schmidt disclosed that Google had conducted more than 24,000 experiments during the prior year to determine which algorithm changes would improve the quality of its search results. A staggering 96.1% of their tests failed, resulting in a mere 3.9% success rate.

Yet testing is precisely what gave Google an advantage over their competitors: former Yahoo executives admitted that “Google just outran us. We didn’t have that experimentation engine.”

Research from Microsoft, which has been replicated at other companies, supports Yahoo’s intuition. They found that teams and companies that run many tests perform better than those that run only a few.

It makes sense that greater testing results in greater success because companies that test more have honed their skills in maintaining agility and thinking creatively during periods of uncertainty. As we all enter the Age of Uncertainty, this matters — a lot.

Simply put, companies that run more tests demonstrate true business resilience. It’s only natural that an increased testing pace will lead to more competitive breakthroughs.

The Value of Testing

Google understands why testing matters. In an interview published in 2016, the company’s Captain of Moonshots, Astro Teller, popularized the term “dynamic stability.” He explained that while maintaining stability in the past required standing still, our modern world of rapid technological change and global disruption means that only agility can create stability now.

In essence, he pointed out that the modern tech landscape is changing so quickly that it feels like we’re each riding a bicycle through it at top speed. To stay stable on our bikes, we must always keep moving because our bikes will fall over if we stop moving or slow down too much. To succeed in the modern world, Teller said, “We’re all going to have to learn that bicycle trick.” And we’ll have to keep trying new ideas to maintain our dynamic stability.

As Google demonstrated, that “bicycle trick” should include constant business testing. Why “constant?” Because it’s nearly impossible to predict which business ideas will succeed. As Jeff Bezos stated, “If you already know it’s going to work, it’s not an experiment.”

And certainly, don’t assume an idea will work. Look at the data you have, identify trends and pose questions, and, if you’re unable to come to a conclusion, test your ideas.

How to Test Business Ideas the Smart Way

Ready to encourage a culture of testing at your organization? Don’t just leap in headfirst. Avoid chaos with a well-structured testing process that reduces uncertainty. The experts at Accounting Business Solutions and Harvard Business Review recommend that you:

• Test in cross-departmental teams and assign each team a goal. This will help teams understand that business changes have a ripple effect through many departments. Also, ensure each goal and test aligns with your company’s strategy, mission, and values.
• Follow proven testing steps. First, develop a variant to test that differs from your SOPs (Standard Operating Procedures). Next, decide who you will test it on. (Existing customers with at least $1K in purchases? Tardy employees? Customers who have called your service team more than 3 times?) Finally, determine which metrics you will use to measure your outcomes.
• Test the smallest possible variable. Most of Google’s tests were “imperceptible to users” and focused on tiny changes. When you keep tests small, you can better control your variables and design better future tests that further reduce uncertainty.
• Repeat the test a few times. Google has the luxury of testing their changes on potentially millions of users. Your sample size might be as small as 10 people. To separate your test’s signal from the noise, run the test a few times before taking your results seriously.
• Organize and store your tests by goal. To prevent duplicate tests, store your tests and results in a secure, shared repository. Organizing your tests by goal instead of by business unit or feature makes them easily findable. This will discourage your Service team from ignoring a relevant test simply because it was run in the Sales department (for example).
• Treat your failures with equal value as your successes. Multiple failures are likely to answer more questions than a single big win can. If you encourage leaders to find the “why” behind each result, you’ll be better prepared to avoid repeating failures and to replicate successes across other areas of your organization.

Use a Modern ERP to Drive Testing Success

Today’s modern ERPs, such as Acumatica Cloud ERP, are engineered from the ground up to deliver cutting-edge tech that meets your most pressing business concerns – but different ERPs have different capabilities. Only some ERPs natively support business testing initiatives; many require purchasing expensive add-ons or bolt-on solutions before your team can begin testing their ideas.

To keep your testing costs low and your testing rate high, look for an ERP that puts you in the driver’s seat. It should let you:

• Start with your Your ERP should house all your data in one place, including data from Salesforce, industry-specific applications like CAD, and eCommerce data. It should empower your team to quickly and easily build custom reports and generic inquiries, slice and dice the data, and test hypotheses – without calling IT for help or hiring a developer to code costly new reports.
• Make data access democratic. Building a truly data-driven culture requires giving every employee secure, role-appropriate access to your company’s data so that they can back up every idea with the facts. Ensure your ERP allows unlimited user access instead of per-user licensing fees so that you can give every employee the access they need.
• Set up real-time tests. Once you begin testing, you must monitor your tests at all times. To do this, ensure your ERP can automatically convert raw data into real-time insights available in custom reports, generic inquiries, detailed customizable widgets, and customizable dashboards. ERP solutions with advanced AI chat assistants for real-time data can also help your team quickly make sense of complex results.

Learn How Real-Time Data Helps Ignite Success at Your Company

• Leverage AI to search and explain tests. We mentioned earlier that you should create a repository of all your tests, grouped by goal rather than by business unit or feature. A modern ERP can help you go one step further if you make sure it has a secure AI chat assistant that can find and compare prior test results quickly and put prior test data in context. This helps further democratize your team’s access to relevant, role-appropriate data because the AI can translate test findings for your less technologically minded staff members.

Are You Getting the Most Out of Your Data?

According to a report from Forrester Research, companies experienced in using a data-driven approach for insights are 8 times more likely to say they grew by 20% than companies less experienced in leveraging their data.

What they say is true: “Data is the new oil” (or gold).

In fact, that analogy is quite apt. Like oil or gold, data requires you to extract it before you can gain value from it.

Oil extraction takes drills. Gold extraction takes pickaxes and hydraulics. Data extraction takes a modern, cloud-based ERP like Acumatica that provides your entire company with the tools to find, isolate, extract, and refine your data through constant business testing. In this way, you can maintain resilience and thrive amid our world’s current and future uncertainties.

Savvy leaders know that this is just one key use of an ERP system:

Discover All 7 Key Benefits That Modern ERP Delivers

 

Accounting Business Solutions is a respected accounting and business management software solution provider. We provide experienced sales, consulting, implementation, training and support services for small to medium-sized companies located in south central and southeast Texas. Our portfolio of solutions includes cloud based, hosted and on-premise options.

 

Sources

EXPERIMENTATION WORKS: The Surprising Power of Business Experiments, Stefan H. Thomke, 2020. https://www.amazon.com/Experimentation-Works-Surprising-Business-Experiments/dp/163369710X

 “Want Your Company to Get Better at Experimentation?” Harvard Business Review, January 2025. https://hbr.org/2025/01/want-your-company-to-get-better-at-experimentation

Thank You for Being Late: An Optimist’s Guide to Thriving in the Age of Accelerations, Thomas Friedman, 2016. https://en.wikipedia.org/wiki/Thank_You_for_Being_Late

“The State of the Insights-Driven Business, 2022,” Forrester Research, 2022. https://web.archive.org/web/20250130181706/https://www.forrester.com/blogs/the-state-of-the-insights-driven-business-2022/