There is a ton of noise about AI today. A quick search on Google for “Artificial Intelligence” returned 82M results and almost 4M news stories. Bloomberg reported in June that mentions of AI in earnings transcripts has risen from less than 60 in 2016 to over 250 in just Q1 of this year! In a 2017 PricewaterhouseCoopers report 72% of executives “believe AI will be the business advantage of the future.”
It’s clear there is a will. So what’s in the way?
There are a few specific challenges that prevent companies from enjoying the massive positive impact that AI, and specifically Machine Learning (ML), will have on their business.
- The hype and reporting about advancements in AI paint a wholly inaccurate picture about the state of AI, and sets an unrealistically high-bar for what AI projects look like.
- Too many companies think in terms of small numbers of big-wins, instead of maximizing the potential of AI across numerous small wins.
- Machine Learning technologies and the AI market are still very immature – making hiring costly and difficult, and ML projects are time-consuming, complex and risky to execute.
We’re talking and thinking about AI all wrong.
Much of the current chatter about AI is focused on ambitious projects like autonomous cars, robots, or Natural Language Processing (NLP) to understand and respond to human speech. Some even hint at the more SciFi “General AI” – the concept of AI that successfully performs any intellectual tasks a human can.
Certainly these breakthroughs are noteworthy and exciting. These massive research projects don’t (yet) apply generally to the existing day to day needs of businesses. They end up as massive moonshots that require the best of the best of the best and take years for the technology to be ready and usable. Google began research on the self-driving car in 2009. It’s likely to be another 10 years before we see widespread adoption of the technology.
Part of what makes these advancements exciting, is that they represent the combination of a number of different breakthroughs in machine learning. Autonomous driving requires computer vision, real-time predictive analytics, scores of sensors, routing algorithms, etc.
These projects are critical to the field of AI. They drive significant advancements in the core science and technologies that make AI work. But, they distract from the myriad ways AI and ML can improve business right now for millions of companies. They set an unrealistically high-bar for what an AI project looks like, and the cost and resources required to get started. Spending millions of dollars and years of time on a project that may not actually work is a really difficult call to make.
AI Research is getting all the press today, but it’s the application of AI that will transform industries. There are hundreds of areas where companies can achieve transformational impact with machine learning right now:
- Anomaly detection – Fraud, market shifts, yield management
- Supply chain optimization
- Inventory/Manufacturing optimization
- Customer/audience analytics – segmentation, predictive analytics, funnel optimization
- Route management – deliver predictions
- Sales/marketing effectiveness
- Customer, Employee retention
For these projects, companies typically have the historical data to build models from, understand the direct correlation between improvements in accuracy and the top-line, and most importantly – they know how to put these into action immediately!
We think way too small.
I’ve talked to numerous data scientists in startups and fortune 1000 companies. Across the board, they talk about their pipeline of projects in small numbers. They think in terms of the 2 – 3 models they are going to build this year. I can’t tell you how often I’ve heard “How many models does a company really need?”, or “Once we built a model, we’re done, right?”.
It reminds me of the famous bad tech predictions:
“I think there is a world market for maybe five computers.”
Thomas Watson, president of IBM, 1943
“There is no reason anyone would want a computer in their home.”
Ken Olsen, founder of Digital Equipment Corporation, 1977
“No one will need more than 640k for a personal computer…”
Bill Gates, 1981
What if you could:
- Reduce the cost and length of your sales cycle by focusing your sales team’s effort on the leads most likely to convert?
- Maximize revenue by setting your pricing to the optimum point in the elasticity curve based on customer buying behavior and market dynamics?
- Differentiate your solution by delivering orders of magnitude better accuracy than alternatives?
- Minimize churn and dramatically improve satisfaction by proactively engaging customers based on changes in usage and behavior?
- Beat your competitors to market by quarters or even years through reduced development and production cycles?
Would you do only one?
One of the more damaging impacts of moonshot thinking is that too many companies think they need a “Big Win” to get value out of AI. These projects then require big budgets, long runways, and recruiting expensive and hard to find talent. As a result they either fail to start experimenting, or launch their own moonshot – which often fails.
In November of last year, Gartner predicted that by 2019, 90% of enterprises will have invested in C-Level data science leaders, yet half of these investments will fail due to lack of focus on core-business improvement.
We’ve experienced this cycle with each transformational technology. PCs, ERP, Internet computing, eCommerce, SaaS, and Cloud-computing, all fundamentally transformed how we do business today. Each of these innovations unlocked brand new opportunities for those that embraced them. Transformation came through dramatic improvements in the type of things businesses were already doing. It wasn’t a moonshot that brought the value of these innovations to companies. It was the cumulative impact of all the small wins.
Looking for the AI moonshot distracts a business from the multitude of opportunities to put wins on the scoreboard. Even worse, the high level of investment required in terms of hiring, cost, time and risk, scare off a majority of companies from even starting.
It’s really hard!
The basic workflow of an AI/ML project is straightforward. The product or business lead works with their data-scientist(s) to frame a key business problem as an equation that a computer can optimize.
The data scientist then collects and prepares data to be trained by machines. Model development is an iterative process of feature engineering (data transforms), choosing algorithms, optimizing parameters and options for those algorithms, training models, then repeating the whole process until they are satisfied with the results. Finally, the finished model needs to be deployed into production.
While straightforward, it’s by no means simple. The model building phase alone typically takes 2-3 months!
NetFlix famously paid $1M dollars for a better model for movie recommendations. The winner of the contest did indeed produce a model that produced significantly higher accuracy. However, the model was never put into production. The size and complexity of the model developed was so high, that NetFlix did not have the computing power to make use of it.
It’s not enough to maximize accuracy. Outside of academia and competitions, data scientists have to produce high-quality models that can be put into production.
The skills and experience required to execute this process are significant. Today, a data scientist has to be a domain expert, software engineer, mathematician, statistician, machine learning expert, database & big-data developer and a business analyst. Finding someone with this collection of skills is incredibly difficult – leading to a significant supply and demand imbalance in the industry.
A study by McKinsey projects that “by 2018, the U.S. alone may face a 50 percent to 60 percent gap between supply and requisite demand of deep analytic talent.” The cost of hiring a data scientist can be $200k or more.
When you add it all up, it’s no wonder companies are struggling to take advantage of the tremendous opportunities AI offers.
- AI is poorly understood and expectations are high – often leading to magical thinking.
- Executives and leaders all believe AI will provide competitive advantage and board rooms are demanding something be done.
- Moonshot thinking sets the bar high for scope and cost and prevents many from even getting started.
- Those that do start find It’s expensive and difficult to build a data science team.
- Market immaturity makes It time consuming and risky to execute projects.
- Companies are thinking too big in terms of scope, and too small in terms of the number of projects – resulting in failed starts, poor delivery and difficult to justify investments.
There is hope…
The impact of machine learning for business is tremendous. We are witnessing the rise of the next major transformation of industry. Robert Zampetti of Ernst and Young recently said, “This whole arena that we are broadly calling artificial intelligence is actually a sea change, it’s not a bolt-on; it’s more profound—it’s the culmination of the information revolution; just like assembly lines became a paradigm in the industrial era, artificial intelligence will become a paradigm in the information era”.
It takes a new approach. Chasing the big-win AI project magnifies the cost and complexity of hiring and execution, minimizes the number of projects companies expect to deliver, and maximizes the risk of disillusionment. Companies that focus on a number of “small win” projects will see disproportionate returns on their AI investment. Each win increases corporate confidence in these projects, and the cumulative impact will vastly exceed the sum of its parts.
The only way to do this is to dramatically reduce the cost, complexity, time and resources required to build new models. Today, it can take 2-3 months to deliver one new model ready for production. That doesn’t even include the time it takes to collect and prepare the data! This reality is what leads to the limited expectations of only 2-3 models per year. New technical approaches are needed that allow data scientists of all skill levels to produce elite quality models in days instead of months.