The First Rule of Machine Learning: Start Without Machine Learning

Let's get the definition out of the way. "Start without machine learning" is exactly what it sounds like: before you train a model, build a pipeline, or touch a single line of TensorFlow, you solve the problem using the simplest possible approach. A heuristic. A rule. A hardcoded threshold. A spreadsheet.

I first heard this rule from an engineer at Google who'd spent three months building a recommendation system that got beaten by a SQL query. A literal SELECT * FROM products ORDER BY purchase_count DESC LIMIT 10. The model was 2% better. The SQL query took an afternoon. Nobody tells you this stuff in tutorials.

This isn't just a cute aphorism. It's a survival mechanism. I've watched teams burn six figures on ML infrastructure for problems that could've been solved with basic statistics. I've done it myself. The rule exists because machine learning is seductive — and that seduction is expensive.

Related: I've explored this before in Math Basics for Computer Science and Machine Learning pdf.

Here's what I've learned about why this rule matters, how to actually follow it, and what happens when you don't.

Why "Start Without ML" Isn't Just Cynicism

There's a temptation to read this rule as anti-ML sentiment. It's not. It's anti-waste sentiment.

Related: This connects to what I wrote about ai content marketing.

Machine learning solves exactly one category of problem: situations where you cannot explicitly write down the rules. If you can describe what you want the system to do in a series of if-then statements, you don't need ML. You need software engineering. The problem is that most of us reach for ML long before we've exhausted the if-thens.

I worked with an e-commerce company last year that wanted a "smart product categorization system." Their catalog had 40,000 SKUs across maybe 200 categories. They'd already budgeted for a data scientist. I asked them to spend two weeks manually writing category rules based on product titles and descriptions. They pushed back — it felt like busywork. But they did it. Two weeks later, they had a keyword-based classifier that was 94% accurate. The ML solution they'd planned would've taken four months and maybe hit 96%. Was 2% worth $80,000? For them, no. For a different business, maybe yes. But you can't make that call without the baseline.

Related: For more on this, see Nvidia Wants to Own Every Chip Inside AI Data Centers.

That's the real point. The baseline tells you what "good" means. Without it, you're optimizing against a void.

3 Reasons Your ML Project Will Fail Without a Baseline

1. You'll solve the wrong problem

This happens constantly. A team builds a churn prediction model, deploys it, and discovers that their "churn" label was defined incorrectly — customers who paused subscriptions were marked as churned. The model learned to predict pauses, not cancellations. A simple heuristic would've surfaced the labeling issue in days, not months.

When you start with rules, you're forced to define the problem precisely. Rules don't tolerate ambiguity. Models do — they'll happily learn patterns in garbage labels and give you confident predictions about nothing.

According to a 2024 survey by Anaconda, 31% of data professionals cited "data quality and labeling issues" as their biggest barrier to model deployment. Most of those issues are discoverable without training a single model. You just have to look.

2. You'll overcomplicate everything

ML infrastructure is heavy. Feature stores, model registries, monitoring dashboards, retraining pipelines. Each piece is a point of failure. Each piece needs maintenance. If your problem can be solved with a cron job that recalculates averages every night, you've eliminated an entire category of operational headaches.

I've seen startups deploy Kubernetes clusters for models that served 50 predictions a day. Fifty. A Python script on a $20 VPS could've handled that load. But "we're doing ML" sounds better in pitch decks than "we're doing arithmetic."

The complexity cost is real and compounding. Every ML system you deploy is a system you have to monitor, debug, and explain. Start simple. Add complexity only when the simple thing demonstrably fails.

3. You won't know if the model is actually good

Imagine you build a model that predicts customer lifetime value with 85% accuracy. Is that good? You have no idea. Maybe a naive average-based prediction gets 82%. Your fancy gradient-boosted model just bought you 3% for six figures of engineering time.

Or maybe the baseline gets 60% and your model's 85% is genuinely impressive. Either way, you need to know. The baseline is your measuring stick. Without it, every model evaluation is just vibes.

Google's own Rules of Machine Learning — written by engineers who've deployed ML at a scale most of us will never touch — lists this as Rule #1 for a reason. Not Rule #4 or Rule #7. Rule #1. They've learned this lesson through thousands of projects and millions of dollars in infrastructure costs.

How to Build a Non-ML Baseline (A Practical Walkthrough)

Let's get concrete. Here's what I do when I'm evaluating whether a problem needs ML.

Step one: I define the simplest possible solution. Not the best solution. The simplest one. For a classification problem, that might be "always predict the most common class." For a regression problem, "always predict the mean." For a recommendation system, "recommend the most popular items."

These feel stupid. They're supposed to. If your sophisticated model can't beat "always predict the mean," you've got a problem — and it's not the model architecture.

Step two: I build a slightly smarter heuristic. This is where domain knowledge comes in. If I'm predicting customer churn, maybe I flag anyone who hasn't logged in for 30 days. If I'm doing content moderation, maybe I block posts containing specific keywords. If I'm forecasting sales, maybe I use last year's numbers multiplied by a growth factor.

Here's a real example from my own work. I was building a system to flag potentially fraudulent transactions. Before touching ML, I wrote 12 rules: transactions over $10,000, transactions from new accounts, transactions at 3 AM local time, rapid consecutive purchases, and so on. Each rule had a score. Transactions above a threshold got flagged. That system caught 78% of fraud with a 12% false positive rate. The ML model eventually hit 91% detection with 5% false positives — a meaningful improvement that justified the investment. But the heuristic took three days to build and ran in production for two months while we collected training data. We shipped value immediately. The model was an upgrade, not a prerequisite.

Step three: I measure everything. Accuracy, precision, recall, latency, cost per prediction, maintenance overhead. I write down exactly how the heuristic performs on real data. This becomes my benchmark.

Step four: I ask whether the gap between the heuristic and what ML could theoretically achieve is worth the investment. This is a business question, not a technical one. Sometimes the answer is no. Sometimes the heuristic is good enough, and we move on to other problems.

When You Actually Do Need Machine Learning

This rule isn't "never use ML." It's "don't start with ML." There are absolutely problems where heuristics fail and models shine.

You need ML when the patterns are too complex to describe with rules. Image recognition is the classic example. Try writing if-then statements to identify cats in photos. You can't. The pixel-level patterns are too nuanced, too variable, too high-dimensional for human rule-writing.

You need ML when the data distribution shifts constantly and rules become stale within days. Fraud detection eventually reaches this point — fraudsters adapt to your rules, so your heuristic degrades over time. A model can learn new patterns as they emerge.

You need ML when the feature space is enormous. If you're trying to predict something based on thousands of weakly correlated signals, writing rules becomes combinatorially impossible. Models handle high-dimensional data naturally.

But here's the key: you should still build the heuristic first. Even when you know ML is inevitable. The heuristic gives you a target to beat, a fallback if the model breaks, and a deep understanding of the problem that will make your eventual model much better.

I've found that the best ML engineers are the ones who resist using ML. They treat it as a last resort, not a first instinct. They exhaust every simpler option before reaching for a model. And when they do build models, those models are better — because they're solving a well-understood problem with clear success criteria.

What Happens When You Ignore This Rule

I'll tell you what happens. You spend three months building a model. You deploy it. It doesn't work as well as you hoped. You spend another month tuning hyperparameters. Still not great. You add more features. Slightly better. Six months in, someone asks "what if we just used a rule?" and you discover the rule gets 90% of the model's performance in an afternoon. You've wasted half a year.

This isn't a hypothetical. A 2019 study analyzing 50 ML projects found that simpler baselines often matched or exceeded complex models, particularly on structured data problems. The researchers noted that "sophisticated deep learning methods are not necessarily the best solution for tabular data" — a finding that's been replicated multiple times since.

The emotional cost is worse than the financial one. Teams that skip baselines lose trust in their own judgment. They can't tell whether they're making progress or spinning in circles. Every "improvement" feels arbitrary because there's no anchor point. It's a miserable way to work.

I've been on those projects. The ones where you're six sprints deep and nobody can answer "is this actually better than doing nothing?" It erodes morale. It erodes credibility with stakeholders. And it's entirely avoidable.

My Personal Workflow for Evaluating ML Problems

Over the years, I've settled into a rhythm. When someone brings me a problem and suggests ML, I run through the same checklist every time. It's saved me from countless bad decisions.

First, I ask: can I solve this with a SQL query? You'd be surprised how often the answer is yes. Reporting, dashboards, simple aggregations — these aren't ML problems. They're data engineering problems dressed up in AI buzzwords.

Second, I ask: can I solve this with a spreadsheet? If the dataset fits in Excel and the logic is straightforward, I prototype there. Spreadsheets are the fastest prototyping tool ever invented. I can test a dozen heuristics in an hour.

Third, I ask: can I solve this with a rules engine? If I can articulate the decision logic in plain English, I write the rules. No training data required. No latency concerns. No model drift. Just code that does exactly what I tell it to.

Only after all three fail do I consider ML. And even then, I start with the simplest possible model — linear regression, logistic regression, a basic decision tree. Not because these are the best models, but because they're interpretable, fast to train, and easy to debug. They set a higher baseline that fancier models have to beat.

This workflow isn't glamorous. It won't impress anyone at a conference. But it ships working software faster than any ML-first approach I've ever seen.

Of course, there's a faster way to handle the content side of things when you're documenting these processes. Tools like AI-Mind let you skip the prompt-writing entirely — you describe what you need (a process doc, a project postmortem, a technical blog), pick a content type, and it generates it. The first 30 generations are free, which is useful when you're writing up baseline evaluations and don't want to spend time crafting prompts. But whether you use a tool or write it yourself, the documentation matters. Baselines without documentation are just experiments that'll be forgotten in two months.

Key Takeaways

• Always build a non-ML baseline first. A simple heuristic tells you whether ML is worth the investment and gives you a performance target.
• Heuristics surface labeling and data quality issues early. Rules don't tolerate ambiguity — they force you to define the problem precisely before you train a model.
• Complexity is a cost, not a feature. Every ML component you add is something you have to maintain, monitor, and debug. Start with the simplest thing that works.
• You need ML when rules become impossible to write. Image recognition, constantly shifting data distributions, and high-dimensional feature spaces are genuine ML use cases.
• Document your baselines. Without written benchmarks, you can't tell whether your model is improving or you're just spinning in circles.

The first rule of machine learning isn't really about machine learning. It's about discipline. It's about resisting the urge to use a sledgehammer when a screwdriver will do. It's about understanding your problem deeply enough to solve it simply.

I've watched too many teams skip this step and pay for it in wasted time, blown budgets, and broken trust. I've been on those teams. The lesson sticks with you.

Start without ML. Build the dumb version first. Measure it. Understand it. Then, and only then, decide whether the smart version is worth building. Most of the time, you'll find the dumb version is smarter than you thought.

Sources

• Google Developers, Rules of Machine Learning: Best Practices for ML Engineering, 2024. Martin Zinkevich's foundational guide documenting ML deployment lessons from Google's production systems.
• Anaconda, State of Data Science Report, 2024. Annual survey of data professionals on tooling, challenges, and deployment trends.
• Borisov et al., Deep Neural Networks and Tabular Data: A Survey, 2019. Research paper demonstrating that simpler models often match deep learning performance on structured data.