The First Rule of Machine Learning: Start Without Machine Learning

“The first rule of machine learning: Start without machine learning.”

If you’ve spent any time in the ML trenches, you’ve probably heard this. It sounds like a zen koan. It’s actually a hard-won engineering truth that comes from watching too many teams pour months into a neural network only to get beaten by a basic if-else statement. I’ve seen it happen. I’ve also been the person who built the complex model first, convinced I was being clever. I wasn’t.

Here’s what this rule really means. Before you touch TensorFlow, before you spin up a GPU instance, before you even think about hyperparameter tuning, you need to build the simplest possible solution to your problem. A heuristic. A rules-based system. A dumb baseline that requires zero learning. And here’s the uncomfortable part: that dumb baseline often solves 80% of the problem. Sometimes more.

Where This Rule Actually Comes From

This isn’t some random productivity advice. The phrase was popularized by Martin Zinkevich, a research scientist at Google, in his 2016 paper “Rules of Machine Learning: Best Practices for ML Engineering.” The document is a goldmine of practical wisdom — 43 rules distilled from years of deploying ML systems at scale. And rule #1 is deliberately provocative.

Zinkevich’s point wasn’t that machine learning is bad. It was that ML introduces complexity, and complexity has a cost. Every model you deploy is something you have to monitor, retrain, debug, and explain. If a simpler system can do the job, you’re saving yourself a mountain of technical debt. According to a 2015 Google paper on hidden technical debt in ML systems, even a simple ML model creates massive maintenance overhead because of the entanglement between data, code, and model parameters. The paper is brutal. It basically argues that ML systems have all the normal code maintenance problems plus an entirely new layer of ML-specific problems that most teams aren’t prepared for.

So the rule isn’t anti-ML. It’s pro-sanity.

3 Reasons Your ML Project Will Fail If You Skip This Step

I’ve consulted on a handful of AI content projects where teams jumped straight to fine-tuning GPT models before they even understood their own editorial guidelines. It never ends well. Here’s why skipping the “no ML” phase is a recipe for pain.

1. You Don’t Actually Know What “Good” Looks Like

If you can’t define success without a model, you definitely can’t define it with one. Building a heuristic forces you to articulate your problem in concrete terms. What are the inputs? What are the outputs? What counts as a correct answer? These questions seem obvious until you try to answer them.

I worked on a content classification project a few years back where the team assumed they needed a deep learning model to categorize support tickets. Before building anything, I spent two days writing a simple keyword-matching script. It caught 73% of the tickets correctly. More importantly, it exposed the fact that the team hadn’t agreed on what the categories actually were. Half the “misclassifications” were just disagreements between team members. That’s a people problem, not a machine learning problem. No amount of training data fixes that.

2. Baselines Keep You Honest

Without a non-ML baseline, you have no way to know if your fancy model is actually adding value. I’ve seen teams celebrate a 92% accuracy rate on a sentiment classifier, only to realize later that a simple rule — “if the review contains the word ‘refund,’ it’s negative” — would have hit 88%. That four-percentage-point improvement cost three months of engineering time and a $15,000 cloud bill.

Was it worth it? Maybe. But you can’t make that call if you never built the baseline. The baseline is your negotiating position. It tells you what you’re buying with all that extra complexity.

3. Heuristics Are Easier to Debug and Explain

When a rules-based system makes a mistake, you can trace the logic. You open the code, look at the conditions, and figure out what triggered. When a neural network screws up, you get a probability score and a shrug. Explainability matters more than most engineers want to admit — especially if your output affects real people or business decisions. A 2023 survey by FICO found that 65% of organizations couldn’t explain how their AI models made specific decisions. That’s a liability waiting to happen.

How to Build a Heuristic Baseline (Without Overthinking It)

This is the part where I actually give you a workflow. I’ve used this approach across text classification, content generation, and recommendation systems. It works.

Step 1: Define the task in one sentence. Not three paragraphs. One sentence. “Given a customer email, assign it to the correct department.” If you can’t do this, stop. You’re not ready for ML.

Step 2: Write the dumbest possible solution. I mean it. If you’re classifying emails, start with keyword matching. If you’re recommending products, start with “show the most popular items.” If you’re generating content, start with a template with fill-in-the-blank slots. Don’t optimize. Don’t iterate. Just get something that runs end-to-end.

Step 3: Measure it. Pick one metric. Accuracy, precision, recall, F1 — doesn’t matter. Just pick one and calculate it. This is your baseline number. Write it down somewhere permanent. You’ll need it later when you’re trying to justify the GPU spend.

Step 4: Analyze the failures. This is where the real learning happens. Go through every error your heuristic made. Categorize them. Are they edge cases? Ambiguous inputs? Genuinely hard problems that require contextual understanding? The failure patterns tell you exactly where ML might actually help — and where it’s overkill.

Step 5: Only then consider ML. If the heuristic’s failures cluster around problems that clearly require pattern recognition or contextual understanding, you’ve found your ML use case. If the failures are just data quality issues or unclear requirements, ML won’t save you.

What This Looks Like in Content Generation

This rule applies to generative AI too, not just predictive ML. Before you start fine-tuning a language model on your brand’s content, build a template system. Seriously.

I’ve seen marketing teams spend weeks prompt engineering with tools like ChatGPT, trying to get the tone and structure just right, when they could have built a simple fill-in-the-blank template in an afternoon. The template forces you to define what matters: headline structure, paragraph flow, call-to-action placement, word count ranges. Once you have that, you can evaluate whether an AI-generated piece is actually better than the template — and by how much.

This is exactly why I’ve shifted toward tools that handle the prompt engineering for me. AI-Mind, for example, lets you skip the template-versus-prompt debate entirely. You describe the content you need, pick a format, and it generates the output without you writing a single prompt. The first 30 generations are free, so it’s a low-risk way to test whether AI content beats your manual baseline. But the point stands: you should still build that manual baseline first. Know what “good enough” looks like before you pay for “better.”

If you’re still wrestling with prompts directly, you might want to read my breakdown of why most ChatGPT prompts fail — it covers the same pattern of people overcomplicating things before they’ve nailed the basics.

When You Should Actually Ignore This Rule

I’d be lying if I said this rule applies everywhere. It doesn’t. There are genuine exceptions, and pretending otherwise is just dogma.

You should skip the heuristic when:

• The problem is fundamentally about perception. Image recognition, speech-to-text, natural language understanding — these are tasks where rules-based systems have historically been terrible. If you’re building a facial recognition system, don’t waste time on a Haar cascade unless you’re specifically trying to understand the history of the field. Start with a pre-trained model.
• You’re in a domain where heuristics are actively dangerous. Medical diagnosis, credit scoring, criminal justice — these areas have long histories of biased rules-based systems. ML can still be biased, but at least the conversation is happening. A heuristic that says “deny loans to zip codes with default rates above X%” is just redlining with extra steps.
• You have a pre-existing ML pipeline and you’re iterating. If you already have a trained model in production and you’re improving it, building a heuristic from scratch is a waste of time. The existing model is your baseline.

But for most business problems — classification, recommendation, content generation, forecasting — the rule holds. Start simple. Prove you need complexity before you pay for it.

The Real Cost of Skipping the Baseline

Let me put some numbers on this. A 2024 survey by Anaconda found that 40% of enterprise AI projects never make it to production. The top reason wasn’t bad models or lack of data. It was that the models didn’t solve a clearly defined business problem. That’s a direct consequence of skipping the heuristic phase. When you jump straight to ML, you’re solving a technical problem without verifying that it’s the right problem.

I’ve been there. I once spent three weeks fine-tuning a content scoring model for a client, only to discover that their editorial team already had an internal rubric that scored content with 90% consistency. The model was slightly more consistent — 93% — but the gain was marginal and the maintenance burden was enormous. We shelved the model and just digitized the rubric. Total time wasted: four weeks. Lesson learned.

This is also why I’m increasingly convinced that most content teams don’t need to learn prompt engineering at all. They need to define their standards, build templates, and then use tools that abstract away the prompting layer. I wrote more about this in my piece on zero-prompt AI content generators, but the core idea is the same: don’t add complexity until the simple version stops working.

Key Takeaways

• Build a heuristic before any ML model. A simple rules-based system gives you a baseline, exposes hidden problems, and keeps you honest about whether ML is actually adding value.
• Define success in concrete terms first. If you can’t measure “good” without a model, you can’t measure it with one. Pick one metric and calculate it on your heuristic.
• Analyze heuristic failures to find ML opportunities. The errors your simple system makes reveal exactly where machine learning might help — and where the problem is just bad data or unclear requirements.
• Heuristics are easier to explain and maintain. When a rules-based system fails, you can trace the logic. When a neural network fails, you get a probability score and a lot of questions.
• Most content teams don’t need prompt engineering. Define your standards, build templates, and use tools that handle prompting for you. Complexity should be earned, not assumed.

Here’s the thing about the first rule of machine learning. It’s not really about machine learning. It’s about discipline. It’s about resisting the urge to reach for the most exciting tool before you’ve done the boring work of understanding your problem. The teams that follow this rule ship faster, spend less, and build systems that actually solve real problems. The teams that skip it? They spend months tuning models that get beaten by a spreadsheet.

I know which one I’d rather be.

Sources

• Martin Zinkevich, Rules of Machine Learning: Best Practices for ML Engineering, 2016. Google’s definitive guide to practical ML deployment, containing 43 rules distilled from years of production experience.
• D. Sculley et al., Hidden Technical Debt in Machine Learning Systems, 2015. Seminal paper from Google researchers on the maintenance and complexity costs unique to ML systems.
• FICO, State of Responsible AI Report, 2023. Survey revealing that 65% of organizations struggle with AI model explainability.
• Anaconda, State of Data Science Report, 2024. Annual industry survey tracking AI adoption trends and barriers to production deployment.