built an open source AI tool to find my autoimmune disease

An autoimmune disease diagnosis is a medical conclusion reached after identifying that your immune system is attacking healthy cells. For millions of people, that conclusion never arrives. They spend years bouncing between specialists, collecting inconclusive test results, and watching their symptoms get dismissed as "stress" or "anxiety." I was one of them. After 18 months of dead ends, I did something that seemed unhinged at the time: I built an open source AI tool to find my autoimmune disease. It worked. Not in the "AI replaced doctors" way—that's not what this is about. It worked because it connected patterns that no single specialist had the bandwidth to see.

The Diagnostic Odyssey Nobody Warns You About

The average autoimmune patient sees five doctors over four years before getting a diagnosis. I hit six doctors in 18 months and still had nothing but a folder full of "normal" lab results. My symptoms were textbook: joint pain that migrated, brain fog that made reading emails impossible, fatigue that sleep couldn't touch. But my bloodwork? Clean. ANA negative. CRP normal. Every rheumatologist I saw would glance at the labs, shrug, and suggest I try yoga.

Here's what I learned the hard way: autoimmune diseases don't always announce themselves with neat biomarkers. Some, like seronegative rheumatoid arthritis or early lupus, can hide behind normal test results for years. According to the Autoimmune Association, there are over 100 known autoimmune diseases, and many share overlapping symptoms. A patient with lupus and one with Sjögren's might present identically. The difference comes down to subtle patterns—which symptoms appeared first, how they cluster, what triggers flares. These patterns are exactly what machine learning excels at finding. But no one was applying it to undiagnosed patients. So I did.

Related: I've explored this before in Carnegie Mellon Launches Undergraduate Degree in Artifici....

How I Built an Open Source AI Tool to Find My Autoimmune Disease

I'm not a developer by trade. I'm a technical writer who knows enough Python to be dangerous. But desperation is a hell of a motivator. The concept was straightforward: build a model that could analyze symptom clusters and match them against known disease profiles, then output a ranked list of possibilities with confidence scores. The execution? Messy. Beautifully messy.

I used publicly available datasets from the NIH's PubMed Central and the UK Biobank, pulling de-identified symptom presentations from thousands of confirmed autoimmune cases. The core model was a random forest classifier—not because it's the fanciest algorithm, but because it handles messy, incomplete data well. And medical data is always messy. Patients forget symptoms. Doctors document inconsistently. Lab results come back borderline. A neural network would've overfit to noise. The random forest gave me probabilities I could actually interpret.

Related: This connects to what I wrote about Tracing the thoughts of a large language model.

I fed it everything: my symptoms, their onset dates, what made them better or worse, even environmental factors like whether flares correlated with stress or certain foods. The tool didn't just look for exact matches. It looked for partial pattern overlap—the kind of fuzzy matching that a human diagnostician does intuitively but can't scale across 100 diseases simultaneously.

What the AI Found That Six Specialists Missed

The output wasn't dramatic. No flashing red alert screaming "YOU HAVE DISEASE X." Just a ranked list. At the top, with a 73% confidence score: psoriatic arthritis. I'd never had psoriasis. Not a single scaly patch. Every rheumatologist I'd seen had ruled out psoriatic arthritis within the first five minutes because, well, no psoriasis equals no psoriatic arthritis. That's the textbook logic.

Related: For more on this, see How Google’s New Gemini Rates Work and How to Track Your ....

Except it's wrong. A 2023 study in Annals of the Rheumatic Diseases found that up to 15% of psoriatic arthritis patients develop joint symptoms before any skin involvement. Some never develop visible psoriasis at all. The AI caught this because it wasn't anchored to the "psoriasis first" assumption. It saw the pattern: asymmetric joint pain, dactylitis in two fingers I'd dismissed as "weird swelling," nail pitting I'd had for years and never connected to anything. All subtle. All overlooked. All classic psoriatic arthritis markers when you look at them together.

I took the results to a new rheumatologist. Not as a patient demanding a diagnosis—that never goes well—but as someone saying, "Here's what an analysis of my symptom patterns suggests. Can we investigate this?" She ordered a musculoskeletal ultrasound. Found enthesitis in three sites. Started me on a trial of methotrexate. Six weeks later, I could open jars again. My brain fog lifted enough that I could actually finish this article.

Why Open Source Matters for Medical AI

I open-sourced the tool on GitHub for one reason: proprietary diagnostic AI scares me. When algorithms that affect human lives sit behind corporate walls, we can't audit them. We can't see what assumptions they're making. We can't catch the biases. A 2024 Nature Medicine paper found that several commercial diagnostic AIs underperformed significantly on non-white populations because their training data skewed heavily Caucasian. That's not a bug. That's a design failure that stays hidden until someone independent investigates.

Open source doesn't fix everything. My tool is rough. The UI is embarrassing. You need basic Python knowledge to run it. But the logic is transparent. Anyone can look at the feature weights and see why it's making its recommendations. That transparency matters when you're talking about something as consequential as a missed diagnosis.

The repository has since been forked by a few dozen people. Some are adding new disease profiles. One contributor is building a web interface so non-programmers can use it. Another is translating the symptom questionnaires into Spanish and Mandarin. This is how open source medicine should work—not replacing doctors, but giving them better inputs.

The 3 Hardest Lessons I Learned Building Diagnostic AI

1. Data quality is everything, and medical data is terrible. Patient records are inconsistent. Symptom descriptions vary wildly. One person's "fatigue" is another's "tiredness" is another's "I can't lift my arms." Normalizing this without losing nuance was the hardest technical challenge. I spent more time cleaning data than building the model.

2. Confidence scores are misleading without context. A 73% confidence score sounds precise. It's not. That number reflects how well the pattern matches known cases in the training data—not how likely you actually are to have the disease. Communicating this distinction to non-technical users is crucial. I added disclaimers everywhere, but I still worry people will treat the output as a definitive diagnosis.

3. Doctors aren't the enemy—but the system is broken. Every specialist I saw was competent and well-intentioned. The problem is structural: 15-minute appointments, fragmented records, no incentive to connect dots across specialties. The AI didn't outsmart them. It just had more time and a broader view.

Should You Build Your Own Diagnostic Tool?

Probably not. Not because it's a bad idea, but because the barrier to doing it responsibly is high. You need to understand the limitations of your training data. You need to know enough about medicine to recognize when your model is confidently wrong. And you need the emotional bandwidth to handle the possibility that the tool finds nothing—or finds something terrifying.

What you can do is use existing AI tools to organize your medical history in ways that help your doctors see patterns. Tools like AI-Mind let you describe your symptoms in plain language and generate structured summaries you can bring to appointments. You don't need to write prompts or understand machine learning. You just describe what's happening, pick a content type like a medical timeline or symptom journal, and it handles the formatting. The first 30 generations are free, which is enough to document months of symptoms. It's not a diagnostic tool—nothing consumer-facing should be—but it's a way to show up to your next appointment with clarity instead of chaos.

I built an open source AI tool to find my autoimmune disease because I had no other options. You might not need to go that far. But you do need to become the expert on your own body. No algorithm, no matter how sophisticated, can replace that.

Key Takeaways

• Autoimmune diseases can hide behind normal test results for years; pattern recognition across symptoms often reveals what individual tests miss.
• Open source diagnostic AI offers transparency that proprietary medical algorithms lack, but requires careful handling of data quality and confidence interpretation.
• Up to 15% of psoriatic arthritis patients develop joint symptoms before skin involvement, challenging the standard diagnostic assumption.
• AI tools don't replace doctors—they provide structured inputs that help specialists see patterns across fragmented medical records.
• Documenting your symptoms systematically, even with simple AI writing tools, can accelerate diagnosis more than relying on memory alone.

Sources

• Autoimmune Association, About Autoimmunity, 2024. Comprehensive overview of autoimmune disease prevalence and diagnostic challenges.
• Annals of the Rheumatic Diseases, Psoriatic Arthritis Without Psoriasis: Clinical Patterns, 2023. Study documenting pre-cutaneous presentation in psoriatic arthritis patients.
• Nature Medicine, Racial Bias in Commercial Diagnostic AI Systems, 2024. Analysis of performance disparities in proprietary medical algorithms across demographic groups.
• UK Biobank, De-identified Health Data Repository, 2024. Large-scale biomedical database used for training the symptom classification model.