How to determine if a mental health app is evidence-based
Content
- The 10-minute check
- 1. Vendor navigation/footer: Research, Evidence, Science, Outcomes, Clinical validation
- 2. In-app/help center: Methodology, Clinical team, Safety, Disclosures
- 3. App store listing: “study,” “trial,” “validated,” “clinical”
- A plain-language evidence ladder (quick reference)
- Red flags (fast scan)
- Example walkthrough (more concrete)
- 4. Capture the claim. This is Level 2 because it promises symptom improvement on a timeline.
- 5. Find proof. Look for a named study or registry entry tied to the app.
- 6. Quality-check. The minimum you should expect to see in a summary is the study design (ideally controlled), participant count, an 8-week timeframe, and a recognized anxiety measure such as GAD-7, plus completion/attrition.
- 7. Version match and safeguards. If the app now markets AI coaching, check whether the evaluation covered that feature. Confirm crisis guidance, harm reporting, and clear privacy sharing/retention.
- Disclaimer (educational, not medical advice) and crisis guidance
- FAQ
A mental health app is evidence-based when there’s public, checkable evidence that matches the exact product (and current version/feature set) and the specific claim being marketed. It is not enough that the app mentions CBT, lists expert advisors, or has good reviews. You can usually verify this in about 10 minutes by capturing the app’s strongest claim, finding product-specific proof, then checking basic study quality plus safety and privacy.
Many clinical reviewers use established models such as the APA app evaluation model, the NHS DTAC criteria, and ORCHA-style review domains as a practical baseline for what “good enough” looks like in the real world.
The 10-minute check
1) Capture the strongest claim
Open the app store listing and the vendor homepage. Copy the single strongest claim word for word. Look for phrases like clinically proven, reduces, treats, prevents, equivalent, diagnosis, replaces therapy, in X weeks.
Classify the claim:
- Level 1: Wellbeing support (stress tools, journaling, meditation, psychoeducation)
- Level 2: Symptom improvement (“reduces anxiety,” “improves sleep,” “improves mood”)
- Level 3: Treatment- or diagnosis-adjacent (“treats depression,” “diagnoses,” “prevents suicide,” “clinically equivalent,” “replaces therapy”)
A reasonable default is to hold the app to the bar set by its strongest claim, not its gentlest wording elsewhere.
2) Find product-specific evidence
If the app is evidence-led, the proof is usually easy to find on the vendor site.
Check, in order:
1. Vendor navigation/footer: Research, Evidence, Science, Outcomes, Clinical validation
2. In-app/help center: Methodology, Clinical team, Safety, Disclosures
3. App store listing: “study,” “trial,” “validated,” “clinical”
If that fails, search for public artifacts:
- "[APP NAME]" study
- "[APP NAME]" trial
- "[APP NAME]" randomized or controlled
- "[APP NAME]" protocol or registered trial
- "[APP NAME]" ClinicalTrials.gov (or your country’s registry)
- site:[vendor-domain] evidence or site:[vendor-domain] study
What you want is a named evaluation with enough detail to judge fit:
- Study type (usability, pre/post, controlled, randomized)
- Sample size
- Outcomes measured (preferably validated measures)
- Timeframe (and any follow-up)
- Population
- Funding/conflicts
Common substitutes that do not answer the claim:
- General CBT or mindfulness citations unrelated to the app
- “Clinically proven” with no study name, design, or outcomes
- Case studies that only report engagement, satisfaction, or testimonials
- “Backed by science” without product-specific evaluation
3) Quality-check the study signals
You are not peer-reviewing. You are checking whether the evidence is identifiable, relevant, and credible enough for the claim.
Use these questions:
Compared to what? For Level 2 and Level 3 claims, look for a comparison group (waitlist, usual care, another intervention). Pre/post-only results are suggestive, not definitive.
How many people? Look for a clear participant number and how many were included in the analysis. If the vendor will not state it, treat that as a transparency gap.
What did they measure? Prefer validated measures over custom in-app scores. For anxiety and depression claims, look for familiar instruments such as GAD-7 and PHQ-9. If the only outcome is a proprietary “wellbeing score,” you cannot tell what it means without validation.
How long, and did it last? Short timeframes can be fine for habit-building claims. They are weaker support for big symptom or treatment claims. Follow-up strengthens credibility.
How many finished? High dropout can inflate results if only the most engaged users are counted.
Who benefits if results look good? Look for funding disclosures, author affiliations, and (when applicable) a registered protocol.
4) Confirm the evidence matches today’s product, then check safety and privacy
For apps, evidence only helps if it matches what users get now.
Product/version match
- Does the studied intervention match today’s user journey (program length, modules, level of human support, target population)?
- Do the studies cover the features being marketed now? If the headline value is AI coaching/chat but the study tested static content, the evidence may not support the current claim.
If there is a mismatch, treat the evidence as partial and ask for updated validation or post-change evaluation.
Safety and governance As claims intensify, these become a minimum bar rather than a nice-to-have. Look for concrete, checkable artifacts:
- Named clinical oversight (who is responsible, not just “experts involved”)
- Clear crisis guidance and boundaries
- A way to report harm, deterioration, or unsafe outputs
- Release notes or a change log that shows updates are tracked
Privacy basics You do not need legal training. You need clarity on:
- What data is collected
- How it is used
- Whether it is shared (including advertising)
- Retention and deletion
If you cannot quickly understand sharing and retention, treat the app as higher risk, especially for workplace use.
A plain-language evidence ladder (quick reference)
- Usability/feasibility: people can use it and understand it. Fits Level 1 claims.
- Pre/post outcomes: users improved over time, but causality is unclear.
- Controlled study or RCT: stronger support for Level 2 and Level 3 claims.
- Real-world outcomes plus monitoring: shows performance outside a study setting, with ongoing reporting and safety monitoring.
Red flags (fast scan)
- “Clinically proven” with no named study and no design/outcome details
- Citations only to general CBT/mindfulness research, not the app
- Outcomes rely only on testimonials, star ratings, or engagement metrics
- No sample size, no timeframe, or no description of what was measured
- Evidence clearly relates to an older or different version than what is sold today
- No crisis guidance despite discussing severe symptoms
- No way to report harm, deterioration, or unsafe AI outputs
- Unclear third-party sharing, advertising use, retention, or deletion
Example walkthrough (more concrete)
Scenario: The app claims “Reduce anxiety in 8 weeks.”
4. Capture the claim. This is Level 2 because it promises symptom improvement on a timeline.
5. Find proof. Look for a named study or registry entry tied to the app.
6. Quality-check. The minimum you should expect to see in a summary is the study design (ideally controlled), participant count, an 8-week timeframe, and a recognized anxiety measure such as GAD-7, plus completion/attrition.
7. Version match and safeguards. If the app now markets AI coaching, check whether the evaluation covered that feature. Confirm crisis guidance, harm reporting, and clear privacy sharing/retention.
If you find only general anxiety education citations and no product-specific outcomes, treat “reduces anxiety” as marketing until the vendor can point to a named evaluation that matches the claim.
Disclaimer (educational, not medical advice) and crisis guidance
This article is educational and is not medical advice, diagnosis, or treatment guidance. If you or someone else may be at immediate risk of self-harm or harm to others, do not rely on an app. Contact local emergency services or your local crisis line right away.
FAQ
1) How do I know if a mental health app is evidence-based?
It is evidence-based if it provides public, verifiable evidence that matches the specific app experience and the specific marketing claim, and it aligns with the app’s current features.
2) Is “CBT-based” the same as evidence-based?
No. It describes an approach. It does not show that this app produces the outcomes it claims.
3) What’s the minimum evidence I should expect for an app that claims to reduce anxiety or depression?
Look for outcomes over a defined timeframe using recognized measures (for example, GAD-7 or PHQ-9), with transparent reporting on sample size and completion. Controlled evidence is stronger, especially for specific claims like “clinically proven” or “in X weeks.”
4) Do I need journal access to verify an app’s evidence?
No. Abstracts, trial registries, and vendor evidence pages can provide enough detail to judge basic fit and credibility.
5) What are the biggest red flags that an app isn’t evidence-based?
No named study, only general mental health citations, missing basics (sample size, timeframe, measures), evidence that does not match the current version, weak crisis guidance, or unclear privacy practices.
6) How do I check whether the evidence matches the current app version?
Compare the study description to what the app markets today: key features, program length, and whether support is human-led, self-serve, or AI-enabled. If major features changed, treat the evidence as partial and request updated validation.
7) Are app-store ratings and testimonials evidence?
No. They reflect user sentiment, not clinical outcomes. They can help with usability signals, but they do not validate symptom or treatment claims.
8) What privacy details matter most for mental health apps?
What data is collected, whether it is shared with third parties or used for advertising, how long it is retained, and how deletion works. Unclear sharing or retention should raise your risk assessment.
9) What level of evidence should I expect for treatment-like or diagnosis-adjacent claims?
Stronger evidence and stronger governance. Look for controlled, product-specific evaluation plus clear clinical oversight, escalation pathways, and harm reporting, especially if AI features are involved.