What do apps miss?
When Sloptic graded 1,702 apps in 80 hackathons, it found that...
Nothing is clean
The median is 39.2, and a quarter scored above 60.5. No app scored 0 and the cleanest scored 1. In other words, there is slop in every app.
More stats
- 45.9average slop
- 28.5standard deviation
- 197worst app
What was the slop like?
Different slop have different severities. While most are chronic and indicative of hygiene, quite a few apps have serious or critical problems. The table below shows the number of instances of slop of each kind and how many apps have what:
| penalty | band | findings | share | apps with at least one | |
|---|---|---|---|---|---|
| 1-10 | minor | 7,360 | 79.7% | 1,698 (99.8%) | |
| 11-20 | moderate | 883 | 9.6% | 751 (44.1%) | |
| 21-30 | serious | 433 | 4.7% | 397 (23.3%) | |
| 31-40 | severe | 249 | 2.7% | 233 (13.7%) | |
| 41+ | critical | 309 | 3.3% | 281 (16.5%) | |
| all | 9,234 | 100% | (these overlap) | ||
When evaluated on its single worst slop instance, this is what apps had:
- Hygiene100%1,702 apps
E.g. missing headers, poor accessibility, orange Lighthouse score; the things people often skip.
- Significant45.9%781 apps
Findings that are not just cosmetic, such as dead controls, broken links, missing rate limits, overly slow pages.
- Acute16.5%281 apps
Severe issues that noticeably degrade the user experience or allow attacker access, such as crashes, unusable pages, or exposed backends.
- Exploitable1.2%21 apps
Catastrophic vulnerabilities that an attacker can exploit today.
What do teams miss?
The most common kinds of slop are missing security headers, accessibility issues, and slow performance. Most are invisible to the teams building the apps.
About 4 in 5 of those have low contrast text.
The median score is 80.
The eight most common findings, by share of graded apps. Why is my score bad when my app is fast?
Are AI-built apps sloppier?
Not overall. Lovable apps carry a median slop of 41.4 against 38.7 for hand built apps. Bolt has only 11 apps which is too few to compare.
The difference that matters is in the backend and passive grades can't check them. Full grades show how often AI-built apps leave theirs open.
Are winners cleaner?
Winning apps carry a median slop of 41.8 against 38.6 for everyone else. The corpus report tests this gap on full grades only.
Winning apps also score lower on Lighthouse:
Are faster apps cleaner?
Not exactly. When we measured performance (via Lighthouse) against the slop without the performance axis, the correlation is -0.098 across the1,651 apps, which is close enough to zero to call the two independent.
- -0.098Spearman correlation between Lighthouse performance and the rest of the slop
Breakdown per hackathon
Across 62 hackathons out of 80 with 8 or more graded apps, median slop runs from 25.6 to 56.3, a 2.2x difference. Hover over a bar for the event in question.
How many are exploitable?
Only 1.2% of apps had an exploitable vulnerability a passive grade can see. Note that a passive grade does not send attacks over and only sees what a visitor sees, such as secrets shipped in the app's own code in a bundle.
What didn't get graded
Sloptic attempted 2,685 apps and graded 1,702 of them, or 63.4%. Most of the rest were due to link rot (expired free tier), timeouts, a WAF/bot challenge, or other reasons.
Also excluded are 82 streamlit apps, 2 unknown apps, 1 github-pages apps, 1 firebase apps since Sloptic is currently unable to properly separate what the teams built from these platforms.