What hackathon apps look like

What kinds of problems do apps have?

Findings have various different severities. While most are chronic, a nontrivial number of apps have serious, severe, or even critical problems. The table below shows the number of findings of each kind as well as how many apps have at least one of them.

Grading an app by its single worst finding gives the stats below. Each one contains the ones under it, so almost 3 in 5 projects carry a significant problem and virtually every app overlooks some hygiene.

(A significant problem has a penalty more than 20, and an acute problem has a penalty of more than 40.)

• Hygiene99.9%1,624 apps

  E.g. missing headers, poor accessibility, orange Lighthouse score; the things people often skip.

• Significant58.9%957 apps

  Findings that are not just cosmetic, such as dead controls, broken links, missing rate limits, overly slow pages.

• Acute25.8%420 apps

  Severe issues that noticeably degrade the user experience or allow attacker access, such as crashes, unusable pages, or exposed backends.

• Exploitable2.9%47 apps

  Catastrophic vulnerabilities that an attacker can exploit today.

Winners ship more slop

Counterintuitively, winning apps have 11.8% higher median slop than the rest.

The same is true for Lighthouse:

As you can see, winning does not correlate with app cleanliness. In fact, the opposite tends to be true. Most hackathons employ human judging, which rewards ideas, features, presentation, and the demo over durability. Winning apps tend to ship more features, meaning more surfaces to misconfigure or get wrong, and human judges do not have time to judge quality consistently over hundreds of apps.

Fast != clean

Lighthouse performance barely predicts anything else. Measured against slop with the performance axis taken out, the correlation is -0.071 across 1,571 apps, which is close enough to zero to call the two independent. In other words, speed and durability don't have any relationship.

• -0.071Spearman correlation between Lighthouse performance and the rest of the slop (without the performance component).

Breakdown per hackathon

Across 61 hackathons out of 80 with 8 or more graded apps, median slop runs from 28.3 to 105.8, a 3.7x difference. Hover over a bar for the event in question.

Yet exploits are rare

Only 2.9% of apps carry something an attacker could use today. The largest single finding is an exposed backend, with 18 apps serving a Supabase or Firebase database that anyone could read, because row level security was not turned on. Of those, 7 returned records in bulk and 3 held personal data in its columns.

What didn't get graded

Sloptic attempted 2,685 apps and graded 1,625 of them, or 60.5%. Most of the rest were due to link rot (expired free tier), timeouts, a WAF challenge, or other reasons.

Also excluded are 82 streamlit apps since Sloptic is currently unable to properly separate what the teams built from these platforms.