How does Sloptic find slop?
Short answer: by checking your app for problems that no app should ever have. Long answer:
What counts as slop?
Sloptic defines "slop" as issues that no app should ever have no matter what. Some apps may exhibit behavior that is wrong for this app but may be correct for another app, or a different use case. For example, a table any visitor can read can be right for a marketplace's product listing but wrong for a user database. Allowing duplicates may be correct for logs but incorrect for payment transactions.
But certain behaviors, like exposed secrets, injections, unhandled server errors, or pathologically slow apps, are wrong for any app you come across. No app on earth exists where such behaviors are "correct". Thus these are considered slop and Sloptic docks apps for them.
You might disagree with this definition of slop, but Sloptic operates this way so it can compare two apps against each other, which depends on an issue being an issue in every app. Without this criterion, we run into the oracle problem, which states that there is no way to determine correct behavior without knowing what "correct" means.
How Sloptic scores
- 0the cleanest score
deduction only
Nothing is earned for passing but you get penalized for failing, similar to how quality and security failures work in the real world. A lower score is better.
- likelihood × impactexpected harm
risk priced
Sloptic penalizes slop based on expected harm, or how often it hurts * how bad it is (i.e. the classical risk formula).
- 2 to 98points per check
varies based on check
Some checks are tiered, meaning they have a penalty range and escalate whenever worse issues are found in that category, while others apply a fixed penalty. Lighthouse performance is penalized to the tune of 90 - N (with N being the Lighthouse score).
- each repeat of a kind
damped
Repeats count less after the first instance, so your app is not penalized repeatedly for the same issue (double jeopardy).
- no capfour subtotals summed
unbounded
Security, quality, accessibility and performance each report their own subtotal and the four sum to the score. There are no limits on how high the score can be.
Where do scores come from?
A penalty is not a matter of taste. Every number traces to a published authority, and where a finding lands inside that authority's range is set by what the check saw. You can find the full rationale here. Sloptic derives its scores from four authorities:
- security holesCVSS, the industry scale for how bad a vulnerability is, reconciled against the Bugcrowd rating taxonomy.
- quality failuresISO/IEC 25010, the software quality standard, crossed with Nielsen's severity scale for how much an issue hurts a user.
- performanceGoogle Lighthouse, where Sloptic penalizes the distance below Lighthouse's standard for "good".
- accessibilityaxe-core, priced by the impact rating it assigns each barrier against WCAG.
Additionally, Sloptic's security checks map to a requirement in the OWASP Application Security Verification Standard (ASVS) 5.0. 49 of the 63 security checks map to a requirement while 9 more are close to one. See here for the full mapping.
Two kinds of checks for slop
Passive checks read what your app already shows your visitors.
Active checks go looking for holes by sending attacks and writes to your app. Doing that to a site you don't own is considered unauthorized testing, so they only run when ownership is proven.