Evidence before automation
Treat a score as one piece of evidence. The important questions are what it measures, how it was evaluated, and when a person should review it.
“Ad domain” is an operational label, not a complete description of a business or a website. A domain can serve mixed purposes or change over time. Define the unit being classified: a hostname, a request, a script, or a page element.
A useful label policy describes inclusion criteria and ambiguous cases. Keep advertising classification separate from judgments about malware, trustworthiness, or legal compliance. Those questions need different evidence and should not inherit the same score.
Potential inputs might include observed request context, known delivery patterns, and a reviewer’s notes. Assess whether each input is current, relevant, and permissible to use. A suggestive domain name alone is a weak basis for an enforcement decision.
Store the model or method version, the observation time, and the explanation available to a reviewer. Avoid collecting more user information than the investigation needs. An explanation should help someone challenge a classification, not merely repeat the predicted label.
A cautious policy may send uncertain cases to review rather than automatically adding them to a list. Select that boundary using a representative evaluation set and the costs of mistakes. A false positive affecting sign-in is different from a missed decorative banner.
Test the model’s score meaning before calling it a probability. If a value is only a ranking signal, say so. The AI score fieldnote explains calibration, abstention, and evaluation choices without suggesting a universal cutoff.
Every automated addition needs a way to be questioned. Record the reason for a reversal, retest neighboring rules when appropriate, and consider when old evidence should expire. Keeping a record forever is not the same as keeping it accurate.
Connect these review states to an explicit API data contract. If you evaluate an outside supplier, ask how they expose corrections through their premium-list maintenance process.
No. A score depends on the task definition, evidence, model, and evaluation setting. It should not be treated as a guarantee of safe or complete filtering.
A policy can reserve ambiguous or high-impact decisions for review. Whether automation is appropriate depends on measured errors and the consequences of mistakes.
From Ad Block List Lab
An AI score becomes useful when its target, evidence, and uncertainty are visible. Learn how to evaluate domain classifications and choose thresholds that reflect the cost of mistakes.
6 min readA useful API contract carries more than domains. It explains format, provenance, freshness, and what happens when an update fails.
Explore this topicAI can be involved in creative production, targeting, or optimization. None of those labels automatically supplies a reliable browser filtering rule.
Explore this topic