Why I Published Rules I Had Never Run
For the past few months I have been writing email detection rules in MQL, the query language Sublime Security's platform uses, and collecting them in gists. A few went upstream as pull requests. Most sat in a folder waiting on a review that had not come, and the pile kept growing.
At some point keeping them private stopped making sense. If I believed some of them were useful, the honest thing was to put them where anyone could read them, run them, and tell me they were wrong. So I published them as a rule feed: a public repository that my own Sublime instance subscribes to, and that anyone else can subscribe to as well.
Setting that up took an afternoon. What it exposed took longer to accept.
Not one of the twenty-five rules had ever run against a real message. I had written them, read them back, satisfied myself the logic held, and submitted several for review. None of that is testing. It is proofreading, and the two are easy to confuse when you are working alone.
What this note is. A record of what happened when I ran twenty-five of my own detection rules against thirty days of my own mail for the first time. Two of them fired. Between them they produced twenty-seven matches, and not one of those matches was an attack.
Setting It Up, Start to Finish
The whole stack is free and runs on a laptop, so this part is reproducible by anyone who wants to test their own rules. Four steps.
| Step | What It Does | Reference |
|---|---|---|
| Publish the rules | Public repo, one YAML file per rule | My feed |
| Run the platform | Self-hosted Sublime in Docker | sublime-platform |
| Connect a mailbox | IMAP source pointed at real mail | Message sources |
| Add the feed | Instance subscribes to the repo | Platform docs |
The repository layout is the only thing that has to be exact. A folder named detection-rules at the root, one .yml file per rule. Each file needs a name, a type, a severity, a source block holding the MQL, and a UUID. That is the entire contract. I matched the layout used by two existing community feeds rather than inventing one.
The platform installs with a single command and runs as a set of Docker containers, so Docker Desktop has to be running first. The dashboard comes up on localhost:3000 and the API on localhost:8000. Nothing leaves the machine.
Connecting Gmail needs an app password, which in turn needs 2-Step Verification switched on. Google documents the process under signing in with app passwords. One detail matters for the results: Sublime reads the \All folder, and that folder excludes Spam and Trash. Everything Google had already classified as spam was gone before Sublime saw anything.
If you want to read MQL before writing any, Sublime's own introduction to the language is the clearest starting point, and the production ruleset is public, which makes it the best reference there is.
Limits worth knowing before you start. The Docker deployment is documented as being for small deployments and testing, capped at one hundred active mailboxes. Retroactive hunting reaches thirty days and no further, whatever date range you request, so a longer archive cannot be queried. Several API endpoints returned errors on this build, and I did not establish whether that reflects the self-hosted deployment or my own configuration. None of it blocked the test, but all of it shaped what the test could measure.
Six of Twenty-Five Could Never Have Run
Installing a feed is also the first moment anything checks the rules. Six of the twenty-five failed validation.
| What I Wrote | Why It Could Not Run |
|---|---|
| strings.split(...) | No such function exists in MQL |
| subject | The field is subject.subject |
| .href_url.domain.fqdn | No such accessor exists |
| Escaped quotes in a single-quoted string | Single-quoted strings cannot contain them |
| A pattern broken across two lines | Strings cannot span lines |
Six of twenty-five is twenty-four percent. None of these are style problems. A rule containing a function that does not exist does not fire less often than it should; it does not run at all.
I had been carrying these for three months. Several had been submitted for review. Nothing surfaced any of it, because nothing had ever tried to execute them.
A rule that has never been parsed is not a rule. It is an intention, and the difference between the two is invisible until something tries to run it. That alone is an argument for publishing a feed even if nobody subscribes to it, because installation is the first real review.
Two Rules Fired, Neither Caught Anything
With all twenty-five fixed and active, I ran each one across the corpus. Rules normally only evaluate mail as it arrives, which means mine had never seen the backfilled messages. Retroactive hunting runs a rule over mail already stored, and that is the only reason this test was possible at all.
Twenty-five rules, four thousand and thirty-three inbound messages, thirty days, one query per rule.
I read all twenty-seven and labelled each one. Here is what they were.
| What the Match Actually Was | Count | Verdict |
|---|---|---|
| Security and developer newsletters | 11 | False positive |
| AI industry newsletters | 9 | False positive |
| Daily news digests | 2 | False positive |
| Healthcare and civic notices | 2 | False positive |
| Genuine recruiting correspondence | 2 | False positive |
| Automated billing receipt | 1 | False positive |
Twenty-six of the twenty-seven came from one rule, written to catch fake technical assessments sent to developers. Twenty-four of those twenty-six were newsletters. Security newsletters, developer newsletters, AI newsletters: the ones I subscribe to and read every week.
In hindsight it is obvious. A rule hunting for developer recruitment language alongside links to coding and assessment platforms describes a technology newsletter almost perfectly. I had not thought about newsletters once while writing it.
The twenty-seventh came from a rule looking for PDF attachments whose metadata suggests machine generation. It matched an antivirus subscription renewal receipt, because automated billing systems generate their invoices with exactly the kind of tooling that rule was watching for.
The twenty-three silent rules prove nothing either way. They cover wire fraud at property closings, tax authority impersonation, remote monitoring tool abuse and cloud storage payload delivery. A pre-filtered personal mailbox contains none of that, and Google had already removed anything it considered spam before the platform ever saw it. Zero firings here is a statement about the corpus, not evidence the rules work.
The Exclusion Everyone Uses Did Nothing
Twenty-six false positives from one rule is a fixable problem, and the production ruleset has established ways of fixing it. I tested three of them against the identical corpus, changing nothing else, so the numbers are directly comparable.
| Exclusion Added | Matches Left | Production Rules Using It |
|---|---|---|
| None, as originally written | 26 | — |
| not profile.by_sender().solicited | 26 | 177 |
| prevalence in ("new", "outlier") | 3 | 70 |
| No List-Unsubscribe header present | 2 | 10 |
The most widely used exclusion in the entire ruleset had no effect whatsoever. One hundred and seventy-seven of the 1,361 production rules carry a sender-solicitation check, more than any other false-positive control, and on my corpus it removed exactly zero matches.
The reason is specific rather than mysterious. Solicitation is derived from whether the organisation has corresponded with the sender before. My instance holds thirty days of backfilled mail and classifies none of it as outbound, so there is no correspondence history to derive anything from.
The check is not broken and it is not badly written. It assumes an organisational context that my mailbox does not have, and when that assumption fails it evaluates quietly to nothing rather than raising an error.
An exclusion that silently evaluates to nothing is indistinguishable from one that is working. Had I copied that line into all twenty-five rules, which is what reading the production ruleset would naturally lead anyone to do, every one of them would have carried a control that did nothing at all, and the rules would still have looked correct on the page.
What the two survivors tell you
The header check cut twenty-six matches to two. Both survivors came from a recruiter at an established company, sent from that company's own domain, and the correspondence was entirely legitimate.
That is the limit of the rule, and no exclusion reaches it. A genuine technical assessment and a fabricated one are the same message at the level my rule was looking: recruitment language, a link, an invitation to run code.
Narrowing at the front instead of filtering at the back
The closest equivalent in the production ruleset does not attempt to solve that problem. It opens by requiring the sender's root domain to be one of thirteen specific abused hosting services. Everything after that only evaluates on mail that has already passed through a very narrow door.
A newsletter never reaches its content logic, so it never needs an exclusion for newsletters. Mine opens wide on content and would need exclusion after exclusion stacked behind it to compensate. That is not a stylistic preference between two valid approaches. It is the difference between a rule that is narrow by construction and one that is broad and then apologised for.
What I would do next
Three things, in order.
Test against a corpus that can actually falsify the other twenty-three. A personal mailbox cannot, and no amount of care in the measurement changes that. An organisational mailbox or a public phishing corpus could.
Rewrite the recruitment rule to narrow at the front rather than filter at the back, requiring the assessment link to sit on a domain unrelated to the sender's, and dropping the broad content matching that caught the newsletters.
And add author attribution to every rule in the feed, so that anything useful in it is traceable to a person rather than to a repository name.
I set out to publish a feed so that other people could use what I had written. What it produced first was an inventory of what was wrong with my own work: six rules that could not run, one rule that fired twenty-six times without being right once, and a false-positive control I would have copied straight from the production ruleset without ever noticing it was doing nothing. All of that cost one afternoon and a laptop, and none of it was available to me from reading my own rules back to myself.
All figures are from the author's own testing, conducted in October 2026 against a self-hosted Sublime Security platform running locally in Docker, connected by IMAP to the author's personal Gmail account. The corpus is 4,033 inbound messages across 3,162 message groups, read from the platform's hunt interface on 29 October 2026 over a rolling thirty-day window; because the mailbox continues to receive mail, this figure drifts by a few messages between runs. Per-rule counts were obtained through the platform's hunt API with the window pinned to 29 September 2026 through 29 October 2026, one query per rule, and are reported as total matching messages rather than message groups. The hunt API returns a maximum of fifty message groups per response, so per-rule counts are exact only because every rule returned fewer than fifty; the corpus total was therefore taken from the interface rather than the API. Retroactive hunting is capped at thirty days on this build regardless of the range requested. Rule validation failures were reported by the platform at feed installation; all six have since been corrected. The twenty-seven matches were individually read and labelled by the author. Exclusion comparisons were run against the identical pinned window with no other change to each rule. Counts of production rules using each exclusion were taken from a clone of the public sublime-security/sublime-rules repository containing 1,361 rule files, counted as files containing not profile.by_sender().solicited, files containing a profile.by_sender().prevalence check, and files referencing a List-Unsubscribe header, giving 177, 70 and 10 respectively. The thirteen-domain comparison rule is link_job_recruitment_lure_with_suspicious_hosting.yml in that same repository. Sublime reads the \All IMAP folder, which excludes Spam and Trash, so messages already classified as spam by Google were never presented to the platform. This note describes one ruleset tested against one mailbox at one point in time and makes no claim about how these rules would perform in an organisational environment. This field note represents independent research produced for educational purposes.