Added new rule for detecting rating solicitation scams#4492
Added new rule for detecting rating solicitation scams#4492jacob-mazurkiewicz wants to merge 11 commits into
Conversation
Test Rules Sync - Action RequiredThis PR was not automatically synced to test-rules because the author is not a member of the To enable syncing, an organization member can comment Once triggered, the rules will be synced on the next scheduled run (every 10 minutes). |
IndiaAce
left a comment
There was a problem hiding this comment.
Hey Jacob thanks for your contribution!
I think this is a clever idea to provide coverage for these scams. Early telemetry looks okay, however I have a few suggestions (left below). Also, I'm curious to hear your thoughts on reinforcing the "scam" nature of these two examples you provided. I see some arrant false positives on some document sharing messages that are requests to view documents/proposals and leave feedback. They are no doubt malicious, but outside of the scope of this rule. I'm curious to see what the nlu classifier gives as a .topic for both of your samples? Perhaps the "giveaway" angle is something worth pursuing?
Keeping your .intent use for both current and previous threads, but adding another "and" stanza with something like
and any(ml.nlu_classifier(body.current_thread.text).topics, .name == "Advertising and Promotions" )
I also see some patterns (with some samples I found as well) where each of these seem to provide "instructions" to the user, notice your two samples contain that set of 4 instructions. That might be an angle worth pursuing.
Let me know your thoughts!
Co-authored-by: Luke Wescott <69780712+IndiaAce@users.noreply.github.com>
Co-authored-by: Luke Wescott <69780712+IndiaAce@users.noreply.github.com>
Co-authored-by: Luke Wescott <69780712+IndiaAce@users.noreply.github.com>
…nded topic set for nlu_classifier flags
Appreciate the feedback @IndiaAce! Good to know! Can I ask if there is a location for ground truth values that I also checked for my samples what the predicted topics were. The primary ones I got were: I am thinking of adding these topics to your "Advertising and Promotions" suggestion with a confidence != low condition. I implemented your suggestions and hardened some of the rules to cut out false positives and overlap into other rules. It looks like my sample is still flagged with these improvements as well. Let me know what you think! |
|
Hey Jacob, appreciate the iteration here! I still think the core detection idea is great, but I think we can scope this back a little bit? I ran some hunts with the proposed logic here and I had some success with it (also tested it against your sample) NLU intent does quite a bit of the heavy lifting. If cred_theft or job_scam fires with confidence, we don't need all the extra layers. We just need a specific body regex to scope it to the employer review context, plus a "why it's malicious" gate (credential harvesting or monetary bait).
Let me know what you think, nice work on this so far! |
Thanks for the great feedback @IndiaAce! I think these are fair points - especially if the I implemented your suggested logic - but left in a regex check for employer review related keywords in the subject of the email and added some additional words to the body-level regex checks that I think are indicative of review solicitation and monetary incentive language. This way we properly scope down to the attack type we're hoping to catch here. I like your proposed structure of a hierarchical-like filtering system - this has been fun iterating on with you. Let me know if you have any other suggestions or comments! |

Description
This new rule targets attempts for solicitation of ratings/reviews of employers. These false solicitations can contain malicious links or be attempts at phishing information or credentials from the receiver.
Associated samples
Example one:
Screenshot (insights)
Rule flagged example one

Rule flagged example two
