Determining the effectiveness of applying a security setting
The following security settings would be a good example here.
Close connection if no PTR exists
Close connection if no PTR Match
I used to have Spambot Detection enabled but I disabled it a few months back because it appeared to me that the only emails being affected by this were legitimate senders.
I currently have both PTR checks above enabled. We have had a few legitimate senders affected by this.
I also have Outbreak Protection enabled
However there is no easy way for me to list all senders (domains) that were affected by security setting abc. This would really help to ensure that a particular security setting is optimal in that it's blocking 99% spam and less than 1% of legitimate emails. I think it would make sense to have some kind of search function to enable administrators to compile a complete list of sender domains that were blocked because of a security setting which would make it easy to pick up legitimate senders that are being blocked as a result of a security setting. In addition, if I saw 95% of the emails being dropped were from legitimate senders, I could disable that security setting.
I feel that we could be missing legitimate emails and don't even know about it because the sender either doesn't realise or does not let us know.
Thank you for sharing your idea to collect statistics on the effectiveness of security settings. It will be considered for future versions.
If you’d like to implement something right now, you can probably accomplish this using a powershell script.