If anyone would like to test this, you likely need to find an inbound message which has a "Return Path" field that is different than the "From" field.

The (GW) USER will see the "From" field in their message but it appears SMG is using the "Return Path" field (if one exists).

You can compare what the USER see's and what SMG is processing by comparing:

1. What the user see's in the "FROM" field in their groupwise message

2. What SMG considers to be the sender by locating the corresponding message in the message tracker

The users will obviously use the data in the GW FROM field to add entries in their BlackLists, but SMG does not use that if a "Return Path" field is provided. 

Many Junk Mail advertisers fill both these fields with different content.  Often, the "From" field contains the address of their "client" while the "Return Path" field points back to an address at the junk mailers domain...

Maybe I ask the wrong question. But do you have a blacklist filter which is connected to a "User Black List" to catch those messages?

Yes.  One for a "Global" Blacklist (which I maintain independently) and one which links to the QMS (User) Blacklist

I should also be clear...

The QMS/User Blacklists are worki just fine.  HOWEVER, users have to supply the "sender" address that SMG "wants".

This is not obvious to users if the SMTP message contains a "Return Path" field (as many junk mailers provide)...

Edward - did you get the solution to this one?  If not, would you open a case? - Thanks Pam

Just got back from the holidays.  I need to bring our "production" SMG system up to the current version now that we have had time to confirm all appears well for the update process and behavior of our "test" system. 

I want to confirm we are still seeing the same filtering limitations/issues with the current version BEFORE I open a case...

Yup, still an issue with the latest release. 

If a spammer specifies a "Return Path" address which is different than the "From" address, then SMG will still let that email pass through even though it is blacklisted.  User's only "see" the "From" address in their email so that is what they will specify - but then SMG doesn't block the email (cause the "Return Path" field is different....

Unless a casual user knows and understands how to read their "message source" they won't even know about any "Return Path" field...

Case has been opened.

Please note, It is "Return Path" which overrides the BlackList "From" field (not "Reply To" as I had noted in the last couple of messages.  I will go back and edit them now...)

After a couple of weeks back-and-forth trying to get Tech Support and the Devs to understand this SIMPLE issue, we finally got past that yesterday.  (If only they would READ my opening case notes rather than skimming them. Anyway...)

MFDevs confirm this behavior and say it is "By Design".  (Thats developer-speak for "Yes you are correct but we are not going to fix this")

To which I basically responded:  "Then it's not {Effin} properly designed!"

The basic problem (from a users perspective) is that they get email that they want to block, they create a blacklist entry which equals (or is some wildcarded permutation of) the "FROM" field they see in their mail client (GW), and the Blacklist Entry doesn't work for THEM!

Nothing else REALLY matters.  This is the only source field that "casual" users see and the Blacklist is NOT working for them.  Thats all they care about.  They don't want to HAVE to become SMTP protocol gurus to process MIME headers so they can simply use a Blacklist option presented to THEM!  ...and they SHOULDNT have to!  SMG should be smart enough to scan email and use ANY field in the MIME Header.  ..and if QMB Blacklists are running during the SMTP transaction BEFORE all the MIME info is exchange then the QMS process is running at the WRONG TIME.

Software should work for us - we should NOT have to work for the software.  ...and there is absolutely no reason why the QMS Blacklist couldn't be made to work properly.  If its currently "working as designed" then it wasn't designed PROPERLY.

Just to recap on what the underlying technical issue is (as none of us here are casual users):  If an email message has a "Return Path" field (which the user DOESN"T see unless they are capable and willing to parse the message source) which is different than the "FROM" field (which the user DOES See), then QMS will NOT Blacklist the email based on the "FROM" field...  This results in users creating Blacklist entries which "Don't Work" and calls to tech support which ultimately end up on my (our?) desk.  It also results in explanations to users about how they can open up their Message Source and compare the Return-Path to the From field and see if they are different and then which one to use.  This, in turn, results in looks of incomprehension with replies of the basic form of "Well thats *&^%$.ing stupid!" to which I have no argument...

Bottom Line:  If you are going to open up a configuration Option to End Users, then that Configuration Option should actually WORK for end users!

Anyway, this gets better (and even ends in a bit of double IRONY).  So, if you have hung on this long, please hang in a little bit longer for the payoff...

We are seeing a fairly significant amount of email (getting through the rather opaque SPAM filter) which we would like to block.  (Hence the Blacklist) A fairly good portion of THAT email (20-30%) is from "SPAM Services" which send out mail from paying "clients" to a list of email addresses they have collected.  (As we'll find out later, some of these services operate in a bit of a grey zone and also offer legit (opt in) mailing services)

Mail from these SPAM services very often has a "Return Path" which is different from the "From" field.  The "Return Path" often (always?) points back to the "Service" while the "FROM" field typically references the "Client".  All that SMG users see is the "client" in the "From" field.  The ONLY thing that QMS Blacklists is the "Return Path".  If they are not equal, then the Blacklist entry doesn't work (FOR users)!

Tech support has asked me to send them complete Mime files for several such messages just so they could UNDERSTAND the problem.  Apparently, my detailed description in my original case notes, or my subsequent copy/paste of the explicit fields in question was not enough...

OK, so here is where the irony comes in!

I have another case opened about the inability to delete Blacklist entries.  I get a call this morning from the tech about a message he sent to me yesterday.  I did not get it!  Here it comes...  Wait for it... It was Blacklisted!!!

IRONY #1:  MF doesn't have to look very far for examples of email which have a "Return Path" which is different than the "From" field.  The MF case support email system itself is doing this!  (...and apparently sharing a "mailing" service with another entity who I have chosen to block).  Here's one example (Please don't ask me for the full Mime file ;-)

     "Return-Path": support.case=microfocus.com__0-8puuzqwbgdk4l0@drvtm89zr54gp2.1t-vhdpeay.um9.bnc.salesforce.com

     "From":  support.case@micofocus.com

So, adding a Blacklist entry for "*@microfocus.com" wouldn't work in this example.  The entry would have to be something like "*.bnc.salesforce.com" if you wanted to block all email from the source (...which is exactly what I did.  Ooops, My Bad!)

IRONY #2:  In order to Unblock email from the MF Case System...  Which contains the solution of how to solve the problem of "not being able to delete Blacklist entries"...  I have to delete the Blacklist entry which is blocking email from MF themself!

Too Funny!

Anyway, since I have EVERYTHING configured to quarantine to QMS (except viruses), I could easily just "release" the inadvertently Blacklisted messages from QMS so I could discover how to delete the associated Blacklist entry.  The solution worked and if you are interested you know how to find the other/associated thread on this forum.

(We don't reject or drop connections on ANYTHING.  This includes SURBL, RBL, and SPF.  The problem is that SO MANY email administrators have misconfigured their systems and if an email is rejected than we have no hope of recovering it.  An amazing number of systems have a malformed SPF record or legit remote users of those systems are not following "the rules" and relaying mail through their local ISP.  Many "legit" systems are misconfigured and relaying mail...  Anyway, IMHO, better to quarantine everything rather than trying to explain to one of MY users that they need to explain to THEIR sender, that they need to explain to THEIR sysadmin that they don't have their system configured properly (and effectively "don't know what they're doing"...)  I've always worked to whitelist the offenders, but by the time you have enough data for the WhiteList entry you have already lost one or more messages.  Yes, EVERYTHING goes to QMS (other than viruses).  Users can also self-service in this manner...)

TLDR:  Still Not Fixed.  "Working as designed" = Not designed PROPERLY!

Edward, that is a great explanation. Thank you for providing it.

I'm sure it took you quite a while to finally put together all the pieces and understand just what was happening. Hopefully this thread will save others a lot of grief.

evanevery said:

Software should work for us - we should NOT have to work for the software.  ...and there is absolutely no reason why the QMS Blacklist couldn't be made to work properly.  If its currently "working as designed" then it wasn't designed PROPERLY.

I couldn't agree more with you on this point. I can't believe that any developer designs a product to misbehave but we are only human and oversights are a fact of life. Sometimes these oversights can result in a design deficiency and this certainly appears to be the case in this instance.

If the product were properly documented we might be able to refer to the documentation to show that the observed behaviour does not conform to the documentation to make our case but even without such documentation it is a bit of a stretch to suggest the product was designed to work this way. Unfortunately, this is not the first time I have seen this.

A design deficiency is a bug but if the product was designed to work this way and we are unhappy with how it works we must request a product enhancement. Without any documentation, it is just too easy to claim that the product is Working as Designed!