[grc] Question about Google blocking e-mails to radio station deejay list

[al davis]

On Mon, 22 Feb 2021 10:12:31 -0800
Leigh Robartes via grc <grc at maillist.peak.org> wrote:
> I have a question for those who are in charge of e-mail lists for 
> deejays at stations, or perhaps for the administrator of this list.   
> Our e-mail list administrator was noting some e-mails sent to our modest 
> e-mail list (about 160 addresses, since paired down to 77 addresses) are 
> sometimes being blocked by Google, at least the ones sent to gmail 
> addresses.   Our e-mail list is an important organizing tool, and a good 
> way to keep in touch w/ former deejays who might want to start 
> volunteering again now that Covid has changed their life around.
> 
> 
> The administrator claims that if we move our e-mail list to Google 
> Services we would avoid the problem, but obviously we can't do this for 
> ethical reasons, mainly because (as I am told) moving to Google Services 
> would mean all of our @krfp.org accounts would be become disguised 
> @gmail.com accounts.  



> Google has been harvesting the actual text of 
> e-mail messages sent and received from gmail accounts for years, and 
> although they apparently no longer sell that info for advertising 
> purposes, the company still has cached everything written, which is a 
> violation of privacy.

You want spam filtering?  If so, the reason xxxxx has been "harvesting
the actual text" is because you requested for them to do that.

Spam filtering based on actual text, based on an algorithm you do not
control, is filtering of the worst kind.  It could at any time be
extended to filter based on political views, street address, or
anything else you might find bothersome.

The first, and hopefully most, filtering must come from proper use of
standards and tracing, under your control.


> 
> 
> We got a list of e-mail messages Google blocked from an ISP and a 
> message from a tech there who wrote:
> 
> 
> <<<begin snip>>>
> 
> 
> to accept email from other servers, the spf record contains “?all”, 
> meaning make up your own mind about accepting email from servers not 
> listed. 

Your SPF record is a way that you can instruct recipients about how to
judge whether mail claiming to be from you is legitimate or not.

If your SPF record ends with "?all", you are saying that you have no
idea how to know, and leaving it up to the recipient to decide, however
they want.

This is a mark of an improperly set up mail server, so it is likely
that a responsible big server will aggressively screen all of your
mail, to the point of rejecting most of it.

Ending with "-all" tells the recipient to reject any mail claiming to
be from you that does not come from your designated server.

Ending with "~all" tells the recipient to accept mail claiming to
be from you that does not come from your designated server, but mark it
in some way, so the recipient knows there is something wrong. Putting it
in the spam folder is one way to do this.

A quick check tells me that your spf record is something like:
(with some names changed to protect your privacy)

"v=spf1 include:_spf.example.com ~all"

I see:
1. ~all (see above)
2. passing it on to your provider, who says this:
(I changed the numbers)

"v=spf1 ip4:11.22.33.0/24 ip4:22.33.44.55 ~all"

Here they listed the IP numbers of the servers that are GOOD.

So the instructions say to accept as non-spam from the designated
servers, accept but mark as spam if not.

If the recipient is doing this they are following your instructions.


Lists add another layer to this.

Now it is not just your part, and the recipient's part, but also the
part of anyone who posts to the list.

A member of a list posts to the list.  What does that look like?  It
goes through the list and the list messes with it.  Now what?

Let's assume that you have somebody "mememe at example.com" on your list,
who posts to it.  Suppose the poster's SPF says "v=spf1 mx -all".  That
says to accept only if it comes from the one designated server, to
reject if it comes from anywhere else.

If the list leaves the From line alone, so it still says
"mememe at example.com", but now it comes from your list server rather
than from the poster's designated server, the recipient should reject
it .... meaning to bounce it or not even pass it to spam.

This is why many lists (including this one) change the From to its own
address.  (grc at maillist.peak.org here).   If your list doesn't do this,
you should reconfigure it so it does.

In the old days, the usual way was to leave the From alone, leading to
faking of email addresses.


Another factor here is "DMARC" records, and a "DKIM" signature.  You
should have this also set up to be as secure and restrictive as you
can.  I don't want to go into detail on this here, other than to say
that it lets you specify in great detail how you want recipients to
deal with stuff that appears to come from your address.  (but doesn't
really).

Before it even gets in the door, there is the IP reverse lookup test.

Basically it goes something like this ......

Suppose my server wants to send you some mail.

My server calls your server and says...

"HELO my.email.server.com".

Your server checks the caller id and it says "33.44.55.66".

Your properly configured server does a quick check ....

Does 33.44.55.66 correspond to my.email.server.com ?
and does my.email.server.com correspond to 33.44.55.66 ?

It really does need to check both ways.

When this is confirmed, now it is ok to move on to SPF and DKIM.



If everything on your end is correct, and a big provider is rejecting
your mail, call them and demand they fix the problem.  If you are
correct and professional, they will fix it.

But before you do this, make sure it is all correct and tight on your
end.

If this case, you are already using a provider for your email service.
I am not familiar with the provider you are using.  You are paying them
for the service.  Ask them to fix it.

> The above message implies Google is not letting _non-spam_ e-mails 
> through, justifying the practice by saying they are suspected spam.

There is some truth to that.  Too much, actually.  Sometimes Google
(and hotmail, yahoo, etc ... let's be fair) reject email that is
perfectly set up, with everything correct.

> 
> 
> _My Question:_
> 
> 
> How can you operate an e-mail list (some of them, such as GRC, which 
> have hundreds of addresses) without getting messages blocked and without 
> going through Google Services? 

1. Make sure your end is complete, correct, and tight.
	- IP-lookup, spf, dkim, dmarc
	- you follow your own rules,
		especially when forwarding mail through your list.

2. Get on their case if you are clean, and they are still rejecting
your mail.


> Do we need to change to a different service? 

Only if the service you are using is messing up.

DO NOT just switch to google, yahoo, aol, hotmail, etc.. in response to
this problem.  If you do (for this reason) you may hide the problem
from yourself, but in the big picture, for everyone else, you become
part of the problem.

Another point .. instructions to the users, the people who are using
gmail and losing mail ....  It is their duty to do something about it.
If the problem is mail going to spam when it shouldn't, they should
click the button "not spam" to help gmail's filters learn what to do.


>  What do you use? Have you been able to avoid problems like this?

Quick check ....  From what I can see, this here list, hosted at
peak.org, seems to do it all correctly.