(2023-05-26) Gmail Blocking Some FluxGarden Registration Emails

It seems like Gmail is Blocking Some FluxGarden Registration Emails - got a few bounces today with

550 5.7.26 This mail is unauthenticated, which poses a security risk to the sender and Gmail users, and has been blocked. The sender must authenticate with at least one of SPF or DKIM.

• for original dev logs, see references to SMTP in (2020-10-12) Building user management in WikiFlux

I've seen this happen once or twice before, and then it went away shortly on its own.

Today I got a few, so digging in.

I registered myself, confirmed I didn't get a 500/502 error like at (2023-01-01) Can't Reset FluxGarden password. But got the same bounce.

I tried forwarding one of the emails to the same user from my own/manual gmail address, and got the same bounce. I tried sending a fresh email to my vanilla gmail account, and got the same bounce! (The sender account is hosted by gmail, but the domain is registered elsewhere. Also, the sender account isn't @flux.garden, it's a different domain I already pay Google for. Their best practices say Use the same domain for sending email and for hosting your public website. But is that really a requirement?

For now I copy/pasted to send each from my vanilla gmail account.

Oct'2023:

• article about changing rules for less-secure apps.
• google support doc: "Control access to less secure apps"
• doc: "Transition from less secure apps to OAuth"

Ticket from Mar'2022 to ask OAuth support to Flask-Mail. No comments. And not much has been happening with Flask-Mail.

I probably need to try the Flask... Discord channel?

Feb'2024

• hmm there's a Flask-Mailman extension meant to replace unmaintained Flask-Mail
• maybe the real answer is to send via SendGrid - I can do 100 sends/day for free, and there's a Flask-Mail-SendGrid extension

Mar17'2024 - try getting working on laptop

• pip install flask-mail-sendgrid
• create Twilio account (since they own SendGrid)
• do "Single-Sender Verification"
• create API key
• pick "Web API" not "SMTP Relay"
  • actually I don't think I needed to do this
• edit config.py
• edit .env
• start looking for relevant code - argh
  • I think all this stuff is within the User library I adopted (2020-10-12) Building user management in WikiFlux
  • it looks that library already supports SendGrid, I just have to figure out how to control that.
• will it work just by setting SendGrid as the SMTP server? So I do that.
• register on laptop -> flask_user.EmailError: SMTP Connection error: Check your MAIL_SERVER and MAIL_PORT settings
• ah, password is supposed to be the api-key (but rest of settings seem ok/current)
• nope same error
• look more closely, the true/immediate error is

"/Users/billseitz/.pyenv/versions/3.8.5/lib/python3.8/smtplib.py", line 398, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")

• duh, the issue was needed to set MAIL_USERNAME = 'apikey' (not an email address)
• live server: just update config.py and .env -> all good!
• but try sending an email just using gmail-webapp from bill@simplest-thing.com to fluxent@gmail.com and it still bounces back
• realize that, while I'm using Google to provide email/cal/etc for simplest-thing.com, the domain is still registered at Hover.com, so maybe that was really the issue
  • add SPF entry v=spf1 include:_spf.hostedemail.com include:hover.com ~all
  • add DMARC v=DMARC1; p=none; rua=mailto:bill@simplest-thing.com; ruf=mailto:bill@simplest-thing.com;fo=1
  • DKIM - go here to get code, enter it into TXT record at hover.com
• now emails just disappear - no bounce, but they don't get received. Nice job, Google.
• I think I used the wrong SPF, Google's page suggests a different value, which makes more sense.
  • nope still disappears
  • also I'm concerned I might have broken the app-sending - maybe I'll need a fancier SPF record, per this page?
    • trigger an app-email - ah good it went through
• manual email still just disappears when sent to gmail.com - though it shows up at fluxent.com (which Google doesn't have any involvement with)
• find this check-mx page at google, which responds with "If your domain wants to use MTA-STS (optional), the MTA-STS TXT record must comply with RFC8461. MTA STS is malformed."
  • But I don't think I'm using that.
  • ah, it's the TXT with google-site-verification
  • read more, smells like this is (a) optional, and (b) a pain (mainly because of process of publishing a policy)
• omg discover that earlier emails had gone through, they'd just gone to Spam (which doesn't get includes in "All Mail" view!)
  • also discovered I can look at outcomes on emails at the emailLogSearch
    • hrm that seem inconsistent....