E-Mail Codelogin Soft-reject timings are wrong

API Bugs
1

Dear OpenAI Devs,

I run an E-Mail service and was surprised by users being unable to login when prompted for code confirmation via E-Mail, with the Code E-Mail being severely time delayed and thus being invalid by the time the users enters the E-Mail. I found out why and fixed it via whitelisting, but I believe your E-Mail service that you use to send Code confirmations: mandrillapp.com as used by MailChimp is misconfigured.

My spam filter is Rspamd and it’s detection issues a soft-reject for OpenAI mails. Here is the ranking:

URIBL_GREY (2.5) [mandrillapp.com:url,mandrillapp.com:dkim]
SUBJ_EXCESS_QP (1.2)
SUSPICIOUS_URL_IN_SUSPICIOUS_MESSAGE (1)
RBL_SENDERSCORE_REPUT_9 (-1) [198.2.145.224:from]
URI_COUNT_ODD (1) [5]
FORGED_SENDER (0.3) [noreply@tm.openai.com,bounce-md_31165340.6783399a.v1-544db3b5d88146aaa8aee30ecab5e0e7@mandrillapp.com]
BAD_REP_POLICIES (0.1)
MIME_GOOD (-0.1) [multipart/alternative,text/plain]
MANY_INVISIBLE_PARTS (0.05) [1]
DKIM_TRACE (0) [mandrillapp.com:+,tm.openai.com:+]
RCVD_IN_DNSWL_NONE (0) [198.2.145.224:from]
MISSING_XM_UA (0)
REDIRECTOR_URL (0) [mandrillapp.com]
R_DKIM_ALLOW (0) [mandrillapp.com:s=mte1,tm.openai.com:s=mandrill]
MIME_TRACE (0) [0:+,1:+,2:~]
ASN (0) [asn:14782, ipnet:198.2.128.0/19, country:US]
DMARC_POLICY_ALLOW (0) [tm.openai.com,reject]
FROM_HAS_DN (0)
TO_MATCH_ENVRCPT_ALL (0)
R_SPF_ALLOW (0) [+ip4:198.2.145.0/24]
TO_DN_NONE (0)
RCVD_COUNT_TWO (0) [2]
RCVD_TLS_LAST (0)
PREVIOUSLY_DELIVERED (0) [<redacted>]
FROM_NEQ_ENVFROM (0) [noreply@tm.openai.com,bounce-md_31165340.6783399a.v1-544db3b5d88146aaa8aee30ecab5e0e7@mandrillapp.com]
GREYLIST (0) [greylisted,Sun, 12 Jan 2025 03:45:13 GMT,new record]
ARC_NA (0)
RCPT_COUNT_ONE (0) [1]

The soft-reject prompts an E-Mail service to resend the E-Mail to confirm it is not a spam bot. This does happen correctly, but with a delay that is configured too high. The delay you have set is about ~15mins.

As such, by the time the users gets the code, it is already timed out.

The ranking for the Email being delivered is here:

URIBL_GREY (2.5) [mandrillapp.com:url,mandrillapp.com:dkim]
SUBJ_EXCESS_QP (1.2)
SUSPICIOUS_URL_IN_SUSPICIOUS_MESSAGE (1)
RBL_SENDERSCORE_REPUT_9 (-1) [205.201.136.238:from]
URI_COUNT_ODD (1) [5]
FORGED_SENDER (0.3) [noreply@tm.openai.com,bounce-md_31165340.67833be6.v1-6890d7c7819c4605936a573098ea23c2@mandrillapp.com]
BAD_REP_POLICIES (0.1)
MIME_GOOD (-0.1) [multipart/alternative,text/plain]
MANY_INVISIBLE_PARTS (0.05) [1]
DKIM_TRACE (0) [mandrillapp.com:+,tm.openai.com:+]
RCVD_IN_DNSWL_NONE (0) [205.201.136.238:from]
MISSING_XM_UA (0)
REDIRECTOR_URL (0) [mandrillapp.com]
R_DKIM_ALLOW (0) [mandrillapp.com:s=mte1,tm.openai.com:s=mandrill]
MIME_TRACE (0) [0:+,1:+,2:~]
ASN (0) [asn:14782, ipnet:205.201.136.0/21, country:US]
DMARC_POLICY_ALLOW (0) [tm.openai.com,reject]
FROM_HAS_DN (0)
TO_MATCH_ENVRCPT_ALL (0)
R_SPF_ALLOW (0) [+ip4:205.201.136.0/23:c]
TO_DN_NONE (0)
RCVD_COUNT_TWO (0) [2]
RCVD_TLS_LAST (0)
PREVIOUSLY_DELIVERED (0) [<redacted>]
FROM_NEQ_ENVFROM (0) [noreply@tm.openai.com,bounce-md_31165340.67833be6.v1-6890d7c7819c4605936a573098ea23c2@mandrillapp.com]
GREYLIST (0) [pass,body]
ARC_NA (0)
RCPT_COUNT_ONE (0) [1]

It’s fairly interesting, that there are things rspamd finds incorrect about the way openai sends its E-Mails and issues a soft-reject in the first place. Anyhow, the 15 minute delay is clearly a misconfiguration in the context of OpenAI’s Code. I’m sure other smaller Mail Services run into the same thing.

1 Like
2

Let me see if I have the situation understood correctly.

You run an email scanning service that “soft-rejects”, which seems like an actual rejection, by erroneously considering OpenAI login code emails as spam.

The 15min delay between the resend request being honoured and your software making that request due to a false spam detection is to long for your system to correctly deal with?

3

OpenAI’s mail server is sending a second email for account verification but only after the time window to verify the request has expired.

OP is saying that the second mail should be send sooner.

6

I Self-host E-Mail for multiple domains, and users among that is a Spam scanner, as standard for E-Mail providers. The program / service used for spam filtering is the industry standard RSPAMD.

What happens to an E-Mail following a scan by the Spam filter and anti virus scanner is described with actions. This is not specific to RSPAMD, but the way E-Mail and bounces are handled in general. One of those actions is called a soft-reject, where a filter found inconsistencies with the way this E-Mail was sent, which is the case for the E-Mails sent by OpenAI, see the report above. (Mainly due to OpenAI’s sending service being greylisted)

A soft-reject happens when a Spam filter isn’t sure and provides HTTP Code 400 back, prompting the E-Mail sending service to resend the E-Mail, to reduce the likelihood that the sending service is a Spam bot.

OpenAI’s email sending service (MailChimp) does this, honors the standard way to respond to such a bounce message, but does so on a delay of 15 minutes, which is too slow for the Time limited code provided by OpenAI. Normally, this happens instantly, within seconds. For some reason MailChimp as configured by OpenAI doesn’t do this instantly, but with that delay. I whitelisted OpenAI’s E-Mail sending service but I shouldn’t have to do so.

2 Likes
7

There is still a bounce-resend delay as of 21st of January. Actually 20 minutes, see screenshot below. OpenAI’s config of Mandril is clearly incorrect in the context of login codes.

image
image1897×435 50.4 KB

1 Like