Universal email verification plugin (for DOI Double Opt-In and more)

Has your proposal been discussed on the Mautic Forums already?
https://mautic.slack.com/archives/CFYFTLK6K/p1748966042880959
and
https://forum.mautic.org/t/universal-email-verification-for-doi-double-opt-in-and-more/35906

Is your feature request related to a problem? Please describe.
Double Opt-In a.k.a. #DOI (or to be more general: a mechanism for proving the authenticity of a form submission) is an important feature these days.

Mautic has nothing like that built-in, so people have to build complex things manually and redundantly, struggling with all sorts of challenges from "email is not sent immediately" to "how can I prove xyz?"

There are 3rd party solutions and tutorials, but none even close to being universal and user-friendly.

Describe the solution you'd like
The following specs describe the solution that resulted from a lot of iterations and feedback from others.

It consists of 3 stages:

• mvp

• basic feature complete

• future ideas

We intend to implement this as a high-quality 3rd-party plugin first, for Mautic 5. Maybe it can be part of Mautic 7 core.

The general concept is to see the DOI (email verification of a form submission) as integral part of the form - and simply have form actions that are only triggered once the DOI link has been clicked. No campaigns needed.

Stage 1: MVP

a) Mautic form backend view

• Changes in the Mautic form backend view / “Actions” tab:

  • Headline “Actions immediately after form submit” above existing dialogue

  • Headline “Actions after successful email verification” underneath existing dialogue

  • All form actions in “Actions immediately after form submit” automatically show up in “Actions after successful email verification” as well

  • (NOTE: “actions after email verification timeout” is out of scope for Stage 1)

  • (NOTE: “update contact conditionally” is out of scope for Stage 1)

• New “Email Verification / DOI” tab in Mautic form backend view

  • Headline “Email Verification (“Double Opt-In”) Details

  • “Verification email to send” - standard dropdown to select from existing emails - MANDATORY

  • “Follow-up email to send” - standard dropdown to select from existing emails - OPTIONAL

  • “Thankyou page redirect URL (after successful email verification)” - URL input field - OPTIONAL

  • “Verification error redirect URL (after unsuccessful email verification, e.g. timeout or invalid hash value)” - URL input field - OPTIONAL

  • (NOTE: “Skipping the Email Verification” is out of scope for Stage 1)

b) DOI Link

• new email token {doi_link} can be used in emails, which is rendered as link to our doi endpoint

  • obviously this will have to contain a hash value → generated as HMAC of (submission_id, email, timestamp). (That avoids exposing a predictable auto-increment ID in URLs and protects against rainbow-table lookups.)

  • only the hash is stored (not the values in clear at this point.)

• When {doi_link} is clicked and hash is recognized:

  • Redirect browser to feedback page

    • if configured: “Thankyou page redirect URL (after successful email verification)”

    • else: To an unstyled HTML page that says “Email verification successful.”

  • Timestamp is saved in form_submissions.doi_date_confirmed

  • “Actions after successful email verification” are initiated

    • This should be the actions as they were defined when the form was submitted.

    • NOTE: If it is easier to use the actions as they are defined when the email is verified, please discuss with PM

• If {doi_link} is clicked and something goes wrong, e.g. hash is not recognized:

  • Redirect browser to feedback page

    • if configured: “Verification error redirect URL ”

    • else: To an unstyled HTML page that says “Something went wrong! Email verification unsuccessful.”

c) DOI Email and Follow-Up

• Upon form submit: “Verification email to send” is sent to leads.email

• After “Follow-up wait time” (as defined in plugin config) has expired and {doi_link} has not been clicked : “Follow-up email to send” is sent to leads.email

  • This can be triggered by a console command (via cron)

d) Plugin configuration

• plugin configuration allows setting

  • “Follow-up wait time (hours)” - integer input

    • (?) text: When do you want the follow-up email to be sent for missing email verification? (Hint: You need a cron job to be set up for this!)

  • (Out of scope: set global timeout)

  • (Out of scope: activate housekeeping)

Stage 2: basic feature complete

• Skipping the Email Verification

  • e.g. based on form field values or contact field values

  • or based on existing Mautic cookie and preexisting DOI for that cookie (i.e. we KNOW the owner of this cookie has already proven that tehy also own the email address given)

  • including modified form submission feedback in case of skipping

• “conditional actions” (outside of this plugin?)

  • e.g. update contact (if <form field> <operator> then <contact field> <value | calculated value>)

  • NOTE: This can currently be replaced by a campaign

    • Start campaign from form action

    • have condition on form field values MOI=1

    • then set contact field MOI=1 and something like MOI_confirmed concat (timestamp)

• Persist form field status at time of submission (write forms.cached_html to form_submissions.doi_formstatus)

• Token expiry setting

• Handling of “leads.email empty / no form field maps to leads.email”

Stage 3: Future ideas

• new form action: "update Marketing Opt-In" (= extra convenience!)

  • fixed custom fields “moi” (bool and audit) - or rather dedicated table?

  • select form field that decides Opt-in given or not

  • select contact's MOI bool field (sets to true or false depending on form field value)

  • select contact’s MOI audit field (adds timestamp/action/form submission to field value)

  • campaign condition and segment filter to check “moi” bool status

• allow multiple moi flavors (e.g. per brand)

  • define available moi flavors in plugin config, and optional default)

    • plugin comes with one generic moi flavor

    • user cannot delete all flavors

  • user can choose applicable moi flavor in form action

• Direct support for Mautic Landing Pages as feedback pages (“Thankyou page redirect URL” / “Verification error redirect URL”) after DOI link click (those can of course be given as URL)

• Set generic feedback pages (“Thankyou page redirect URL” / “Verification error redirect URL”) in plugin configuration

  • → Use those as preset in new forms

• Multi-language features (normally not required as forms are currently single-language, too)

  • translated doi emails

  • language-aware redirects

  • language-aware generic feedback pages

• multi-brand (i.e. URL aware) generic feedback pages

• per-form “Follow-up wait time”

• housekeeping, i.e. cleanup of non-confirmed DOIs

  • global or per-form cleanup timeout definition

• NHI honeypot awareness / support

• Offer only emails in “Email Verification (“Double Opt-In”) Details that contain the {doi_link} token

• MAYBE LATER, MAYBE NEVER: Generic Audit trail

Describe alternatives or workarounds you've considered
Can't recall. A ton.

Additional context
I think I covered it all :)

Does this issue could impact on users private data?
yes

Funded by
Leuchtfeuer Digital Marketing

#Mautic

#Mautic

#Mautic