Why emails go to Spam or junk folders. It’s annoying isn’t it? You send someone an email you don’t hear back from them, you ask them about the email only to be told that it ended up in spam. Why does this happen, though? Find out why emails go to junk or spam folders, what can cause this, and how to prevent emails going to junk or spam folders.
What is Spam?
Spam, by definition, is unsolicited emails, or an unsolicited email. An email that you haven’t requested, don’t want, and haven’t opted into receiving, that’s spam. Some people might argue that marketing emails are spam, but if you opted in to receiving these emails, or didn’t opt out of receiving these emails, they’re not technically spam. This is because you’ve consented to receiving them.
Email marketing isn’t technically spamming, provided these emails are sent to people that opted in (or didn’t opt out) or receiving these emails.
Why Spam is Sent.
Spam is an easy way for a message to reach a large number of people, with a minimal amount of effort. The objective of spam varies. Some spam is unsolicited marketing emails, and whilst this isn’t too malicious, it can be annoying. Some spam is sent to trick you in to doing something, such as clicking a link that leads to some kind of computer infection, or to trick you in to providing login credentials. This is called phishing. Generally speaking, if an email that takes the form of “you need to click this or reset a password here to prevent an unwanted situation occurring”, it’s likely to be a phishing email and should be treated with caution (don’t click the links!).
You can often identify if an email is suspicious by looking a the source code of the email, or hovering over links to see where they lead (don’t actually click the link though!). If the link leads to a website on a domain that differs to where the email purports to be from, it’s probably a phishing email.
Usually, the overall objective with spam is ultimately to make money. There might be some steps between the spam email being received and money being made, which can convolute things, but usually money is the ultimate motivation.
Why Spam is a Problem.
From your perspective spam ranges from being annoying to posing a security risk (if you click links in malicious emails). From our perspective spam costs us money. This cost comes in the form of disk space, and electricity. The mail server processing spam emails uses electricity. Spam being stored (i.e. not deleted) in mailboxes can mean we have to add more disks (which we have to buy) to servers, so that they don’t run out of space.
You might think something like “spam emails are pretty small, they can’t take up that much space” and you wouldn’t be wrong, but if you consider there are hundreds of thousands of spam emails, possibly more, being stored, that all adds up.
The disk space usage specific to spam emails being stored increases in line with the number of mailboxes being operated. For mail providers, this can add up to a pretty big problem.
What mail providers do about spam.
Mail providers tend to have some kind of default anti spam methodology in place, just to mitigate what I’ve outlined above. This tends to operate on a kind of “put whats believed to be spam in a spam folder, and periodically purge spam folders”.
The “believed to be spam” methodology isn’t foolproof. This is because:
- With the exception of AI and machine learning, computers don’t know things, and they don’t have any intuition. Everything a normal, non AI, computer does happens because a human wrote code that does that.
- A lot of spam emails have legitimate qualities, and a lot of legitimate emails can have spam like qualities.
Writing a program that filters spam, but not legitimate emails is pretty difficult due to this lack of knowledge and intuition, and the spam/legitimate email overlap.
A lot of anti-spam methodology uses a rule based scoring system, and a score threshold. A spam score threshold for different actions (put in spam folder or outright reject, for example) is defined. A set of rules are applied to emails, and when a rule is triggered, a certain amount of score is accrued, the more rules that are triggered the more spam score is accrued. Whilst these rules do vary between providers, they can contain some things that you might not expect.
Spam Scoring.
Some aspects of spam scoring is based on the content of emails.
If an email makes reference to a payment or an invoice, it can accrue some spam score. This is because of invoice spamming, which is a technique where spammers send someone an invoice asking for payment that’s completely made up. You probably think this is fairly unrealistic on a personal level, and you’d probably be right. Then again, consider the overworked employee in the procurement department who’s paying many invoices a day, that hasn’t had any lunch and it’s also Friday at quarter to 5 and they just want to go home. They’d probably be a bit easier to trick with some invoice spam. OK, sure, the chances of getting an email to that particular person is quite slim, but then again, a spammer can send several thousand emails in under an hour in a few clicks. The odds of reaching the fraught employee in procurement increase with larger volumes of spam emails. To a spammer, it’s worth sending thousands of emails if one payment is received.
Whilst what I’ve outlined above is one example, there are others. Different domains have different spam scores, believe it or not. Use a .com and you’re in the realms of normality, but use a .digital domain, and an anti spam service may allocate more spam score just due to the use of the .digital domain. Free domains (such as .top) tend to be allocated more even spam score.
Using plain text, or only writing a small amount of text is also likely to gain a greater spam score, simply because a lot of spam takes this format. That’s why your test email didn’t arrive in your inbox. A small amount of text and a link, yeah, that looks like spam! More score!
Spam scores also vary according to how the email was generated.
Information detailing how an email was generated is often included in email headers. An email that was generated using PHP (such as a contact form email) will gain a greater spam score, simply due to it being generated using PHP. This is because a lot of spam is generated using PHP. Due to this, you have to be very aware of how to improve contact form generated email’s deliverability, to keep the spam score of these emails low enough to be received.
Spam scoring also takes in to account email authentication. A lack of email authentication records being in place, or being misconfigured, can result in higher spam scoring, or even emails being flat out rejected. Gmail now won’t accept emails from a domain with no SPF or DKIM record, for example.
Email Authentication.
Email authentication is a method used to validate emails. If emails aren’t validated, they’re likely to receive a greater spam score, or even be flat out rejected.
DNS records are used to authenticate emails, and these come in the form of DNS TXT records. TXT records are effectively words (letters, numbers and punctuation) advertised using DNS to allow them to be looked up by recipient mail servers upon the receipt of an email.
The domain of the from address of an email is what’s used to look up these email authentication records by recipient mail servers upon the receipt of an email. So if I send an email with a from address of person@netnerd.com the email authentication records for the netnerd.com domain are what’s checked.
Email authentication records take two forms Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM).
SPF (Sender Policy Framework)
SPF is used to define permitted senders, which are the platforms where emails should originate from, for the domain un question. For example if this is the SPF record that’s advertised for domain.com it means that emails with @domain.com from addresses can only originated from a server with 1.2.3.4 as it’s IP address:
v=spf1 ip4:1.2.3.4 -all
DKIM (DomainKeys Identified Mail)
DKIM adds a cryptographic signature to email headers, allowing the recipient’s server to verify that the message has not been altered during transit and that it was sent from an authorised domain. When an email is sent, a private key is used to generate the cryptographic signature, which is added to the email headers. A public key (published in the publicly available DKIM record in DNS) is used to verify the cryptographic signature. If the verification process confirms that the signature matches the hash of the email’s signed content, the email is authenticated as legitimate and unaltered.
There is also DMARC records (again a DNS TXT record), which is used to define a policy (what should happen if DKIM and/or SPF checks fail), and an email address to which reports about SPF and DKIM failures should be sent. Whilst these don’t validate the source of emails, or the emails themselves, some tools will flag the lack of policy or reporting as a problem.
Why the email you sent went to junk or spam.
The email that you sent went to junk or spam because it either had spam like qualities due to its content, or how it was generated (and therefore triggered spam rules), or because it lacks email authentication records, or due to a combination of the two.
It got put in the spam or junk folder because the recipient provider deemed it to be spam like enough to be of no interest to the recipient, and for it to be automatically deleted in the future, so as not to consume disk space.
How to improve your email deliverability, when using netenerd.com.
Deploying DKIM and SPF records are DNS records validate the source of your emails, and the emails themselves.
The “Email Deliverability” facility in the web hosting packages we provide can be used to deploy DKIM and SPF records. As these records and DNS records, your domain needs to be using our nameservers, and you need to have your mailboxes hosted with us, for the “Email Deliverability” facility to be able to do the job for you. This doesn’t mean you should change your domain’s nameservers to ours, or migrate your mail services, what it means is:
If your domain IS using our nameservers, AND your mailboxes are held with us, you can use the “Email Deliverability” facility in cPanel to deploy DKIM and SPF records.
If your domain is NOT using our nameservers but your mailboxes ARE held with us, you can use the “Email Deliverability” facility in cPanel to deploy DKIM and SPF records, but you’ll then need to copy the DKIM and SPF records from the Zone Editor in your cPanel with us, in to the DNS management area specific to the nameservers your domain uses.
If your mailboxes are NOT held with us but your domain IS using our nameservers, you’ll need to obtain SPF and DKIM records from your mail provider and then deploy them in the Zone Editor in your cPanel. Do not use the “Email Deliverability” facility in cPanel to deploy DKIM and SPF records if your mailboxes aren’t held with us.
If your mailboxes are NOT held with us but your domain IS NOT using our nameservers, we’re not involved with this situation. You’ll need to liaise with your mail provider and the party that manages your domain’s DNS.
Sorry about all the capitals, but we want this to work for you!
If your mailboxes are held with us, and you’re using our nameservers, it’s quite straight forward to deploy DKIM and SPF records.
Click the “Email deliverability” icon in your cPanel:
If you see the word valid against your domain, this usually means that the DKIM and SPF records are already installed:
If you see a message saying “Problems Exist…” you can click on the “Repair” button (please be patient the repair takes a few minutes) to add DKIM and SPF records:
The SPF and DKIM records will then be added to the DNS zone for your domain, which you can access by clicking on Zone Editor > Manage (on the same line as your domain).
If your mailboxes are NOT held with us, but you’re using our nameservers, it’s quite straight forward to deploy custom SPF records, you can add these custom records by clicking on “Email deliverability”, but you’ll have to initially obtain the required SPF record from your mail provider before being able to proceed with the below:
And then clicking the “Manage” button:
And then click the “Customize” link:
What you’ll then need to do depends on the type of record you have been given by your email provider.
Usually you’ll be provided with an include record that would look something like this:
include:spf.protection.outlook.com
or this:
v=spf1 include:_spf.google.com ~all
In both cases, it’s the part after include: that you’ll need.
As per the examples above this would be:
spf.protection.outlook.com
or
_spf.google.com
You’d then add the include by clicking “+ Add A New “+include” Item”:
And then entering the part of the record you’ve been given by your mail provider that’s after “include:”
As per the examples given above:
or:
After adding the custom SPF include you’ll need to click the “Install a customised SPF record” button at the bottom of the page to commit your change (please be aware propagation applies, which could take up to 24 hours):
Alternatively, if your mail provider has given you an IP address (for example 1.2.3.4) you’d add this using the “+ Add a new ip:4 item” facility:
Again, you’ll need to click the “Install a customised SPF record” button at the bottom of the page to commit your change (please be aware propagation applies, which could take up to 24 hours):
Bulk emailing, and marketing emails.
We strongly advise using a bulk email provider if you’d like to send marketing emails, such as Mailchimp, Mailerlite, or Brevo. If you use a provider of this nature you will have to obtain their SPF and DKIM records, and then add them to your domain’s DNS using the guidance above.
In addition to this, you’ll need to make sure that the marketing emails you send conform to bulk sender guidelines. Whilst these vary between providers, you can find this guidance for the following providers here:
