You get a message from your bank warning you of a threat to your account and inviting you to click a link that validates your personal information and updates your password.
In this context, you're clearly suspicious, but phishing works by providing the right contextual clues such that those flying by at 90 mph, as normal people do on the internet, might not pause long enough to reach the point of suspicion.
One of those all-important contextual clues is “trust signalling,” which is a kind of psychological priming.
I can influence your reported support or opposition to an issue by first telling you a heartwarming human interest anecdote or instead by appealing to cold, hard economic facts. In either case, I've “primed” you by getting your brain to think about the issue from a certain point of view. The easiest thing for it to do is answer the question from the same frame.1
The phishing email frames the message by warning you of a threat, thereby subtly priming you to think it isn't one. As long as the message looks right—it does seem like it came from your bank—then, paradoxically, it's the genuine threat of identity theft that most compels a click. In wanting to ensure your money is safe, you do the opposite.
Importantly, it isn't that these fraudulent messages have to work on everyone all the time to be successful. They just have to catch enough of us at the right moment that they pay for themselves and a little extra.
Maybe you're thinking you're too clever to fall for anything like that. But almost certainly you have, because what’s being hacked here isn’t a machine, and it isn’t beaten by cleverness.
Disinformation appears to be anti-disinformation.
Just as phishing campaigns try very hard to appear as a defense against phishing, disinformation campaigns appear as the opposite—as journalists, fact-checkers, and community standards moderators purporting to destroy the very thing they’re peddling. They want you to know We’re here to help.
Just as phishing campaigns play on the urgency of a real threat, disinformation campaigns use existing fears of genuine out-groups—opposing political parties, domestic terrorists, anti/fascists, pedophiles—to scare you into some reaction.
Just as phishing campaigns only have to work occasionally to be profitable, disinformation campaigns don't have to snare everyone to move the needle on an issue. In fact, they work best when both medium and message are targeted not to everyone but specifically to you.2
Disinformation appears from your side.
It's important here to draw a distinction between disinformation and plain old information. A traditional article or op-ed is the latter. It tries to convince you with counterargument. It may be wrong, even false, in which case it's misinformation. But being misinformed is not the same as being disinformed. Only one is deliberate.
Just as a phishing attack comes from “your bank,” disinformation comes from “your side.” It wasn’t the North Vietnamese (or the Russians) feeding false body counts to the evening news during the Vietnam War, just as it wasn’t the terrorists (or the Russians) who lied about WMD in Iraq.
There are in fact numerous sources of disinformation, some of which you may not consider to be hostile.
Disinformation appears on the news.
Often, the people who disseminate disinformation, on the news or on social media, genuinely believe it themselves, which of course makes them excellent salespeople.3
No outlet is immune, and unfortunately, even a biased source will seem honest by occasionally admitting error. That proves nothing.
Keep in mind the purpose of disinformation is not to convince you of falsehoods. Its authors will certainly lie if they have to, but they’re just as likely to employ honesty as deception. In fact, in as much as disinformation has to appear true, it often will be based in fact.
I'm going to repeat that because it's very important.
Disinformation appears true.
I don’t mean that disinformation’s lies are convincing, although that’s also the case. I mean disinformation can literally be true.
If that seems strange, consider that in casual language we call that a lie of omission. I don’t have to speak a falsehood to lie. I can print only facts. Just not all the facts.
Write this down somewhere you will see it regularly:
Disinformation isn't about right or wrong.
People think disinformation is evil, so they assume the point is to deceive. But disinformation isn’t about truth or falsity, good or evil, right or wrong. The purpose of disinformation is to influence. That's all.
Like advertising, disinformation aims to tell you whatever you need to hear to alter your beliefs or behavior. If that's a lightly edited version of the truth, all the easier!
Sometimes, the goal is straightforward, such as to convince you to oppose a politician or tolerate a war. At other times, the goal is oblique, such as to convince you to care very deeply for a minor injustice at the expense of a major one.
Deflection is a powerful weapon, as any martial artist will tell you.
Unfortunately, just like advertising, it’s not clear how effective disinformation actually is. There’s no question, for example, that the most alarmist claims about disinformation are themselves disinformation intended to scare us into accepting a counter-regime of top-down information control.4
But there’s also no question that plenty of people believe genuine political fantasies, from WMD to Russiagate, which raises the question: Why?
Disinformation exploits intelligence, not stupidity.
It’s clear when you succumb to a phishing attack. Money in your bank account goes away, which is painful and embarrassing. So, too, is succumbing to disinformation.
But how would you know? What’s the clear marker that you’ve fallen?
It seems so unlikely, right? I mean, if you’ve succumbed, then everyone you know and the thousands you’re connected to online have succumbed as well, because you all believe the same obvious truths, and honestly, how likely is it that all those intelligent, well-intended people are wrong?
Never mind that right now, you believe equally large groups of other people are wrong all the time, so such things must be common. The difference is, those other people are not very smart, are they? Nor well-intended. The enemy never is.
People think the proper defense against disinformation is good intentions, or intelligence, but it’s not. The dull and disinterested are not the marks. They’re not paying attention. Disinformation targets smart, well-intended people. Turns out, they’re actually very good at reasoning badly, probably because they’ve had more practice.
Disinformation isn’t going anywhere.
Human beings are social apes. We spontaneously form tribes the same way stray dogs spontaneously form packs. Tribes collectively define and enforce a truth. They have to. Any that didn’t would cease to cohere. What would bind them? It just so happens that tribes these days are virtual rather than local.
The definition and maintenance of a complex, collective, cohesive reality is the normal, proper function of the socially networked human brain. That’s simply what it does when you put any number of them together, and the more intelligent you are, the better you are at participating.
The true inoculation against disinformation is not intelligence but humility, which is why disinformation thrives now, in the attention economy, and why it’s not going away anytime soon.
This and other phenomena have led some public opinion researchers to conclude there is no such thing as a stable public opinion, or if there is, that it’s not what pollsters are measuring. See: The Nature and Origins of Mass Opinion by John Zaller (1992).
Lincoln famously noted that you can't actually fool all the people all the time. PR firms and political machines have taken the lesson to heart and are using the technological tools of market segmentation to customize messages based on the recipient, but as with advertising generally, it’s not clear how genuinely effective this is.
The question is whether there is such a thing as an unbiased fact-checker. I don’t mean that in the sense that we’re all cognitively biased. I mean it in the much richer epistemological sense: How do we know what is true?
A fact-checker compares a claim to some source of truth. Otherwise, they’re just making things up. So, where is this source?
I mean it. Please point it out.
Why is it half-covered by a curtain? Why are the fact-checkers having a confab in the back? If it’s simply to consult the Big Book of Truth, then why not publish it? If you give us the list of true things and false things, then we can police ourselves and save everyone’s time. If some deliberation is involved, then as a matter of truth-seeking, shouldn’t those deliberations also be public?
They are in science and law, or at least they’re supposed to be. That’s what distinguished them from religion. The body of knowledge, the law, is public and subject to scrutiny by anyone. Why is that not the case here?
Of course, there is no Big Book of Truth, and what the fact-checkers are comparing claims to, perhaps quite innocently, is the consensus among the ruling tribe, or rather the tribe that wants to rule.
If information is power, then he who controls the fact-checkers has the power.
See: https://harpers.org/archive/2021/09/bad-news-selling-the-story-of-disinformation/