The Innumeracy of the Rulers

Have you ever wondered just how mathematically illiterate our rulers really are? Buckle up, Buttercup: it’s worse than you think.

Lawyers Are Not Good Thinkers?!

During a period of youthful idealism, many years ago, I fell in love with formal logic. And I mean, in love. I had that same idealism that led Leibniz to hope for “calculemus” as a path to world peace. (Yes, I knew about Gödel’s Incompleteness Theorem and what that did to Russell & Whitehead’s Principia Mathematica; I meant we should get as close as possible to that limit.) To get a formal understanding of certain ritual prayers in my religious community, I even went so far as to learn denotational semantics, hoping it would help. (It did not help. Opened a lot of other doors, but not that one.)

Now… I knew that I myself was naïve. But I had no idea just how naïve. (That’s the nature of naïveté: almost by definition we are blind to our own, almost by definition!)

I actually thought – true story – that our world leaders would (or should!) be seized by a dread sense of their deep responsibility, and therefore be deeply moved toward correct thinking. Most of them are lawyers, like it or not, so clearly lawyers must be deeply concerned with the proper application of logic, no?

Well… no. I was a theoretical physics grad student in those days, but had friends who where in various law schools in Boston. I asked a friend at Harvard Law whether lawyers studied logic. He laughed at me (though not in a mean way). He then explained, in a way seared into my brain so deeply I remember it today, 45 years later:

Once in class, I pointed out to someone a flaw in their reasoning, saying ‘That’s post hoc.’ He said ‘Thank you’, thinking it was a compliment.

So, umm… no. No, they are not concerned about needing logic to guide their thinking. They seem, if anything, more concerned with sophistry: the art of abusing logic to craft arguments that sound correct in order to hoodwink others into supporting a preferred conclusion.

People with some intellectual ability seem to me to sort into 2 tribes:

• Math/Models/Things Tribe: those who do mathematics, models, and comparison with the objective world of things,
  vs
• Words/Stories Tribe: those who get very good at words and their use to persuade, inform, or command people instead of things.

Guess which tribe thinks it has a right to rule?

Of course we want people to be able to use language properly and clearly, or else we’d be constantly ibis/redibis-ing each other until it was just sad slapstick. But I am continually amazed at people who think words are somehow the way reality works, e.g., that they can out-talk physics and make the world work differently, as if reciting a magickal spell.

And so it is today, when my face was more or less rubbed in 3 examples.

Statistical Significance in Court

It appears US Supreme Court Justice Sotomayor, for whom I generally have ‘significant’ admiration, said something that is either naïve on the surface, or in need of interpretation to find the deeper meaning:

On the surface, this is obviously wrong: of course statistical significance matters, since it’s what tells you if your observations are real or not, i.e., whether they’re an artifact of noise.

So let’s dig into this a bit. The source, other than court documents, appears to be an case study article in Significance, a journal of the Royal Statistical Society. ^[1]

(NB: The title’s phrase “Student vs Fisher” is a delicious pun. ‘Student’ was the pseudonym under which WS Gosset published in the early 20th century, since his employers at Guinness saw little point in scientific publication. Fisher, of course, is RA Fisher, one of the great founders of 20th century statistics. They had a famous feud about balanced vs random experiment design, and the practical utility of statistical significance. But Gosset was apparently a friend of both Fisher and Karl Pearson; this was a feat of notable diplomacy because both Fisher and Pearson had enormous egos and absolutely despised each other.)

The matter in the case was whether a pharmaceutical company had to disclose side effects that were not statistically significant. It makes sense not to do so, since such side effects might well have been noise, i.e., “not real” in the statistical sense of reproducibility. There could, of course, be a relation, just not one that the available datasets support adequately.

The paper chronicles a long, sad, back-and-forth discussion which mostly just confuses the issues. This is only to be expected, when both judges and lawyers have no statistical or mathematical training. They are confusing 3 entirely separate statistical domains:

1. Statistical Significance: This measures whether an observation is real, i.e., unlikely to be just an artifact of noise in the cruel wold of the Null Hypothesis that says it shouldn’t happen regularly.

   If you achieve statistical significance, then the observation is likely to repeat in a replication experiment. This is what the much-maligned and misunderstood $p$-values are all about.

2. Strength of Effect: Once you’ve achieved statistical significance, you of course want to ask whether the “real” effect is a big deal or not! It could be real, but still be too tiny to matter.

   This is the subject of a variety of statistics, like Cohen’s $d$ and Cohen’s $h$. It is also the subject of a good many pragmatic measures. For example: during my days analyzing gene expression data, we used:

   • a $p$-value for a ratio with controls to assess statistical significance of a gene’s expression going up or down, and
   • the fold induction ratio, i.e., the expression ratio with respect to controls to measure strength of effect.

   It was typical to demand $p \lt 0.05$ for significance and a fold ratio ≥ 2 (up or down) as a minimum strength of effect.

3. Causality: This is the hardest thing of all to measure! Just finding $A$ and $B$ are associated in a statistically significant and large strength of effect way is not enough.

   • $A$ might cause $B$, or
   • $B$ might cause $A$, or
   • some unmeasured quantity $C$ might cause both $A$ and $B$, or
   • you might have conditioned collecting your data on a factor $D$ which influences the normally independent distributions for $A$ and $B$, or
   • … well, all sorts of things, really!

   This is the subject of things like the Granger causality test, as well as much more elaborate causal inference methods in network systems like genes with Bayesian networks.

Both significance and strength are necessary, but not sufficient, before one can start to infer causality. (More importantly, one might ask why causality is even the standard, since significance and strength establish that you have repeatable effect strong enough to matter. It’s a deep problem to decide where to stop.)

It’s not surprising that lawyers might be confused about which of these does what, and which should be used in a given situation.

What is surprising – and disturbing – is that they seem to feel they should have the power to prescribe that! It used to be in the US we had something called the “Chevron deference”, in which courts were expected to show deference to scientific expert opinion, e.g., in regulatory agencies. In one of the more stupendously ignorant own-goals Republicans have recently scored, this has been weakened: lawyers should decide everything.

Brrr!

Bayesian Probability in Court

Bayes Rule is an absolutely essential way for reasoning about conditional probabilities (which is to say: about almost everything, when you get down to it). For example, as we previously wrote on how to interpret home COVID-19 tests, we’d like to know the probability we don’t have COVID-19 if we test negative.

• This is the Negative Predictive value, or $\Pr(\mbox{No COVID} | \mbox{Test Negative})$.
• However, all the reported data on the test is for engineering the test, like the False Positve Rate $\Pr(\mbox{Test Positive} | \mbox{No COVID})$.

The relevant relationship is Bayes’ Theorem:

\[\Pr(\mbox{No COVID} | \mbox{Test Negative}) = \frac{\Pr(\mbox{Test Negative} | \mbox{NoCOVID}) \Pr(\mbox{NoCOVID}) }{\Pr(\mbox{Test Negative})}\]

We used the data on the test box to infer that when the Weekend Editrix tested negative, we could be about 89% sure that she was in fact negative. Phew!

So it was a bit… surprising?!… to hear that an English Court of Appeals decided Bayes Rule was inadmissible, and that probability could never be used as a measure of uncertainty! ^[2]

This is not just unusual, but flouts the very foundation of the Bayesian view of probability: that probability expresses our uncertainty about events, even events which have already happened but about whose outcome we remain ignorant.

To be fair, this decision and related decisions have all been rehashed, and probability in both US and European courts is safe again.

But consider: a judge just said “well, that doesn’t apply to me” and hoped that the natural world would obey!

Fortunately, the natural world has a way of slapping back.

Counting Votes with… Fractions

Truro, Massachusetts in the US is a small town on the outer Cape, just below the tip of Cape Cod. It’s a charming place for tourists, and for all I know, also a lovely place to live. It was the subject of satire in the SF comedy Men in Black II, in which we learn that everyone working at the Post Office is either a space alien or someone who specializes in dealing with space aliens.

But keep in mind what’s behind the satire: that this is such a deeply tolerant place that even space aliens can have good jobs and be good community members. Go ahead and laugh, but remember what we value up here in New England.

Alas, that’s not a complete summary. As reported in a startling article in the Cape Cod Times ^[3], their local politicians have some mathematical literacy problems with… grade school fractions.

Back in 2009 – and no, I still haven’t forgiven them for this – they were voting on some zoning nicety whose exact details do not matter. What does matter is that:

• There were 206 votes cast at the town meeting, and 2/3 were required to pass.
• The vote was 136 for and 70 against.

The town clerk, whom I mercifully will not name, got out her calculator and noted that:

\[206 \times 0.66 = 135.96 \le 136\]

… so 136 votes was enough to pass the 2/3 threshold.

But “an anonymous caller” pointed out that using 4 decimal places, the outcome changes:

\[206 \times 0.6666 = 137.3196 \gt 136\]

… so the 136 votes were not enough to pass the 2/3 threshold.

At the risk of being pedantic, I will point out that exact rational arithmetic is possible here:

\[206 \times \frac{2}{3} = 137\frac{1}{3}\]

You can round down to the nearest integer human and get 137, but more in the spirit of the law would be to round up to 138 to ensure that at least 2/3 votes are cast in favor. Either way, the measure should fall with only 136 votes.

But… that is most definitely not what they did! Being members of the Word Tribe, mathematically phobic at best and illiterate at worst, they sought support from their peers and superiors – Word Tribe approval.

• Calling around to other towns revealed that everybody did it differently, which is maddening enough.
• But almost incomprehensibly, they decided to ask the Massachusetts Attorney General for an official legal ruling on the value of 2/3!

Which value of 2/3 is “right for you”? C’mon, this is not a place for relative truths: you can’t assert “my truth about 2/3”. (Actually, asserting “my truth” is almost always nonsense: truth is what is, independent of who realizes it. Or even whether that truth is knowable in the first place, as Gödel taught all of us.)

Sooner or later, such devotion to words and stories instead of reality leads to nonsensical acts like asking for a legal ruling on the value of 2/3.

Do not do that.

The Weekend Conclusion

The title is a joke about the Hypostasis of the Archons (or the Reality of the Rulers) from Codex II of the Nag Hammadi library.

Alas, it is a joke few will even perceive, and even fewer appreciate. Still: this bit of Gnostic writing is useful, as an example of what happens when people go very far down the rabbit hole and think their words and stories capture reality, or even control reality. It reads, frankly, like religious visions on acid. There might be something there, but it’s hard to tell past all the deflections and mysticism.

You could just say what you actually mean, even if it takes you several tries. That’s always an option, assuming you have a meaning in the first place.

As Eugene Gendlin reminded us:

What is true is already so.
Owning up to it doesn’t make it worse.
Not being open about it doesn’t make it go away.
And because it’s true, it is what is there to be interacted with.
Anything untrue isn’t there to be lived.
People can stand what is true,
for they are already enduring it.

Seek truth. Embrace reality. Escape the words of fantasy that whisper in your ear about having personal beliefs about the value of 2/3.

As George Santayana urged:

“The truth is cruel, but it can be loved, and it makes free those who have loved it.” – Little Essays (1920) “Ideal Immortality”

(Ceterum censeo, Trump incarcerandam esse.)

Addendum 2024-Aug-21: Enfield’s book

Somebody asked me if I’d seen this post made Hacker News at Y Combinator. No, I very much had not. (Gotta get the page view counter working again so I can detect reader surges!)

Notably, they cited a book by Nick Enfield on this subect. ^[4]

I haven’t read Enfield’s book, but as far as I can tell from the reviews it makes pretty much the same point, but with more data (and somewhat less bile) than I did above. Basically, language is not about precise description and conveyance of information, or at least not completely. Rather, it is mostly about social coordination, i.e., getting people to do things. (Though it sounds from the reviews as though he’s buying into the Sapir-Whorf hypothesis, which is a slightly dicey choice.)

Still, yeah: people involved in social coordination and power over others prioritize language over logic, mathematics, and science. It could be a reasonable division of labor, if the putative leaders would at least listen when we nerds tell them what’s possible or advisable.

---

Notes & References

1: ST Ziliak, “Matrixx v. Siracusano and Student v. Fisher: Statistical significance on trial”, Signficance, 2011-Aug-25. DOI: 10.1111/j.1740-9713.2011.00511.x. ↩

2: D Speigelhalter, “Court of Appeal bans Bayesian probability (and Sherlock Holmes)”, Understanding Uncertainty blog of the Statistical Laboratory at the University of Cambridge, 2013-Feb-25.

NB: Understanding Uncertainty is, unfortunately, no longer updated as of 2022-May-23, so this reference points to what amounts to an archival copy. ↩

3: MA Bragg, “Truro zoning decision hinges on single vote”, Cape Cod Times, 2009-Apr-30.

Yes, this story is from 15 years ago. No, I still have not forgiven them. Stupidity like this leaves a welt upon my soul. Those heal only slowly, with much harrumphing. Another 15 or 20 years oughta do it.↩

4: NJ Enfield, “Language Vs. Reality: Why Language Is Good for Lawyers and Bad for Scientists”, MIT Press, 2022-Mar-09. ISBN: 9780262046619. ↩