The mistake is accepting uncritically that expected value is the best metric to optimise. Nobody ever proved that expected value is a strategically superior metric. In fact it would be quite hard to prove that since it is not true. It leaves people vulnerable to making very stupid decisions as illustrated in Pascals Mugging.

Optimium strategy involves at a minimum considering your available opportunities and available resources. Opportunity alone is not enough.

That can't be reasonably estimated though. Putting aside the fact that we can't really assert the relation you posit, there is also a finite probability that the mugger is some sort of illuminati member with the ability to create an arbitrary amount of money. Ie, there is some tiny-but-positive probability that he can create an arbitrary amount of money.

At that point, the expected return can be made large compared to the probability that the mugger is lying.

* Helping a googleplex of people immediately vs over a period of time are two different complexities of action.

* Recall that hypotheses are selected from an ambient pool of possibilities. Then we might imagine that some hypotheses dominate others, so that regardless of how much evidence is offered, we always insist that the evidence supports a simpler alternative. To wit:

"Well, if I'm not a Matrix Lord, then how do you explain my amazing powers?" asks the Mugger.

"Street magic," you say. "Very impressive sleight of hand. Perhaps some smoke, mirrors, lasers, assistants."

* A Matrix Lord asking $5 of a person on the street in order to commit miracles is inherently irrational. If they just wanted $5, or wanted to deprive the person of $5, or wanted to humiliate and embarrass the person, or force them to accept certain philosophical truths, then those all could be achieved via Matrix Lordery. Therefore the Lord in this story is being a pointless dick, and it's silly to expect rational arguments to be part of the conversation. To wit:

"Just give yourself $5. Give yourself any reward you like, for helping people; it's not my place to set or fulfill the price of such powerful entities, is it?" you ask.

"But...but don't you want the feeling of doing good?" asks the Mugger.

"Not really, no," you reply. "I have investments and equity already, and those dollars already have ripples that affect people far beyond my direct control. I don't feel much of anything about those investments. And it would be irrational for me to value a $5 investment more than $5. Really, if you can do all of this good, then you should turn yourself into an exchange-traded fund, and let people buy your time to do good in the world," you muse.

"But...but this offer is for you, and you alone," the Mugger insists.

"Okay, but why me? Let's talk about the Self-Sampling Assumption!" you say. The Mugger groans.

The point is that, at the time when the Mugger declares himself to be an Operator from the Seventh Dimension who can offer large rewards etc, there is no evidence to suggest he's saying the truth. No evidence at all. Accordingly, the probability that he's telling the truth must be zero. Where does a non-zero probability value come from?

Are you then saying that the probability of any reward should never be placed to zero because that would not maximise rewards?

For example, at this point in time I believe that the probability that I can fly if I flap my arms up and down is zero. I have no evidence that this is possible and I understand enough of the relevant physics to know that this is not just improbable, it is impossible.

However, if tomorrow I flapped my arms and found that I could fly, there would be nothing stopping me from re-evaluating my belief and assigning a higher probability to the chance that I can fly if I flap my arms.

But I think the problems begin with the misguided ambition to be able to predict the future even when there is no evidence to support any prediction. You can't know what you can't know. You can assign any probability you like to what you can't know, but even if you end up assigning the right probability that will be the result of blind chance, not the result of correct reasoning.

Anyway this is why I prefer logical inference to probabilistic inference. I understand that I'm in a minority on this, but for me it makes a lot more sense to maintain a state of provisional belief with an absolute value (in {0,1}), provisional in the sense that new evidence can always change your belief, than to live in a perpetual state of uncertainty which never resolves itself no matter how much evidence you see, because there is always a chance that you're wrong. There always _is_ a chance that you're wrong but it just seems cumbersome to have to maintain a ledger of competing probabilities for everything that has happened, and everything that hasn't yet happened, just on the off chance that anything can happen, including mutually exclusive events.

In principle, anything might happen. In practice, not everything will. There must be a sensible way to figure out what we need to prepare for and what we can safely ignore. And the whole Pascal's Mugger paradox, while it's meant to attack Pascal's Wager's logic, ends up for me as an illustration of why proabilistic inference is deeply borked.