The recent base-rate thread highlighted a significant problem with making one-to-one, working-to-outcome assessments for practical goals, goals aiming to change ones material circumstances.
It isn't that one will always be wrong when making working-to-outcome assessments; it's that one could be wrong some of the time, and there's no way to know with certainty which judgments were accurate and which were faulty. For an aspiring magician that cares about truth, and wants to progress as efficiently as they can, that's a problem. In a nutshell, if one has unknowingly made a faulty judgment, that's very likely to detract from their understanding of how magic works (for them), which can further be expected to hinder progress, at least in the short term.
My experience of discussing the base-rate issue in occult studies forums has been somewhat consistent: some deny the base-rate problem while others who accept it seek to mitigate it with extenuating circumstances, like the idea that magical results should come about in uncanny ways. Still others tacitly accept that its a problem but choose to ignore it, which I think is significantly different from denying that it's a problem. The first two responses, in my opinion, are likely to compound the problem over time, and I speculate that it leads to some of the more inane discussion and outlandish claims on the occult studies internet. No forum, in my experience, is immune to those by the way.
That last option though, accepting but ignoring, may counter-intuitively be the best option for some people. If one's proclivities are somewhat contrary to a planned and structured approach, but commits to honoring the truth and abhors the specter of self-deception, and if their magic practice is in fact the centerpiece of their life, then I think some 20 years of practice may well impart the truths that really matter, without addressing the base-rate problem head on.
For those whose proclivities are more keen on a planned and structured approach, the following method is offered. Being that sort myself, it's tempting to believe that it's the better of the two approaches (or other approaches not listed), and I would have said so in the past. But I no longer think that way about it. I've come to believe that if a person doesn't like doing a given thing (in the occult studies domain), they're not likely to do it well enough to make to work for them anyway. All else being equal, people are probably best off going with the things that they really enjoy doing.
To keep the post lengths more manageable, the method itself will follow in another post.