Source
r/math•
WizardMask

How do you use math to describe the intuitive notions 'almost correct' and 'most likely explanation'?

For the sake of convenience, I'm going to use a parody of science for my examples. Suppose the universe is actually a many-dimensional Platonic embodiment of a pizza bagel. It began in a mystic oven, which puffed it into being with spicy tomato sauce, extra cheese, mushrooms, onions, and bell peppers. Here on Earth, scientists observe these features very indirectly.

If someone posited all of this, except that they proposed the wrong variety of cheese, we would say their explanation is 'almost correct'. It would be more correct than the current Big Bang theory. What notion of distance are we using? From my experience, it seems like this concept (or maybe class of concepts) is used pretty consistently between different people.

Given incomplete information, if we wanted to guess the 'most likely' state of this universe from all possible models, including all possible ways of producing a pizza bagel, all possible variations in the temperature of the oven, all possible arrangements of toppings, all different varieties of cheese, etc., what would we use in place of probability? How do varying levels of precision in these models factor into this comparison?

I won't be at my computer reliably, so I might take a long time to respond to comments.

Posted Dec 23, 2013, 9:12 PM UTC

08
estherglycol•

Deciding how to express that the pizza bagel with the wrong cheese is closer to the pizza bagel with the right cheese than the big bang theory is an area of study in Machine Learning. You have to find a way of representing the objects you are talking about so that a computer can measure the distance between them. This is called feature selection. Then the computer could put the pizza bagel theories close together but far away from big bang theory.

"Most likely state of this universe"-type of questions are also studied in Machine Learning.

6
WizardMaskOP•

Is the distance metric generated by feature selection ever considered independently of computational constraints (or with oracle machines)?

1
estherglycol•

Yes. Usually. I learned Machine Learning in a very theoretical setting and I don't remembered hearing anything about computational constraints. Here is the corresponding wikipedia page about "the distance metric".

http://en.wikipedia.org/wiki/Kernel_trick

Instead of a distance metric they use something resembling an inner product. By "Mapping observations into an inner product space..." they mean exactly feature selection.

2
WizardMaskOP•

This looks like (an introduction to) what I'm looking for. At first glance, it looks like representations aren't limited to finite-dimensional vectors except as a matter of convenience within a particular context. Thanks!

1
tacticsAlgebra•

There's no math in this post.

3
jellyman93•

"How do you use math to describe..."??

3
sciencifying•

A theory can be considered "most likely" when it is the simplest to fit past observations. The concept of simplest theory can be formalized, such as in Solomonoff's (impractical) theory of inductive inference.

When you already have a prior probability assigned to every possible hypothesis, you can also use bayesian probability theory to update your beliefs.

2
[deleted]•

[deleted]

1
ghyspran•

It's wrong to think of a scientific theory/hypothesis as being "correct" or "not correct". Scientific hypotheses make predictions, and what matters is whether these predictions are accurate. As far as I know, there's no "mathematical" quantification of this property for a hypothesis, we simply note where a hypothesis makes accurate predictions (the accuracy of the prediction often being probabilistic and quantifiable), and note where its predictions are innaccurate or unlikely.

As far as determining what is "most likely", we do use probability. Depending on your goals/assumptions, you would use one of a few different methods for assigning confidence probabilities to hypotheses (e.g., Bayesian inference).

There's a lot of philosophical information on how we know what we know in regards to scientific thought. If you want to read a bit more on this stuff, I might suggest Hempel's Philosophy of Natural Science. Amazon has older printings for <$2.

1