AI for playing 20 Questions for University Project (Thought you guys might enjoy seeing it learn)
questions20.herokuapp.com/Posted Jun 7, 2017, 12:43 PM UTC
37
questions20.herokuapp.com
Some overlap in the questions. You asked if it was a predator and if it was a carnivore. While they are two different things, their sets have a lot of overlap.
1
Thanks, yes, the bot may not have had a better question to ask at that point (for the guesses it had). It chooses the questions from its database to maximize the split of answers between yes, no and unsure. They're also not hand chosen questions but submitted online by users (and a few we've added to help it along).
We hope it can work out what questions to ask.
1
I liked the concept. Would love to play this when I'm bored!
1
Thanks! Glad to hear it
1
First time posting so let me know if I need to change anything about this. A team mate and I are working on a bot that plays 20 Questions. We're looking for more data and thought you might enjoy it. If you have any feedback or questions feel free to PM me or post it here. We'll make the GitHub public once we have submitted the code (it is part of an assignment we are working on). Thanks
Welcome and thanks for sharing!
I tried 3 times (Cheetah, Platypus, Panda) and it got the Cheetah and Platypus, but Panda was only the 9th guess. Granted, I put "unsure" on 3 answers.
I noticed that you always ask the same 20 questions, and I wonder how you came up with them, and also where you got your data. You could probably do better with more questions that get selected based on discriminating power (entropy) for the most likely options.
BTW, I thought it was really awesome to be able to see how the guesses and percentages changed, and how some people apparently were thinking of unicorns and pokémon. Sometimes it's a little weird though, like when Lion was the top guess above Cheetah, and then I said the animal had a spine and they switched places. I guess this must mean someone who played didn't know lions had spines. If you want to get really sophisticated, you could try to take people making mistakes into account explicitly, but it's probably overkill for your project.
Anyway, it looks cool! I hope you get a good grade! Also, if you're interested in AI and 20 questions, you might also like this famous Alan Newell paper from 1973: You can't play 20 questions with nature and win.
Thanks u/CyberByte. So, the questions are chosen based on entropy (using an algorithm based on decision trees, entropy weighted by probability, but we cache parts of the tree that we have already used and recompute them every ~hour so that the tree updates).
The reason that animals with the same expected answer are affected differently by a question's response may be noise in the responses (as you guessed) but it may also be due to Laplace Smoothing which I put it to make sure that a few wrong answers didn't completely mislead the bot. We considered trying to use some form of clustering to isolate people who answered questions incorrectly or even groups of people who hold similar misconceptions but we decided it was outside the scope of the project.
Thanks for the paper, I'm really enjoying the opening. Newell has such a unique perspective on the work done in Psychology and AI. It's comforting to see that he didn't feel we were making enough progress at the time.
P.S. I think I may have found the issue causing the question choice fixedness. Thank you
How'd the submission go?
Thanks, we got to roughly 30% correctly predicted and did fairly well for the assignment. I personally had hoped for higher than 30% but this may have been unreasonable considering how many different animals there are and the wide variety of beliefs about them.
We had some issues with vulgur inputs (the poor thing got stuck predicting gentials for a while) but otherwise was alright. It also needed to be able to know that after a certain level of certainty it should focus on learning about the animals rather than about the solution to the game is was playing. This extra data might have allowed it to use the submitted questions more (which it did fairly badly with). It seems that even after it was 0.95 certain about the animal, it would ask questions that we would consider to be irrelevant to try to force the last few points of certainty out.
Thanks for the interest. I am considering a rewrite of the project focusing on performance and fixing the above issues but can't really justify it as it doesn't really provide a service to anyone. I'll try to upload the performance graphs soon, in the submission rush I didn't actually get a copy from my team mate.