Were there any famous non jedi/sith force users that turned down the offer?

So I'm making a character that grew up on a city planet being force sensitive and survived through stealing and found a holocron that helped him get better at using his abilities. When he got his own ship, he became a smuggler and also explored jedi/sith temples, looking for more knowledge. He thought the jedi were too good for his lifestyle, and the sith too violent, so he learned whatever he found without a master or colleague saying "that information is forbidden to know by the order!"

And I was curious if there was any other character who had a similar story.