[TTSS] How Smiley knows for sure that Toby is not the mole when interrogating him?

Still re-reading TTSS, and now I ask myself the following questions :

Does Smiley know for sure that Toby is not the mole when he interrogates him and forces him to reveal the location of the Lock Gardens 'safe house' (in chapter #34) ?

To this question, the answer is obviously YES. After all, Smiley pulls in Esterhase not because he needs any more information (what Esterhase tells him just confirms what he's already guessed) but because he just needs Esterhase's co-operation to set up the safe house to trap the mole. But of course this works if Smiley knows for sure Toby is not the mole.

So the question is: how Smiley knows for sure that Toby is not the mole?

I could only find three possible explanations in the novel, but I'm not sure... Are there any others?

Firstly, in chapter #23, Smiley spends the night studying and taking note on the various documents Guillam brought him. He finds a pattern with nine reports : the topical reports produced by the source Merlin coincided with period when Polyakov was in London or Esterhase had been traveling. In other words, whenever a “fresh, up-to-date” report arrives from Merlin, either Polyakov is in London or Esterhase has just travelled. Can Smiley draw any conclusion about Esterhase from this anaysis ? This is not a coincidence. Toby Esterhase is somehow part of the chain that brings this secret information into the Circus. He is not just sitting on the side. He is being used as part of the Witchcraft scheme. From this, Smiley can conclude that Toby is part of the pipeline — but he cannot really exclude that he's the mole, right ?

Secondly, when Smiley talks to Westerby (Chapter #28), he learns that the Russians knew in advance that Prideaux was on a mission to meet a general. Westerby relates that he told Esterhase this and he was at first interested, but a day later hauled him over the coals for passing along bad intelligence. Smiley realised from this that Esterhase is a puppet, not the puppet-master. In other words, Esterhase learns the true purpose of Prideaux's mission from Westerby, but then tells off Westerby for bad information, suggesting Esterhase has been told it is bad information - thus suggesting Esterhase is not the real informant, but simply one of the innocent underlings. Nevertheless, Esterhase's interview with Westerby isn't totally conclusive -- after all, if Esterhase was the mole, he would still have to protect his cover by pretending to be surprised by Westerby's revelations. By this stage, however, Smiley has seen enough of Esterhase to know that he isn't clever or subtle enough to pull this off. Smiley immediately guesses, and even hints to Westerby, that someone else must be pulling Esterhase's strings: “I expect you wondered who he’d been talking to in between,” said Smiley sympathetically."

Also, there could be another third explanation why Smiley knows Toby is not the mole prior to interrogating him : We know, in retrospect, that Smiley already has a very strong suspicion of the mole's identity. ("He knew, of course. He had always known it was Bill"). If Smiley knows deep down that Haydon is the mole, Toby is not the mole...

Your thoughts ? Anything else I've missed ?