Football Association chairman Greg Clarke says former England boss should have been sacked “three or four years ago".
Sampson's three-and-a-half year tenure as England head coach was brought to an abrupt end last month after details of "inappropriate and unacceptable behaviour" during his time as Bristol Academy boss came to light.
Sky sources reported he lost his job because of a relationship with a player.
The FA's original investigation in 2014 concluded that Sampson needed mentoring and tutoring about the boundaries that needed to be maintained between coaches and players, adding he "did not pose a risk working in the game".
Clarke says if the FA's top brass been made aware of the details earlier, Sampson would not have been allowed to continue in his role.
He told The Telegraph: "When you get to the point where the new chairman and the new chief executive find out something that wasn't shared with the board a long time ago [details of Sampson's time in Bristol], do you think - that's a shame, we'd have done something if we'd known, or do you make a decision?
"Martin [Glenn, the chief executive] said - 'Look, I found this out yesterday.' I said - 'Right, what do you think?' He told me, I agreed with him and we had a board conference call. We sent out some papers, we asked some questions about legalities, facts, what happened when. And we made a decision. Now, that's the sort of decision that should have been made three or four years ago, but you can't use that as an excuse to duck the decision today."
Sampson has also been at the centre of allegations of bullying and racism made by Chelsea Ladies and England forward Eni Aluko. He denied the claims, and was cleared by an internal FA review and an independent investigation led by barrister Katharine Newton QC.
Newton has since resumed her investigation and Clarke says the FA are committed to letting the process come to a natural conclusion.
He said: "It's a complicated one. I'm pleased she raised the issues. If there are issues, we want people to feel safe and raise them. But there are two sets of rights. There are the rights of the people who feel aggrieved, and the rights of the people who've been accused. We had an internal investigation, which came to a very simple conclusion. There had been some daft things said, but none of them posed systemic evidence of either racism or bullying.
"Because of the gravity of the accusations we felt the need to get an independent barrister [Katharine Newton] to conduct independent reviews. I read the report and came to the same conclusion. Some people decided they weren't going to be interviewed. They've now come forward and said 'we would like to speak'. Our barrister is now collecting more evidence. If the conclusions stay the same - fine. If they differ - fine. All we want is to get to the bottom of it."