Stuart Barnes believes Stuart Lancaster was asked to stand down, despite the former England coach and the RFU claiming it was a mutual decision.
The 46-year-old left his role following England's disappointing World Cup campaign when they became the first hosts of the tournament to be eliminated at the pool stage.
Former England fly-half Barnes has rubbished the idea Lancaster moved on as a result of a joint decision and instead thinks RFU chief executive Ian Ritchie asked the coach to leave his position.
"Ian Ritchie is saying the two of them came to a joint decision - that is absolutely inconceivable," Barnes told Sky Sports News HQ.
"They couldn't have said at the same moment 'I've got a great idea, Ian' and 'I've got an idea too, Stuart' and say 'let's resign'.
"It's going to be Ian Ritchie, isn't it? Stuart Lancaster has to fight his corner, even if he thinks it is a hopeless case. Ian Ritchie is the man, he is the CEO, he has to say 'look Stuart, when all is said and done, you are going to have to go'."
Ritchie and RFU chairman Bill Beaumont faced the media at Twickenham on Wednesday where they outlined plans to recruit "the right person" to take England forward.
Barnes stopped short of calling Ritchie's position into question, however, and instead looked to those around him in the organisation to take more responsibility.
"Ian Ritchie is a very articulate and intelligent man," he said. "He was noticeably flustered when the question about his position came up in the press conference. He said he would leave that to the board and chairman.
"Bill Beaumont said he has been given a vote of confidence so from Ian Ritchie's position, he is not going to walk.
"I think he has actually done quite a lot of good things. I will say this in Ian Ritchie's defence because he is coming in for a lot of flak, when he moved into this position he was new to rugby, he came from Wimbledon.
"Had he said unilaterally 'I want this man', people like myself would have asked how he was qualified to make that call. He had a board that helped to make that decision. Among others, Rob Andrew [RFU professional rugby director] was on that.
"These are the people who really put it to Ian Ritchie that Lancaster was the man, so Rob Andrew's position is scarred, shall we say, by the Martin Johnson episode and then Stuart Lancaster, and Ian McGeechan, who played an important part, I think, in convincing Ian Ritchie also. It is a bit of a black box there."