By Lawrence White and Andrew MacAskill

LONDON, May 5 (Reuters) - The former CEO of a software company suing RBS for allegedly conspiring to push the business into administration faced the dismissal of his case on Friday, after a judge ruled it was unlikely to succeed based on an interim hearing in March.

Scottish businessman Neil Mitchell was seeking as much as 128 million pounds ($184 million) in damages on claims that Royal Bank of Scotland (RBS) conspired in 2007 with co-defendants KPMG and U.S.-based fund Cerberus Capital Management to sell assets of his company, Torex Retail plc, for below their value.

A judge said the case should not proceed on the grounds that Mitchell's claims did not have a real prospect of being proven, according to a court ruling seen by Reuters on Friday.

High court Judge Malcolm Davis-White said in his ruling that some of the claims were "confused, imprecise and unclear to such an extent that I consider it vexatious" and said he had "serious doubts as to the truth" of some of his testimony.

The ruling followed an interim hearing in March that sought to establish whether the case was likely to succeed at trial and should be allowed to continue.

Mitchell told Reuters he planned to appeal the judge's decision and that he was determined to go to trial.

"This interim hearing was unfairly conducted as a mini-trial without the benefits of disclosure, my eight witnesses, whistleblowers or me giving testimony," Mitchell said.

Mitchell is one of the most prominent and vocal critics of the bank, and formerly led a large group of customers alleging they were mistreated by RBS's business restructuring division during and after the 2008 financial crisis.

RBS welcomed the ruling. "We have consistently maintained, over a number of years, that Mr Mitchell's claims are without merit," the bank said in a statement. "We are pleased that the court has reached that same view and has rejected the claims in their entirety."

Mitchell alleges RBS pushed Torex Retail into administration - a form of protection from creditors under which external managers from KPMG were brought in - in order to force its sale to Cerberus and so rid its books of a bad loan it was owed by Torex, documents filed at Britain's High Court of Justice show.

Mitchell said that he had sold assets including a house, cars and investments trusts for his children to fund the 10-year pursuit of his claim against the bank.

The state-backed bank has admitted some wrongdoing over its handling of small businesses, but has said there was no evidence it pushed companies into bankruptcy.

The lender has set aside 400 million pounds ($518 million) in a compensation scheme to reimburse some customers who were affected.

($1 = 0.7720 pounds)

(Reporting by Lawrence White and Andrew MacAskill; editing by Susan Thoams) ((Andrew.MacAskill@thomsonreuters.com; +442075421726; Reuters Messaging: andrew.macaskill@thomsonreuters.com))