There are people at the Fed "who would do Donald Trump in" says the former Brooks Brothers rioter.