Cabinet ministers are said to have become increasingly frustrated by the Health Secretary's outspoken pronouncements.