Conservatives ‘not where we need to be’ for an election ‘right now’, says party chair