It is now time to end the use of the hotel for housing asylum seekers," a Northamptonshire village councillor has declared.