Such unannounced inspections treated staff like "naughty children", says Brian Lightman, head of the Association of School and College Leaders.
On Monday, Ofsted said there would be 40 no-warning inspections this month.
These will test the feasibility of unannounced inspections, proposed after the Trojan Horse inquiries in Birmingham schools.
There were calls for inspections to take place without any advance warning after claims some schools in the city were being taken over by hard-line Muslim groups,
'Counter-productive'
This followed concerns that some of the schools were able to conceal unacceptable behaviour when they knew Ofsted inspectors were about to call.
But head teachers' leader Mr Lightman said: "Moving to no notice for routine inspections is unnecessary and would be counter-productive. It stifles creativity and treats professionals like naughty children."
Sir Michael Wilshaw has announced a wave of no-notice inspections
"Schools currently only receive half a day's notice. This is the absolute minimum time needed so that key staff and governors are available to meet with inspectors and staff can gather the facts and figures that inspectors need during their visit," said Mr Lightman.
He accepted that where there is an "urgent cause for concern", unannounced inspections could be appropriate, but it should not become the standard procedure.
This month Ofsted plans to test the idea of no-notice inspections, with a wave of two-day unannounced visits across England.
"I'm currently giving thought to whether Ofsted should move to more routine, no-notice inspections as part of our wider education inspection reforms, which we will be consulting on later this year," said Sir Michael.
"In the meantime, under our regional structure, inspectors are well-placed to use their local knowledge and contacts to identify where these sorts of problems may be taking hold so we can respond swiftly and report publicly on what we find."
Tip-off claims
Meanwhile Ofsted is investigating claims that a number of schools in academy trusts in Norfolk were given advance warning of inspections.
The education watchdog also revealed that inspectors have returned for monitoring checks on five of the Birmingham schools placed into special measures after the Trojan Horse investigations.
The National Union of Teachers says it opposes adopting a system of routine no-notice school inspections.
"For accountability to be meaningful, there needs to be proper professional and respectful dialogue," said the union's general secretary, Christine Blower.
"The government should look to and learn from the light touch accountability systems of high-performing countries such as Finland and New Zealand which are based on trusting schools and teachers to do the best by their students, rather than the issuing of threats or penalties."