I usually do this in some violent way, making it hard to run X again when I want to. The Debian manual contains the solution to this (as well as everything in my previous post about lost root passwords):
update-rc.d -f ?dm remove ; update-rc.d ?dm stop 99 1 2 3 4 5 6 .
Where the ? needs to be replaced with any installed display managers. On my system, I only had gdm installed.