Have you ever had to deal with a troll or another troublesome blog visitor? In the real world, we are often advised to deal with bullies as well as small annoyances by just ignoring them and therefore taking away their power. This would seem to be particularly relevant online when a rogue commenter could lose interest quickly and never return. Even if they do stick around, what's the worst they can do.
Unfortunately, this has some drawbacks. In the real world, ignoring really isn't that great of an idea if it could escalate, and it could easily escalate online. In the worst case scenario, you could end up with a cyber stalker on your hands. Deleting or flagging the comment might just give them ammo and cause the problem to quickly escalate.
In my opinion, the best solution is one I have not implemented, mostly because I haven't had the need to: a commenting policy. This way, if you do find yourself needing to delete a comment or block a user, you can direct them to the policy to show them why you are doing it. Consistency will weed out some of the problems and cooperative readers will follow the policy, which will get rid of problems that are only caused by misunderstandings.