Brad has the correct answer. As much as you may not like it, users are allowedto downvote you if they feel your post is not useful. It only becomes a problemif they are targeting users rather than posts.
The task then becomes determining which votes were targeting users and whichvotes were targeting posts. This is impossible to get perfect because even theperson voting may not know with certainty why they are voting a certain way.
Take the example in your question. Even if all 40 of the downvotes were by thesame person, and even if they were targeting you, that doesnt mean that they cannever downvote you again. So the point of the algorithm is to draw that linebetween when that person is "legally" and "illegally" downvoting you. Again asI have said this is imperfect, they could decide to downvote you once a week andget away with it, as long as they do not hit whatever threshold has beenprogrammed in. With enough time between votes it is impossible to know if theyare legitimate without reading the persons mind, and maybe not even then.