Who Wants Brownie Points?

During a heated email insult-a-thon, after my mentioning loss of brownie points, it was brought to my attention that one cannot lose brownie points, only win them, as do the young Girl Scout Brownies, from whom this expression is derived. I, being Canadian, and not having seen let alone tasted a girl scout cookie until about three years ago, do not know much about Girl Scout rules (I really don't know anything about the Canadian equivalent Girl Guide rules, either). Is there anyone who can help settle the issue of whether it is indeed possible to lose brownie points? You will get chocolate chip cookie points from me, regardless of the truth!