Let me simplify this for you, your account is your responsibility, if your brother had JUST said that, and you really had to leave again, you would have maybe, I don't know, locked the computer? Anyway regardless if it was you or your brother, your account is banned.
Also, breaking one fence and getting across there is griefing. You didnt make that town, you had no business breaking or building in it