Usually, when people talk crap about you. especially when those things are far from true, they are bothered by the fact that you are living the life they have always wanted - that you were destined for the greatness they can never acquire.
To all those people, sharing the same problem, be brave enough to say:IDGAF.