Anyone else w/big dog dealing with paranoid pedestrians?

When I take my English Mastiff Monty for walks and he does his business I will literally be going to pull a bag from his bag holder attached to his collar and kneeling next to the poop and people are stopping to make sure I do. I have always been responsible so people in my immediate neighborhood dont bother me because they know us but elsewhere in the city they seem so paranoid. Its like they are looking for me to walk away from it so they can say something. Its either I deal with them being paranoid I am going to litter by poop or I get a comment about how big dogs dont belong in the city. I am getting less and less polite and not liking myself very much afterwards. Someone asked me WHILE I was kneeling down to do it "planning on picking that up?" and I ended up asking them if they wanted to do it instead because it angered me. How do I respond to these characters or how do YOU respond? I used to love walking him and still do but this part is making it less enjoyable for sure!