As Asher is becoming more involved in his art and less involved in his religion, his community, because they all think much like his father in that drawing is foolishness, is beginning a process of shunning him. They know that he is studying with Jacob Kahn (a man who himself became separated from the Jewish community through art) and they know that the Rebbe has given him a special grant to do so and so they resent this. Thus, they also resent Asher. And this is sure to have long-lasting effects on Asher, effects that have already began to show through.
This process of "shunning" him has began with the part of his community that he should be least concerned with, his classmates. They are the least learned in the community and so their opinions should not matter in the least bit to Asher. However, it is being seen that they do. Their taunts such as a poem that they wrote that reads "Asher Lev won't go to hev; to hell he'll go far down below," are beginning to effect Asher which is seen in the way that he is painting them. He paints terrible things happening to these boys, and although Jacob Kahn may think these feelings are benefiting his work, they are certainly not benefiting his emotional composure.
Also contributing to shunning Asher are the two people who should definitely hot be doing it but who have been doing it all along, his parents. It was their neglect of him that essentially led him back into drawing and now they (his father in particular) are constantly ridiculing him because of it. For instance, his father says "I can not reconcile with this," and also goes as far as to say that Asher is disrespecting him by continuing to draw. Again, this verbal abuse will do nothing but harm Asher, and Aryeh coldly does not seem to care much about that.
Thus, with his peers and parents going against him, the only person who is standing by Asher and his art is the Rebbe. This works out for Asher since he is the only one who can make the rest of the community conform to Asher's drawing. However, Asher must be careful not to upset the Rebbe through his work because then he will undoubtedly not have a place within the community.