Should I stay or walk away?

We've been married 9 years (no children) and have been very happy until now. My husband and I are from different cultures and in his culture it is custom that his parents live with him in their old age (didn't know this before). Hence the problem. They are both healthy and financially stable. They recently stayed with us for 6 months. During this time I felt completely isolated. They don't speak English and their customs are different. They have no sense of privacy and feel like it is an honor for US to have them in our house. I have told my husband that I just can't live with them. I feel like I have to be a completely different person when they are around. I can't be my goofy self, can't drink in front of them either. Not that I am an alcoholic but I do enjoy a glass of wine occasionally. I also feel like my house is not the peaceful place it used to be when I come home from a hard day at work. So my dilemma is that he feels I am being selfish and should sacrifice something for our marriage. He is not willing to compromise in any way. I am stuck. I love him but at the same time I can't give up who I am. I feel like if I say yes, I will be unhappy and if I say no, he will be miserable. There is no way I would deprive him from having his parents live with him if it makes him happy. I feel like I have to walk away. What are your opinions and what would you do in my situation? I am pretty miserable and devastated right now. How can this work?