Here in NZ, prostitution, brothel keeping etc are legal. The tax department is getting it's due and we read stories about perfectly normal women choosing to be prostitutes as a career choice. We aren't at the stage (yet) of insisting that prostitution is an alternative to welfare, so you can turn down those sort of jobs and still continue to get state assistance if you need it.
So, coming from a position where it is a legally accepted part of our society, I'm still siding with your Mother. It is an ugly business. People can and will judge you for being a part of it, even indirectly. I know a woman who works as a receptionist in a brothel, her entire family is affected. e.g her kid's friends weren't allowed over to her house, or her parents house, in case they came in contact with some work related issue. The Grandparents were even refused when they offered to ferry kids to sports, friends of their other Grandchildren (i.e nieces and nephews of the receptionist) This attitude isn't uncommon.
NZ isn't a religious country either and my impressions of USA suggest that there is a large number of conservative type Christians there who will add their judgments, and there are enough of them to make their feelings known. It doesn't matter if you agree with them, and their decision making process or not, people will make a judgment. These are the people you will interact with everyday, at work, at school and at play. There is a reason criminals flock together...most people don't want anything to do with them. Do you want your interactions limited in this way?
Take a step back and consider all those people this will affect, your parents, siblings and unborn children. Remember also, that even if you later divorce this man, you will still be considered associated with him.
You call it the family business. You will be his family, so you will be part of the business. Have you thought about that? If he's not connected to it, will he have no further contact with his family? Or does he condone the family business? You can't easily separate them family and family business
However, personally, my big red flag is that he is willing to associate with people who do illegal things. That means he is willing to lie. If he is willing to lie to the Government and general society, why would he not lie to you OP? On what basis do you place your trust?