My husband keeps hitting on the help

Dear Counsellor,

My husband and I have been married for four years now. This year, it came to my attention that he hit on the maid and that is why she left abruptly. She did her work well, I was kind to her, and paid her on time, so I never understood how she just left. She’d worked for about four months. I found out from the lady who sent her to me that she was tired of my husband hitting on her. I spoke to my husband and he denied everything, saying that she mistook his kind nature for flirting. We got another girl, and after about three months of working, she came to me herself and asked me to tell my husband to stop knocking at her door at night. I was shocked. I asked her how long it has been going on and she said a week. I haven’t confronted my husband as yet as I do not even want to think of the next excuse he will give me. How can two girls misunderstand him? I love my husband but is this marriage really worth fighting for?