Gene's personal life was marked by a series of significant relationships and marriages. In 1934, he tied the knot with Ethel McGuire, a switchboard operator at the Dixie Hotel, where Gene was residing while performing with the pit band of the musical "Girl Crazy". This union lasted until 1942, when the couple decided to go their separate ways. However, their love story didn't end there, as they remarried in 1946 and remained together until Ethel's passing in 1955.
Following Ethel's death, Gene married Patty Bowler in 1959, and the couple went on to adopt two children, Mary Grace and Michael, whom Gene affectionately nicknamed "BG". Unfortunately, this marriage also came to an end, with the couple divorcing in 1968.