Whenever one discusses the underlying genetic influences on human behavior, one is opening themselves up to being labelled a determinist. Part of the reason for that is a total missunderstanding of what the word “gene” actually means (not to mention the lack of understanding of the difference between a genotype and a phenotype). But, it isn’t just the public that takes issue with the word “gene”. So do some biologists, not least of which are geneticists.
John Hawks goes into the discussion here, but in the end he favors keeping the word around.
So in my view, “gene” is only problematic if we insist on confusing distinct biological processes. It is defined by transmission contrasts, in a Mendelian sense; it corresponds often (but perhaps not exclusively) with delimited DNA sequences, and it cannot by itself describe more complex functional properties such as methylation and epigenetic interactions.
“gene” means different things in these contexts, and obviously must include many distinct kinds of DNA configurations, from coding regions, to regulatory elements, to conserved noncoding segments. Since “allele” is extended even more broadly (any variant site qualifies), I don’t think “gene” is the problem here.