Per Numbers 13:33, we know that the "Nephilim" which are in reality called "giants" are merely the children of Anak.  The other verse that most use to attribute to the "Nephilim" was Genesis 6:4, which is in reality referring to the "sons of God".