Did Basques come from the Caucasus?

This is a new theory. Do you have same references where the theory is explained more in depth? Is the argument based on linguistics or genetics?

It is no new theory: actually it's a very old one. Some linguists have seen highly unlikely simmilitudes betwee some Caucasian languages and Basque. The relation is very-very unlikely and, assuming there's something to it, it must be a Paleolithic connection: Aurignacian culture came from Hungary/Bulgaria but ultimatley from the Near East and Gravetian culture original from SW Europe, did the opposite way via the Caucasus into the Kurdistan. No further contacts are attested, and that means the least 20,000 years ago.

Basques, as all Caucasoid peoples are ultimately original from the Near East but, most likely, they arrived to their current area either with Aurignacian migration (c. 35,000 BCE). There's some speculation now about V MtDNA haplogroup, typical (but not exclussive) of Basques known to be only c. 15,000 old and known also not to be found among fosil human remains before that date (what seems logical, anyhow).

The only question that remains is what happened (if anything) in the period between Aurignacian and Magdalenian cultures. It seems pretty clear that Magdalenian is a refinement of Aurignacian tech but the transition is located in a valley of Dordogne,while the rest of the Western European region is dominated by Gravetian and Solutrean cultures succesively (c. 22,000-17,000 BCE), which don't seem to have much technological conection with Aurignaco-Magdalenian. Yet other cultural aspects such as mural art or habitation places don't seem so much disrupted, specially in the Basque region, what may point to the same people changing techs as we do now, without that meaning population displacements.

Anyhow, geographical names for north and south, seem to point that Basques originally had rivers to the north (ipar/ibar/iparralde=north, ibai=river, ibar=river banks, ibarralde=riverlands) and highlands to the south (hego/ego/hegoalde=south, igo=high, hego=bird's wing, igoalde=highlands), what fits perfectly with they living in Aquitaine-Gascony, for instance, specially if the hypothesis for the  original name for west being itsasalde=seaside (Krutwig) could be confirmed.

As far as we can say, Basques are direct descendants form Paleolithic Europeans at least as far back as Magdalenian culture (c. 17-15,000 BCE) but they could have been living here even before.