In a vertical closed loop system will there be a PVC pipe out of the ground like you see with a regular water well?

thank you in advance

No.

Hopefully not only heat fused PE pipe should be used. In europe they have used PVC in the past, but PE is much better, longer lasting, more durable

I'm thinking he was referring to the well casing, not the actual heat transfer loop material.

Answer remains "no"