What type of caulk should be used to either glue foam board to a wall surface, or to join together the tongue and groove boards?  Is regular latex caulk enough?

The main thing you've got to make sure is the caulking is NOT petroleum based. It'll melt the foam board.
Pl 300 foam board adhesive is what I use. This stuff sticks like crazy and can be found at any regular hardware store and comes in the same container as caulking does.

Purpose-specific foam-board adhesives are available from a number of vendors. Standard latex may have insufficient adhesion for many purposes, but doesn't usually have solvents that eat foam (the way most standard construction adhesives do.) For sealing seams after-the fact most duct-mastic is fine.

On my house I was actually considering connecting the boards with the "window/door" low expansion great stuff expanding foam. Any thoughts on this?

Regarding the low expansion great stuff, not such a good idea for it still expands and you'd still have to make sure it's in contact everywhere and brace/hold it up. The foam board adhesif is the best cost efficient and end result pruduct in my past experiences.

I wasn't going to space the boards, I was just going to squirt a thin layer and set the other board right on top. Basically using the foam the same way you would use caulk or adhesive.