I've read that housewrap is not necessary with ICFs because the EPS insulation acts as a WRB, but do you put housewrap around each window? Or do you just apply the flashing tape to the window buck and have that go over the insulation a few inches? Thanks.

No housewrap. Just tape.

http://www.constructionspecifier.com/wrb/
you might take a look at this article, it should assist with your question.

I put housewrap at least 8-12 inches to the sides and bottom at each window as a buffer against any moisture that might want to travel along the horizontal seams. I ran the houswrap to the soffit at the top of each window to protect against anything coming down the vertical seams.

peel and stick flashing. No house wrap or vapor barrier.

If you folks don't use house wrap, how do you prevent moisture from getting into a block joint above the window (or other opening) and running in behind the flashing?

It just doesn't, never heard of it happening. Isn't house wrap stapled on to wood framing? I trust ICF bed joints compressed by concrete more than I would house wrap full of holes.