The foam can be just about any color, depends on the manufacturer. Drywall mud will usually be a creamy white color and will be chalk like in texture. foam will be lighter and when you tear out a chunk, you will see the cellular structure to it (small air pockets). It wouldn't be surprising that a bit of drywall mud got into the window opening. It certainly should not be filled with drywall compound. Chip it out however you can without damaging the window. When you replace it with foam, make sure you use a low expanding foam for windows and doors.
Good Luck Eric