Hexadecimal value 0x13, is an invalid character

Using the SharePoint 2007 Lists web service, the following error was returned when the GetListItems method was returning XML containing the data.

 hexadecimal value 0x13, is an invalid character. Line XXX, position XXX.

Looking into the error, it appeared a bad character was being returned. I was eventually able to trace the exact record causing the error and it was due to double and triple exclamation marks (which appear to have been copied from Microsoft Word into the text editor).