last time i worked on punkbuster it used a few methods of detection, one being patterns/crc's of memory, the other being detectin hooks or code modification on its .text segment and it used some generic methods like window names etc... it also used an api to take screenshots of the client and send it to the server.
The ways i used to cirvumvent it was to code my hooks usin hardware breakpoints (vectored exception handlers) to trigger an exception at certain points i want to intercept. I also hooked the GetForegroundWindow api's to BitBlt my own "legit" screenshot so it would upload my "non-hacked" screenshot every xx seconds.
When i hacked it though the game was a quake 3 based engine so i figured a way to manipulate the virtual machine files which wasnt covered by punkbusters protection since vm's wasnt considered a major threat nor a executable (unofrtunately for them this was there flaw

)