Dropping in the occasion 'cuz' or 'ya think' isn't going to pose a problem, but I find reading an entire story with what I've heard one person call "authentic voice" pretty frustrating.

I'm glad it's not just me, then. I have trouble with phonetic spelling in general because I'm not a native speaker. My default phonetic spelling is Italian, and it looks totally different. I have no trouble with ordinary spelling because I associate the sound with the word - when I read something, I tend to hear the words as if the person who plays the character had spoken them - but whenever anyone tries to emulate dialect that way I am immediately at sea.

It's difficult deciding where you should draw the line. I'm with you, I'd keep it simple and just stick to easily recognisable words, that way it doesn't stop you in your tracks.