Andrew Kohut explains in the NY Times:
"Poorer, less well-educated white people refuse surveys more often than affluent, better-educated whites. Polls generally adjust their samples for this tendency. But here’s the problem: these whites who do not respond to surveys tend to have more unfavorable views of blacks than respondents who do the interviews."
And Hillary Clinton won among those folks in NH.
So it's not that some people harboring perhaps-racist feelings lie to pollsters.
It's that those individuals won't talk to pollsters at all--but they'll vote.