Comedian D.L. Hughley is the latest celebrity caught up in the Imus witch hunt. The Kings of Comedy star came under fire from Texas activists after making a joke about the women’s basketball team at Rutgers (which seems to be off limits since the ‘nappy headed ho’ comment).

While appearing on The Tonight Show with Jay Leno, D.L. called the women “some of the ugliest he’s seen in his life” and said some indeed had “nappy hair.”

After the comment aired, Pastor Kyev Tatum of Fort Worth ’s Servant House Baptist Church issued a statement, while others have vowed to protest Hughley’s Saturday show in the area.

“It’s not only that comment, he has a history of demeaning our community in such a way that it’s not funny anymore,” he told Fort Worth ’s CBS 11 News.

Hughley, never one to hold his tongue, issued a response to his detractors, via press statement:

“I believe that freedom of speech is a zero-sum proposition. Too many times I have watched clowns like these pretend to speak for the masses. I can only speak for me. Isn’t there a child you can help teach to read, a war to help stop, an unjustly accused man you can help out of jail? I will not apologize for telling a joke about the world as I see it.”

Hughley’s Tonight Show appearance: