A "damn" (a damnation/ condemnation) directed at something is not a positive thing, but at least it means the person "giving a damn" cares about the subject in one way or another.
The person's level of emotional investment may be quite minimal, but if someone "gives a damn," they at least care something about the outcome of an issue. This is colloquially used in both positive and negative senses.
Example 1: "Tom gives a damn whether or not the Red Sox win the World Series this year, but it's not as if he's betting money on the results. He's just a Red Sox fan."
Example 2: "Troy doesn't give a damn whether his son has good grades or not. What an irresponsible parent he is."