There's certainly an advantage to high toughness vs. high defensive skill when it comes to impact hits, but it's not as pronounced as you'd think.

Let's use an example comparing Antonian Horsemen charging two units. Unit A has a 3/1 defense and unit B has a 1/3 defense. Assuming average dire rolls and all command card play being equal:

Average damage inflicted on unit A - 4.1666

Average damage inflicted on unit B - 3.8333

So on average it's a difference of a 3rd of a point of damage.

Now let's replace the Antonians with Knights of any variety. Same defensive stats for Units A and B with average die rolls and command card play being equal:

Average damage inflicted on unit A - 5

Average damage inflicted on unit B - 5.1

Whoa! Pretty much even. To be fair, there are more units with impact hits having an offensive power of 6 or less than those with impact hits and an offensive power greater than 6. So there is a disparity, but it's pretty small.

Now the difference becomes more pronounced when you get into units with multiple impact hits. Namely Chariots, Triceratops Herds, and Elephants (all 3 varieties). So there's currently 5 units in the game that punish low defensive toughness. Note that the cheapest and most readily available of those units (they're Core for cryin' out loud!) are the Chariots, which live in a low defensive toughness faction