Alas, I came across a better way to take a bunch of rows from views, group them by a particular field, and provide a count for each.
Here is a basic example for Drupal 6 (probably very similar for Drupal 7).
Basic Settings
Style: HTML List
Fields
Node id (exclude from display)
Node title
Arguments
Node title
Recent comments