Performance decreasing from increased rows?

That column specifically was a Roll up from a Query. Basically just a count of how many of an item there are in a Query