The spec you're looking for is most commonly listed as backplane, fabric, switching capacity, or total bandwidth (at least those are the ones I've seen, there are probably others). It's a rating of how much data the switch can actually move around internally.
I'll generally look for something that has the requisite number of ports for a job (and number of PoE ports), then management features (if required), and then consider budget... that will generally net two or three options, and the deciding factor then becomes backplane - the bigger the better.
Note that some switches list it in (mega/giga)bytes per second, others list it by packets per second, so make sure when looking at specs, that you're comparing apples to apples.
Example from a local retailer's website: Cisco SG300-28PP (24 GbE PoE ports, two GbE uplink ports, two GbE combo ports, Layer 3 management, retail $720) lists "Switching capacity: 56 Gbps" while the SF200-24P (24 x 10/100 PoE ports, plus two GbE combo uplink ports, Layer 2 management, retail $325) lists "Switching capacity : 6.55 Mpps". Sounds like a LOT LOT less, until you factor in that they're measuring 64-byte packets; Cisco lists the 200 as 8.8 Gbps. (Note: their SG200-28xx series all list 52 Gbps, so the 28-port gigabit models are pretty close in performance whether they do L2 or L3.)
Someone will probably bring up power budget as a spec to look for as well... for me, that's not a concern, as we're not installing any cameras that pull over 5-6W... I think the P3384-VEs are rated up to 12W with the heater, but we don't enable that).
As with everything else, of course, consider the specs that suit your own needs.