[MacPorts] #69781: portindex: multi-threading appears bottlenecked for 10.13, 10.12, and perhaps earlier

[MacPorts]

#69781: portindex: multi-threading appears bottlenecked for 10.13, 10.12, and
perhaps earlier
------------------------+--------------------
  Reporter:  mascguy    |      Owner:  (none)
      Type:  defect     |     Status:  new
  Priority:  Normal     |  Milestone:
 Component:  base       |    Version:  2.9.3
Resolution:             |   Keywords:
      Port:  portindex  |
------------------------+--------------------
Description changed by mascguy:

Old description:

> Introduced as part of 2.9.0, `portindex` is now multi-threaded. And it
> works beautifully, providing near-linear speedup based on the number of
> CPU cores available.
>
> Testing on my 24-core MacPro2012, things work perfectly for 10.15 and
> 10.14, with shockingly-fast indexing. And the `portindex` process
> consumes nearly all available CPU cores, as expected. (For my system, at
> least 2100%. Note quite the 2400% of all cores, but close enough.)
>
> However, for 10.13 and 10.12, multi-threading appears to be significantly
> bottlenecked: On my system, the `portindex` process barely consumes two
> CPU cores. (And the performance matches the CPU usage, as it's nearly an
> order-of-magnitude slower than it is on 10.15 and 10.14.)
>
> Inspecting the `portindex` process itself, the number of threads appears
> to be correct in all cases. (It's always 27, even for 10.13 and 10.12.)
> So it doesn't appear to be an issue with CPU core count detection. But
> rather, something is severely slowing things down.
>
> Of note, testing was done on two different MacPorts releases - 2.9.1 and
> 2.9.3, both sourced from the official MacPorts installers - but the
> throttling appears consistent across both.

New description:

 Starting with release 2.9.0, `portindex` is now multi-threaded. And it
 works beautifully, providing near-linear speedup based on the number of
 CPU cores available.

 Testing on my 24-core MacPro2012, things work perfectly for 10.15 and
 10.14, with shockingly-fast indexing. And the `portindex` process consumes
 nearly all available CPU cores, as expected. (For my system, at least
 2100%. Note quite the 2400% of all cores, but close enough.)

 However, for 10.13 and 10.12, multi-threading appears to be significantly
 bottlenecked: On my system, the `portindex` process barely consumes two
 CPU cores. (And the performance matches the CPU usage, as it's nearly an
 order-of-magnitude slower than it is on 10.15 and 10.14.)

 Inspecting the `portindex` process itself, the number of threads appears
 to be correct in all cases. (It's always 27, even for 10.13 and 10.12.) So
 it doesn't appear to be an issue with CPU core count detection. But
 rather, something is severely slowing things down.

 Of note, testing was done on two different MacPorts releases - 2.9.1 and
 2.9.3, both sourced from the official MacPorts installers - but the
 throttling appears consistent across both.

--

-- 
Ticket URL: <https://trac.macports.org/ticket/69781#comment:1>
MacPorts <https://www.macports.org/>
Ports system for macOS