llama.cpp Speeds Selected Mac Tests by Fusing Computation

llama.cpp merged a new set of Metal optimizations on September 19, improving selected tests on Apple's graphics hardware by combining suitable operations. The revealing detail is not simply that fewer kernels run. During development, one attempted combination made text generation slower because it removed useful overlap between operations; that change was discarded before the final merge. The result is a selective improvement to an inference runtime, not a new model release or a promise that every Mac application is faster. It offers an unusually concrete way to examine where a performance gain begins—and where claims about its financial value should stop.

The contributor's benchmark table compares the change with baseline b10809. On M2 Ultra, the row labelled qwen35moe 35B.A3B Q4_K_M rises from 90.26 to 104.87 tokens per second in a 32-token generation test, about 16%. Processing a 2,048-token prompt improves from 2309.46 to 2415.57, roughly 5%. On M5 Max, the same model label reaches 117.75 rather than 104.70 tokens per second in a 128-token generation test, about 12%. The 27B Q4_K_M model's M2 Ultra prompt-processing result barely moves: 366.64 to 367.09. These are contributor-reported measurements, not independently reproduced ZharfAI results; even the generation lengths differ between devices.

The rejected fusion joined matrix-vector work with an activation function. Its recorded Qwen3.6-35B-A3B test fell from 88.5 to 81.5 tokens per second, an approximately 8% regression. The implementation notes explain that previously separate work could overlap with other matrix-vector operations; combining it serialized work on the critical path. That slowdown describes the discarded candidate, not the final optimization set. Its model name also differs from the benchmark table's label, so the two records should not be presented as one experiment. The fresh event is the September 19 mainline merge of PR 28948, not the proposal's earlier opening or the launch of a model.

For a Mac-based application, ZharfAI's interpretation starts with that unevenness. Consider a hypothetical service that reads a long document and writes a short answer. Faster generation does not shorten the whole response by the same percentage when most of the wait comes from ingesting the prompt, finding documents or calling another tool. A long-form writing task gives generation a larger share of the total. The appropriate comparison therefore uses the same model, quantization format and realistic input lengths before and after the change. This example is not a published Persian-document benchmark, nor evidence that every application built for a Mac already includes the merged code.

A separate September 19 llama.cpp merge, PR 28389, supplies the correctness counterpoint. Its CUDA change separates the input and output key buffers used by a CUB sorting operation. Sharing them could overwrite input while sorting and produce invalid indices that later triggered out-of-bounds memory access. The reported case involved a particular GPU-sorting path on Maxwell hardware with CUDA 12.5 and CCCL 2.x, not every ordinary request. For an affected workload, avoiding a failed run and its retry can matter more than a headline speed record. This is not evidence of a security intrusion, a defect in every NVIDIA card, or a general solution to model accuracy.

The financial question remains open even when throughput improves. The Metal table does not measure electricity use, answer-review time or deployment expense. On an already purchased machine, a gain could free time on the existing asset rather than reduce a bill; if the application mostly waits on another tool, it might not use that extra capacity. ZharfAI would assess this particular update through time and cost per completed, acceptable task. Measuring power and the number of outputs requiring another attempt would show how much of the computational improvement reaches that outcome. The available benchmark does not supply those answers, and token throughput alone cannot stand in for them.

Friday's economic releases provide a separate demand backdrop, not an explanation for the software benchmark. The Federal Reserve reported unchanged August industrial production after July's 0.2% increase, with manufacturing down 0.3% and utilities up 1.8%. Capacity utilization was 76.3%, or 3.1 percentage points below its 1972–2025 average. August's figures are preliminary and seasonally adjusted. For an industrial equipment or software supplier, the composition matters: a flat aggregate can conceal weaker factory output. It is neither a measure of total GDP nor an inventory of usable AI capacity. A sector-specific sales assumption needs more than the headline industrial number.

The ECB's August consumer survey adds a different caution. Median one-year inflation expectations edged from 2.9% to 3.0%, while expected nominal income growth remained 1.0% and expected nominal spending growth stayed 3.6%. ZharfAI reads this as a reason not to equate anticipated price increases with greater willingness to buy. These are respondents' expectations, not realized inflation or an official ECB forecast. Fieldwork ran August 6–24, so the answers cannot be attributed to September's news. The income and spending measures are also not the inflation median. For consumer-facing businesses, keeping those distinctions intact prevents one price-expectations statistic from replacing an assessment of demand.

On the physical supply side, Virginia Governor Abigail Spanberger signed Executive Order 22 on September 18. Effective immediately, it excludes new data-center projects with anticipated peak electricity demand of 25 MW or greater from VEDP assistance through the Business Ready Sites Program, expedited permitting and similar discretionary state programs. It is not a blanket construction ban. ZharfAI's operational inference is narrower: a developer whose schedule assumes this assistance needs to revisit that dependency. Owning land or ordering equipment does not supply the missing development pathway. Neither the threshold nor the restriction should be generalized to every state or to all existing projects.

The order also restricts future executive-agency nondisclosure agreements covering material project information, while honoring existing commitments and allowing extraordinary exceptions such as national security. Other pieces take longer: a community-engagement toolkit is due within 120 days, and proposed VA-LEAD designation criteria and an implementation plan are due by the end of 2027. Calls to coordinate fair grid-cost allocation are not a finalized uniform tariff. In ZharfAI's assessment, a project budget should distinguish assistance removed now from costs whose eventual rules remain unsettled. A faster Mac workload is not a substitute for a large facility's permit or electricity connection.

Next, the relevant evidence is replication of the Metal results on other workloads and incorporation of the changes into applications that depend on llama.cpp. For the affected CUDA path, stable completion matters more than extrapolating a speed percentage. The ECB's next survey release on October 23 and the Federal Reserve's industrial revision on November 24 will add economic evidence; Virginia's implementation details will clarify the infrastructure consequences. The lead conclusion is deliberately bounded: selective fusion has helped specific tests, but its economic value depends on correct, completed work. Counting removed operations is not enough to establish it.