Qualcomm's Cloud AI 100 Ultra reached the Hacker News front page on November 4, 2026, under the title "150W AI Accelerator Card with 128GB at 548GB/s." The Qualcomm product page is a marketing page, and the chip itself is not new. A commenter pointed out that it launched in 2024, and Qualcomm announced it in November 2023. Still, the thread raises a fair question: what does a card with lots of memory but modest bandwidth actually do for LLM inference?
The best independent data we found is a UC San Diego study that ran 12 open models on the card and on NVIDIA A100 GPUs. This post covers the specs, the HN reaction, what that study found, and who might care.
TL;DR
| Question | Short answer |
|---|---|
| What is it? | A PCIe inference accelerator card from Qualcomm, listed at 150W, 128GB, 548GB/s |
| Is it new? | No. Announced in 2023, available since 2024 |
| What is the architecture? | Four chips per card, 64 AI cores and 576 MB SRAM per card (UCSD paper) |
| How does it compare to A100? | Much lower throughput, much lower power; mixed tokens per watt |
| Where can I get it? | Cloud providers such as Cirrascale, and servers such as HPE ProLiant |
| Biggest caveat | Models must be compiled to Qualcomm's format, which can take hours |
What the specs say
The headline numbers are 150 watts, 128GB of memory and 548GB/s of memory bandwidth. Secondary sources, including a Taiwanese review and a spec-sheet aggregator, describe the memory as LPDDR4x, the same low-power memory family used in phones and laptops, and list 870 TOPS at INT8. We could not retrieve Qualcomm's own product brief, so the TOPS figure is second-hand. Qualcomm's launch material says a single 150W card can run models of about 100 billion parameters, and two cards can run about 175 billion.
Each card holds four chips and a PCIe switch. The UCSD paper says each card has four system-on-chip devices, with 64 AI cores and 576 MB of on-die SRAM across the card.
Why memory capacity and bandwidth pull in different directions
Generating text is mostly a memory-reading job. For every token, the processor reads the model's weights. A fast chip with slow memory waits. Datacenter GPUs use HBM that delivers several terabytes per second. A 548GB/s card has a fraction of that. The upside is capacity: 128GB on one card can hold a quantized 100B-class model that would need several smaller GPUs. If you are weighing how much memory a local or edge setup needs, our piece on Qualcomm's CEO and 100B-parameter models on phones walks through the same memory math.
A back-of-envelope ceiling
Here is a simple estimate, which is our own arithmetic and not a vendor number. Suppose a 70B model is stored in 8-bit form, so the weights take about 70GB. If every generated token requires reading all the weights once, a single stream is capped at roughly bandwidth divided by weight size. At 548GB/s that is about 7 to 8 tokens per second for one request. Batching several requests shares one weight read among them, which raises total throughput, but each user still sees a modest speed. The UCSD measurements fit this picture: single-digit tokens per second for the 70B models in their table, using several devices and a different precision setup.
The same arithmetic explains why HBM matters. A GPU with 3TB/s of bandwidth has a ceiling several times higher for the same model. It also explains why SRAM matters. Data held in on-chip SRAM can be read far faster than data in external memory, so a design with a large SRAM pool can keep hot data close. The card's 576 MB of SRAM is small next to a 70GB model, so it cannot hold the weights, but it can help with caches and intermediate values. How much it helps in practice is an open question that only workload benchmarks answer.
Why Qualcomm is in this market
Qualcomm is best known for phone chips. It entered data center inference with the Cloud AI 100 family in earlier years and pitched power efficiency from the start, which is a mobile company's home turf. The Ultra variant adds more memory and more chips per card to handle larger language models. The company has since talked up on-device and edge AI heavily, and the data center card is a separate line that reuses the same efficiency pitch. Whether that pitch wins customers depends less on peak numbers than on software: compilers, framework support and how quickly new model architectures run. That is where NVIDIA's CUDA ecosystem remains hard to match, and it is the main thing buyers should test.
What Hacker News said
The thread had 35 points and a modest discussion when we read it.
- varispeed argued the specs look dated and wanted far more memory and bandwidth from a new product.
- tokepi replied that the specs look old because the chip launched in 2024, and that it is available at cloud providers such as Cirrascale.
- aetherspawn compared it with Apple silicon, first judging it worse, then correcting that bandwidth is lower but TOPS is much higher, and wondering how much high TOPS helps with low bandwidth.
- zitterbewegung found the design interesting because it pairs a large on-chip SRAM with regular DRAM in place of HBM.
- pearl_panel5 wanted sustained-load numbers, and thway15269037 asked the price, which the page does not state.
These are commenters' opinions, not measurements. They do show the questions buyers ask: how fast under load, and at what price.
What the UCSD study found
Researchers at UC San Diego, including staff of the San Diego Supercomputer Center, compared the card with NVIDIA A100 SXM4 80GB GPUs on the National Research Platform. The paper is Serving LLMs in HPC Clusters: A Comparative Study of Qualcomm Cloud AI 100 Ultra and NVIDIA Data Center GPUs. They ran 12 open models from 124M to 70B parameters through vLLM, including GPT-2, granite-3.2-8b, DeepSeek distills, Llama and Nemotron-70B.
Selected results, from the paper's table for 200 tokens and four parallel requests:
| Model | A100 throughput and power | Qualcomm throughput and power |
|---|---|---|
| GPT-2 | 2,613 tok/s at 1,205 W (4 GPUs) | 218 tok/s at 38 W (1 device) |
| granite-3.2-8b | 318 tok/s at 1,246 W (4 GPUs) | 25 tok/s at 36 W (1 device) |
| DeepSeek-Llama-70B | 104 tok/s at 2,935 W (8 GPUs) | 8 tok/s at 292 W (8 devices) |
| Nemotron-70B | 104 tok/s at 2,983 W (8 GPUs) | 6 tok/s at 148 W (4 devices, one card) |
The paper reports a 2.7x tokens-per-watt advantage for the Qualcomm card on granite-3.2-8b. Under a hypothetical scaling table that assumes perfect load balancing, the advantage ranged from 0.13x to 2.78x, with Qualcomm ahead only on GPT-2, granite-3.2-8b and Nemotron-70B. So the efficiency story is mixed, and throughput is far lower.
Limits of that study
- Only A100 GPUs were compared, not H100 or L40S.
- The scaling table assumes perfect load balancing, which was not measured.
- Power was measured per GPU or card, not per full node.
- Some A100 runs used four GPUs for small models, which may inflate their power, and some A100 entries in the scaling table were scaled from 4 to 8 GPUs by assumption.
- Models must be compiled to Qualcomm's QPC format, which can take hours, and the Qualcomm side was a single node.
- Some tables in the paper have labeling inconsistencies.
The paper is a useful data point, not a purchasing guide. It dates from an earlier model generation than today's GPUs.
Where it fits
For readers tracking AI hardware, the interesting part is the design choice. This card trades throughput for capacity and power. It is closer in spirit to inference-first hardware than to training GPUs. Compare it with the opposite bet in Cerebras' CS-4 wafer-scale accelerator, which chases speed, and with software efforts such as Modular's open-sourced Mojo for heterogeneous hardware that try to make non-NVIDIA hardware easier to program. On the consumer side, see how developers squeeze more from existing silicon in the RTX 4090 tensor unlock and the Llama Windows llama.cpp tray app.
Who should look at it
- Teams running steady, modest-traffic inference on large open models where power and rack space cost more than latency.
- Researchers who want 128GB per card without paying for HBM-class GPUs.
- Buyers who can accept a compile step per model and a smaller software ecosystem than CUDA.
Who should not: anyone who needs the highest tokens per second, the newest frameworks on day one, or a known public price. Qualcomm's page does not list one.
Checklist before you buy
- Ask for the exact memory type and bandwidth in writing.
- Ask which models are pre-compiled, and how long a custom compile takes.
- Benchmark your own model, batch size and context length, not a vendor chart.
- Measure power at the node, not just the card.
- Compare against a current GPU, not only the A100.
Related reading
- Qualcomm CEO and 100B-parameter models on phones
- Cerebras CS-4
- Modular open-sources Mojo
- RTX 4090 tensor unlock
- Llama Windows app
Specs come from Qualcomm's product page title, secondary sources and the UCSD paper, as read on November 4, 2026. Follow @explainx_ai for updates.