Real tokens per second on the hardware inside the Node.

What this hardware actually does

The HX 370 with a Radeon 890M is a very capable mini PC class processor, not a discrete GPU workstation. On comparable machines running llama.cpp it generates at reading speed on 7B to 8B models, comfortably on 30B class mixture of experts models, slowly on dense 14B, and impractically on dense 70B. Here is the shape of it.

Sources for every figure are in the tables below. The single most useful model class for a 64 GB Node is the 20B to 35B mixture of experts family (Qwen3 30B-A3B, gpt-oss-20b, Gemma 4 26B-A4B): they carry the knowledge of a large model while only activating 3B to 5B parameters per token, so they run two to three times faster than a dense 8B on this hardware.

7B to 8B dense models

The Llama 3.1 8B class. Four bit quantization (Q4_K_M) is about 4.9 GB of weights, so the whole model and a long context fit with tens of gigabytes to spare.

Two things stand out. Memory speed matters more than backend: the Beelink on LPDDR5X-7500 runs an 8 bit model as fast as the Framework laptop on DDR5-5600 runs a 4 bit one. And on the 890M, Vulkan is usually faster than ROCm for generation, though not always. The Node ships both and picks per model.

14B dense models

Qwen 14B class at 4 bit is roughly 9 GB of weights. Generation is bandwidth bound, so it lands at a bit more than half the 8B speed.

The Framework result is instructive: CPU, Vulkan, and ROCm all landed between 7.1 and 7.8 tokens per second on a 14B model. When a model is bandwidth bound, the compute backend barely matters for generation. Prompt processing is where the GPU earns its keep.

20B to 35B mixture of experts models

This is where 64 GB of unified memory pays off. These models hold 20 to 35 billion parameters but only compute with 3 to 5 billion per token, so they read the memory bus like a small model while answering like a large one.

Context length costs speed. The same Gemma 4 26B-A4B run dropped from 28.3 tokens per second at an empty context to 22.0 at 32k tokens. Long document sessions should expect the lower end of these ranges.

32B and 70B dense models

They load. That is the extent of the good news. A 4 bit 32B model is about 19 GB and runs at about 4 tokens per second. A 70B model at Q6 is about 45 GB and runs at one to two tokens per second on a 64 GB HX 370 machine. If dense 70B at interactive speed is the requirement, this is the wrong class of hardware, and the compare page says so.

Against other machines

Same model class, Llama 3.1 8B at Q4_K_M or the nearest published equivalent, on the four machines people ask about. Grey bars are reference machines. Prices and the full feature comparison are on the compare page.

Sources: LocalScore #721, r/MiniPCs SER9 Pro, LocalScore #777 Mac mini M4 Pro, LocalScore #1771 DGX Spark, level1techs Strix Halo. The DGX Spark gap on generation is only about 2.5x because generation is bandwidth bound on every one of these machines. Its prompt processing lead is 10x to 20x, which matters for long documents.

Reference points for larger models: Llama 3.3 70B at 4 bit runs at about 5 tokens per second on a 256 GB/s Strix Halo machine (valerian.dtdg.fr) and about 4.8 tokens per second on a DGX Spark through Ollama (Hugging Face dataset). Dense 70B is slow everywhere under $5,000.

Speech and embeddings

WhisperX large-v3 on a Radeon 890M through ROCm transcribed a 4 minute 40 second recording in 1 minute 13 seconds on the GPU at fp16, against 3 minutes 58 seconds on the CPU at int8 (dominic-boettger.com (Aug 2026)). The same author reports GPU throughput settling to about 1.5x real time after 30 minutes of sustained load, which is a thermal story and a good reason the Node has a real cooling path instead of a laptop one.

No published embedding throughput exists for the HX 370 specifically. Generic CPU figures for small embedding models are under 10 milliseconds per query at batch size one (Hugging Face and Intel (Mar 2024)); the Node's measured figure goes here when it exists.

RAG latency on a 10k document corpus

Nobody has published a retrieval benchmark on HX 370 hardware, so this section is generic and labeled as such. What the generic numbers say: on a modern CPU, embedding a query and running an HNSW vector search over tens of thousands of chunks takes tens of milliseconds. A 50k document, 1024 dimension test on a 32 vCPU server returned 70 to 764 queries per second depending on the database, or roughly 1.3 to 14 milliseconds per query (AIMultiple (Jul 2026)). A consumer desktop test over 100k vectors ran FAISS at 344 queries per second (r/LocalLLaMA (Apr 2026)).

Indexing is the slow part. A generic CPU embeds roughly 580 chunks per second with a small embedding model, so an initial index of 5,000 PDFs is a 30 to 90 minute job (PromptQuorum (2026)). After that, incremental updates are seconds.

Why the numbers look like this

• Generation is memory bandwidth divided by model size. The HX 370 reads memory at a theoretical 120 GB/s with LPDDR5X-7500 (128 GB/s at 8000), and the 890M has been measured at 96 GB/s (Chips and Cheese (Aug 2024)). A 4.9 GB model at 96 GB/s is about 20 tokens per second in theory; real runs land at 12 to 17. A Mac mini M4 Pro and a DGX Spark both sit near 273 GB/s, which is why they are about 2.5x faster on the same model.
• Laptop results undersell the platform. The Framework Laptop 13 uses DDR5-5600 SODIMMs at 89.6 GB/s theoretical and 67 GB/s measured, and dropped from 13.4 to 9.9 tokens per second on battery (msf.github.io). The Node is on wall power with soldered LPDDR5X.
• The iGPU has no VRAM of its own. It borrows system memory, and Linux lets you give it most of the 64 GB. Windows caps shared GPU memory near 3.5 GB by default (dev.to), which is one reason the Node runs Linux and ships preconfigured.
• Vulkan versus ROCm. The 890M (gfx1150) is not on AMD's official ROCm support list. On the same Beelink, Vulkan gave about 16 tokens per second and ROCm 7.1 about 12.5 on an 8B model (r/ROCm). One ROCm run did hit 16.7 on Llama 2 7B. This is a moving target and the Node ships both backends.
• Sustained load is a thermal problem. A Beelink SER9 Pro held 84 to 87 °C at about 58 W for a 60 minute run (r/MiniPCs). The Node's bottom up airflow and copper cold plate exist for exactly this. Measured sustained numbers are on the commitment list.
• These are single user reports, not a lab. LocalScore and the llama.cpp discussion tables are the most reproducible sources here. Reddit and blog figures are one person, one box, one afternoon. Sources dated 2024 through August 2026 are mixed; software has improved over that span.

How the Node will be measured

When the first engineering unit is on the bench, this page is rewritten with results from it. The method is fixed now so nobody can accuse me of picking it afterwards.

1. Tool. llama-bench from the llama.cpp release pinned in the shipping image, with the commit hash published. Vulkan and ROCm both reported. Ollama numbers reported separately where they differ.
2. Models. Llama 3.1 8B Instruct Q4_K_M and Q8_0. Mistral 7B Instruct Q4_K_M. Qwen3 14B Q4_K_M. Qwen3 30B-A3B Q4_K_M. gpt-oss-20b MXFP4. Qwen2.5 Coder 32B Q4_K_M. Llama 3.3 70B Q4_K_M. Every GGUF named with its source repository.
3. Tests. pp512 and tg128 (the llama.cpp standard), plus pp4096 and tg256 for long context. Cold and warm. Ten minute sustained generation with the fan and temperature logged.
4. RAG. 10,000 mixed PDF and Markdown documents. Time to index. Query embedding time, vector search time, and end to end time to first token and full answer with the 8B and 30B-A3B models. Same corpus published so anyone can repeat it.
5. Power. Idle, single user chat, and sustained load wattage at the wall, plus noise at one meter.

If the measured Node is slower than the comparable machines above, that will be on this page too. Reserving now means you are buying the platform described here, on the terms described here, and the build log is where you hold me to it.