explainx.ai350K+ learn from us
TrendingAI News TodayExplainx for Business
Pricing
explainx.ai

Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.

follow us

follow on google

Add explainx.ai as a preferred source

corporate training

support@explainx.ai

get started

Find your pathTake Free Evaluation

community

Join the community

learn

mind: share how you thinkpathways — start freeworkshopsbootcampscoursescompare Explainxcertificationsmock testsexplainx universitycorporate traininglearn skills & mcp

discover

skillsmcp serversexplainx mcpestacktoolsmdx readeragentsllmsdesignsdictionarypeopleagi trackerfelony benchranks

company

aboutvisionmissionteaminstructorsteach on explainxpartnershipscommunityhackathonscareers

content

daily AI newsstate of AI — live resultsblogreleasespromptsgeneratorsresource libraryfor LLMsexplainx.ai kids

solutions

all solutionsdeveloper upskillingmarketing upskillingproduct manager upskillingleadership upskilling

newsletter · weekly

Get AI news, tools, and insights in your inbox.

supportcontactprivacytermsdata rightshow we create contentsubmission guidelines

© 2026 AISOLO Technologies Pvt Ltd

explainx.ai

On this page

  • Quick reference
  • The problem TSA targets: GPUs waiting for data
  • How Together Storage Accelerator works
  • How to turn it on
  • What Together measured
  • What it changes for AI teams
  • Where it fits in Together's recent releases
  • Practical steps before you rely on it
  • What is still unknown
  • What this means for you
  • Related reading
← Back to blog

explainx / blog

Together Storage Accelerator: Free GPU-Cluster Caching With a 20x Warm-Read Claim

Together AI, GPU Clusters, AI Infrastructure, Caching, Developer Tools

Part of AI Chips and Infrastructure

Together AI launched Storage Accelerator on Nov 19, 2026: a free cache on every GPU node with about 20x faster warm reads. What it does, how to set it up, and what to check.

Nov 19, 2026·10 min read·Yash Thakker
add explainx.ai
go deep
Together Storage Accelerator: Free GPU-Cluster Caching With a 20x Warm-Read Claim

Together AI launched Together Storage Accelerator (TSA) on November 19, 2026. It is a distributed cache that runs on every node of a Together GPU Cluster, stores objects from S3-compatible buckets and Hugging Face Hub on local NVMe, and serves repeat reads from there. Together says it is included free with every Together GPU Cluster, needs no application changes, and gives about 20x faster warm reads on an H100 node. Those figures come from Together's own launch post, not from an independent test.

This post explains what TSA does, how to turn it on, which numbers Together published, and what you should still check before you rely on them.

Quick reference

table · 2 cols
QuestionAnswer
What is it?A distributed read cache on each GPU node of a Together GPU Cluster
What does it cache?Objects from S3-compatible storage (S3, R2, Together's T4) and Hugging Face Hub
PriceIncluded free with every Together GPU Cluster, per Together
SetupSet AWS_ENDPOINT_URL and HF_ENDPOINT to the in-cluster TSA address
Headline claimAbout 20x faster warm reads, up to 3.4 GB/s per node
Cold readWithin about 5% of a direct download, per Together
TechnologyASTRA, Together's distributed caching system (technical post promised later)
EvidenceOne H100 node, us-central, measured October 28, 2026, by Together
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

The problem TSA targets: GPUs waiting for data

A long gray road with a short green fast lane beside it, showing how a local cache speeds repeat reads for a GPU cluster data caching layerA long gray road with a short green fast lane beside it, showing how a local cache speeds repeat reads for a GPU cluster data caching layer

Together opens with a plain complaint: your GPUs are only as fast as your data pipelines. Many teams keep the source of truth in object storage such as Amazon S3 or Cloudflare R2, or pull models straight from Hugging Face Hub. Every read then crosses a wide-area network, which Together calls a "WAN tax." Reads are slower, cross-cloud egress is billed (Together cites up to $0.09 per GB from AWS), and origins can rate-limit you.

Multiply that by a cluster. A 70 GB checkpoint pulled by 200 nodes means 200 separate downloads of the same bytes. The GPUs you pay for by the hour sit idle while the data arrives. A training run that starts with a 30-minute model download, as Together puts it, wastes compute before it does any work.

This is not a new class of problem. Teams already add mirrors, pre-stage data to local disks, or write their own sync jobs. TSA's pitch is that the platform does it for you at the endpoint level, so you do not build or maintain that plumbing.

How Together Storage Accelerator works

TSA runs as a service on every GPU node in the cluster and presents drop-in endpoints for the storage you already use. The flow Together describes:

  1. A job asks TSA for an object, for example through an S3 call or a Hugging Face download.
  2. On the first read, TSA fetches the object from the origin and caches it on cluster NVMe.
  3. Every later read of that object, from any node in the cluster, is served from inside the cluster.

The key property is "one origin fetch per cluster." If 200 nodes want the same checkpoint, the origin sends it once. Together also says cached data keeps serving when the upstream is down or throttling, so a Hugging Face rate limit or an origin outage does not stall a job that already has the data cached.

Credentials for the origin are configured in TSA. Together says they are encrypted and stored in a secrets vault, so the nodes themselves do not need to hold the origin keys. The post does not detail the access model, so check how it fits your own rules before pointing a private bucket at it.

TSA is built on ASTRA (Accelerated Storage Transport & Retrieval Architecture), which Together describes as the caching technology it built to feed some of the largest GPU fleets in the world, including its own inference fleet. A deeper engineering post on ASTRA is promised but not yet published.

How to turn it on

Together's getting-started instructions are two environment variables:

bash
# S3-compatible storage (S3, R2, T4, ...)
export AWS_ENDPOINT_URL=http://tsa.tsa.svc.cluster.local:18080

# Hugging Face Hub
export HF_ENDPOINT=http://tsa.tsa.svc.cluster.local:18080

The address is a Kubernetes service name, so the setup assumes your workloads run inside the cluster. Both variables are standard client settings: the AWS SDKs read a custom endpoint from configuration, as the AWS SDK reference describes, and the Hugging Face client honors HF_ENDPOINT for the Hub address. That is why Together can say existing boto3 code, transformers calls and CLI workflows work as they are. There is no new SDK and no rewritten data loader.

Together does not say how to enable TSA on an existing cluster or whether it is on by default. Ask your account team or check your cluster console for that detail.

What Together measured

Together published a table of measurements from a pod on one H100 node in a Together GPU Cluster in us-central, taken on October 28, 2026. Reads were single-stream unless noted, and the client wrote to /dev/null or node NVMe.

table · 4 cols
TestWithout TSAWith TSA (warm)Change
2 GiB object from Cloudflare R2143-169 MB/s (12.7-15.0 s)3.0-3.4 GB/s (0.64-0.72 s)about 20x
Small files (5 KB, 2.4 MB) from the same bucket320 ms and 680 ms5 ms and 8 msabout 60-80x lower latency
200 nodes pulling one checkpoint200 origin fetches1 origin fetch200x less egress
First (cold) read through TSAdirect downloadwithin about 5%roughly no penalty

Together states the method: single-stream curl and aws s3 cp from a pod on the GPU node, repeated reads of the same object, MD5-verified against the direct download. It also notes that this was a preview deployment (TSA chart 0.6.0) and that your results depend on origin, object size and network path.

Reading the numbers with care

A stopwatch with a green wedge and speed lines, standing for the warm-read timing Together published for Storage AcceleratorA stopwatch with a green wedge and speed lines, standing for the warm-read timing Together published for Storage Accelerator

A few points matter when you compare these figures to your own workload:

  • Warm versus cold. The 20x number is for a warm read, after the object is already cached. A job that reads each file once, with no reuse, gets the cold path, which Together says is roughly equal to a direct download. The big gains show up when many nodes or many epochs read the same data.
  • The origin chosen. The baseline is a cross-cloud read from Cloudflare R2 at 143-169 MB/s. Together itself says origin speed is regional: the same Hugging Face download ran 5x faster from us-central than from ca-west, so TSA's win is largest where the origin is slowest. If your data already sits in a fast, nearby store, expect a smaller gain.
  • Single stream. The headline speeds are single-stream reads. Real training jobs use many parallel readers, and a multi-process data loader will behave differently.
  • Small-file latency. The 60-80x latency drop is across only two file sizes, 5 KB and 2.4 MB. Tiny files are where per-request overhead dominates, so a cache helps a lot, but a dataset with millions of small files should be tested end to end.
  • 200x less egress is arithmetic from the design (one fetch instead of 200), not a measured bandwidth. It applies when all nodes want the same object at about the same time.
  • Vendor data. Everything above is Together's own testing. We could not find an independent benchmark of TSA at the time of writing.

What it changes for AI teams

For teams that already rent Together GPU Clusters, TSA removes a chore. Common cases where a cache like this helps:

  • Cluster start-up. Pulling a large model or dataset on every new node. With one origin fetch per cluster, node number 200 starts as quickly as node number two.
  • Repeated experiments. Re-running a training or evaluation job against the same dataset, where the second run reads from NVMe.
  • Inference scale-out. Spinning up more replicas that all need the same weights.
  • Hugging Face limits. Jobs that hit rate limits when many nodes download the same repository at once.
  • Egress bills. Fewer repeat reads from S3 or other clouds, where Together cites up to $0.09 per GB.

For teams on other providers, TSA is not an option as described. It is a feature of Together's cluster product, and the announcement does not describe a standalone version.

Where it fits in Together's recent releases

Together has been shipping infrastructure features at a steady pace, and TSA reads as part of one story: make the platform around the GPU faster, not just the GPU. The launch post's own banner links list recent items such as Together Post-Training, Together Link, a polymorphic inference compiler and on-demand B200s on GPU Clusters. We covered two of the neighbouring releases: how Together's compiler agents claim big tokens-per-second gains, and how Together AI and Ollama approach prefix caching for long-horizon agents.

The same theme shows up across the industry. NVIDIA Dynamo's work on RL rollout bottlenecks targets how fast weights and data move to the GPUs, and NVIDIA Mission Control 2.5 targets keeping clusters busy through failures. Ai2's cluster scheduler attacks idle time from the scheduling side. For the memory side of inference, see our explainer on what the KV cache is, which is a different cache from TSA: KV cache holds attention state inside a model, while TSA caches files on disk.

Practical steps before you rely on it

Treat the launch numbers as a hypothesis and test them on your own data:

  1. Pick a representative object. Use a checkpoint or shard from your real pipeline, not a synthetic file.
  2. Measure three reads. Time a direct read, a first read through TSA, and a second read through TSA. Compare the cold and warm figures with Together's.
  3. Test in parallel. Run the read from several nodes at once and confirm the origin is hit only once. Check your origin's request logs.
  4. Check freshness. If you overwrite objects in place, confirm how TSA treats a changed object. The launch post does not say how it handles invalidation, so ask before relying on it for mutable data.
  5. Check data rules. Cached copies live on cluster NVMe. If your data has residency or deletion rules, confirm where and how long TSA keeps it.
  6. Check your region. Together's own note says the gain depends on how slow the origin is from your cluster.

These are our suggestions, not steps from Together's documentation.

What is still unknown

  • Cache size and eviction. The post does not state how much NVMe TSA uses per node or what it evicts when the cache is full.
  • Invalidation. It does not say how changed or deleted objects are handled.
  • Writes. TSA is described as a read cache. The post does not describe write behavior.
  • Independent results. There is no third-party benchmark yet.
  • Availability. The numbers come from a preview deployment, and the post does not give a general-availability date or the exact regions.
  • T4 status. Together's object storage, T4, is in limited preview, so the "pairs naturally with T4" path needs a sales contact.

What this means for you

  • If you run on Together GPU Clusters: try TSA on one data-heavy job and compare against your current download time. It costs nothing extra, per Together.
  • If you pay large egress bills: the one-fetch-per-cluster design is the part most likely to show up on your invoice, but only for data many nodes share.
  • If you compare GPU clouds: add data-pipeline speed to your checklist, and ask each provider what it offers for caching. Do not rely on raw GPU price alone.
  • If you are an engineer reading the claim: the 20x figure is real in Together's test, and narrow. Check it on your own origin, object sizes and read pattern.

For another recent example of reading a vendor speed claim carefully, see our look at filesystem checkpoints and forked rollouts in Prime Sandboxes.

Related reading

  • Together compiler agents: tokens-per-second claims, checked
  • Together AI and Ollama prefix cache for long-horizon agents
  • NVIDIA Dynamo and RL rollout bottlenecks
  • NVIDIA Mission Control 2.5 for self-healing GPU clusters
  • Ai2's GPU cluster scheduler
  • What is the KV cache?
Spotted something out of date? Let us know.
Daily briefFrom Yash · 350K+ students taught

Catch up on AI

One AI story explained in plain English, plus one thing to try. Every day, free. Unsubscribe anytime.

  • ✓ Free
  • ✓ One email a day, no spam
  • ✓ Unsubscribe anytime
Yash Thakker

Written by

Yash Thakker

Yash is an AI expert with over 300K learners. Join his workshops →

View Yash Thakker in People in AI →

Related posts

Nov 20, 2026

Qwen3 TTS on Together AI: Instruction Voices, 10 Languages, $20 per 1M Chars

On November 20, 2026 Together AI shipped hosted Qwen3 TTS with instruction-driven delivery, multilingual presets, and a tuned stack that posts first audio in as little as 97ms. This guide covers models, voices, pricing versus Kokoro and Orpheus, and how to call the API today.

Nov 7, 2026

NVIDIA NVKE: A Managed Kubernetes Service for Your Own GPU Data Center

NVKE hit the Hacker News front page with 123 points. It is NVIDIA's open-source management plane for running a managed Kubernetes service on your own GPU hardware. We explain the architecture, what you can try without a GPU, and why the README says not to use it in production yet.

Nov 3, 2026

NVIDIA Attribution: Open-Source Root-Cause Diagnosis for Failed AI Training Jobs

When one GPU fails, dozens of ranks print NCCL errors, and the exception you see is rarely the cause. NVIDIA has open-sourced Attribution, the system it says it runs on millions of workloads a week, to separate the root cause from the symptoms. Here is what it does, how it works and how to try it.