Open Vision Banana: An Open Recipe That Turns FLUX.2 Klein Into a Segmentation and Depth Model

A team of researchers has released Open Vision Banana, a fully open recipe that turns a pretrained image generator into a general vision model. It fine-tunes the open-weight FLUX.2 Klein 4B and 9B generators on public datasets, and the same model then does semantic, instance, referring and reasoning segmentation, plus relative depth and surface normals. The paper reports that it beats Google DeepMind's closed Vision Banana on zero-shot Cityscapes, 73.9 vs 69.9 mIoU. Data, code and 148 checkpoints are public, according to the project page.

The headline number needs care, and we explain below why. The bigger point is the method: perception expressed as image generation, with no task-specific prediction heads.

TL;DR: what Open Vision Banana is

How does perception become image generation?

The project page gives the loop in plain terms. You give the model an input image and a text instruction, such as "paint the requested Cityscapes classes with their assigned colors." The model generates an RGB image. A deterministic decoder then reads the prediction: for segmentation, it maps each pixel to the nearest prompted color and then to a class label. No extra prediction head or loss is added.

Computer vision segmentation lifting one object cleanly from its background, as in generative visual perception

This idea comes from Google DeepMind's Vision Banana, which frames vision tasks as conditional image editing. The authors write that its "data, weights, code, and implementation details are not public." Open Vision Banana is their attempt to reproduce the idea in the open, and to study which design choices matter.

For depth and normals, the same trick applies: the model draws a depth or normal map as an image and a decoder converts colors to values.

What are the results?

The project page lists selected comparisons. OVB numbers are non-zero-shot unless stated, and the authors warn that comparison setups differ.

Read the table with the authors' own caution. The abstract's 73.9 vs 69.9 figure is the zero-shot Cityscapes comparison. The 75.7 figure on the project page is a non-zero-shot variant with a 20-pass vote, and the page says that score exceeds the reported zero-shot scores of Vision Banana and SAM 3 while noting that training and inference setups differ. The paper says the non-zero-shot variants come within five points of Vision Banana on referring and reasoning segmentation, and stay further behind on instance segmentation. We have not rerun any of these numbers, and the project page packs several comparison rows together, so check the paper's tables before you quote one.

What is legend ensembling?

The paper's most reusable idea is a test-time scaling method for generator-based perception. The authors found that the model often finds the right regions but assigns the wrong colors. So they ask for the same classes with different color assignments, decode each output into labels, and take a majority vote per pixel.

Three steps:

1. Change the color legend: request the same categories with different colors.
2. Predict and decode: the same model generates each image, and colors are decoded into class labels.
3. Vote: for each pixel keep the category chosen by most passes.

According to the project page, zero-shot 9B Cityscapes goes from 65.4 to 74.06 mIoU, first pass to peak vote across 27, 28 or 30 passes. On COCO-Object, 44.9 goes to 59.3 mIoU with a 90-pass vote. The paper summarizes the gain as up to 8.7 points on Cityscapes and 15.0 on COCO-Object, with no retraining. The cost is plain: more passes mean more generations, so inference gets many times slower than a single pass.

What does the recipe teach about building these models?

The project page lists three takeaways from the ablations.

The color interface matters. Training with a fixed color legend gave 54.3 mIoU on Cityscapes, but only 7.1 when the colors changed. Randomized color assignments teach the model to follow the prompt instead of memorizing colors.

Tasks interact. More data or longer training can help one task while hurting another. The final recipe balances segmentation, geometry and retained generative abilities. The abstract adds that individually helpful choices can hurt when combined.

Adapt a lot of the backbone. Tuning only the last few transformer blocks performs poorly. Strong perception needs adapting a substantial fraction of the backbone.

Why this matters for builders

Most vision stacks use a separate model per task: one for segmentation, one for depth, one for normals. A single open generator that does all of them, and still edits images, could simplify pipelines for robotics, photo tools and data labeling. It also shows that open image generators hold usable scene understanding, which is a point we raised when covering open FLUX-family weights and multi-turn editing drift on FLUX.2 Klein.

For comparison, specialist models remain strong. Meta SAM 3 is built for promptable concept segmentation, and NVIDIA MetricTrail targets metric depth in long video. OVB trades some specialist accuracy for one shared model, and the paper is open about where it trails.

What are people likely to ask next?

Why would an image generator understand scenes? The abstract starts from the observation that pretrained image generators encode real-world understanding beyond image synthesis. A model that can draw a street scene has to know where the road ends and the cars begin. Vision Banana exploited that by framing vision tasks as conditional image editing, and OVB shows the same behavior in open models. The claim to watch is how far it generalizes: the paper stresses that its design choices interact, so a recipe that works for segmentation may not transfer unchanged to a new task.

Is the 73.9 number a fair win over Google? It is a fair headline for what the paper measured, but not a controlled head-to-head. Vision Banana is closed, so the authors compare against its reported score. They say plainly that training and inference setups differ across methods. Treat it as evidence that an open recipe can reach the same range, not as proof of a better model.

Does retraining on public data hurt image editing? The paper says all models largely retain the base model's image-editing abilities. That matters if you want one checkpoint for both tasks. We have not tested editing quality on the released weights.

Why does the color legend matter so much? Because the model answers with colors, not labels. If it always saw the same legend, it could memorize colors. The ablation in the project page shows the gap: 54.3 mIoU with a fixed legend on Cityscapes, and only 7.1 when the colors change. Randomizing the legend during training teaches the model to read the prompt. Legend ensembling then uses that same freedom at test time, by asking the same question in several color vocabularies and voting.

What about the people behind it? The project page lists seven authors across the University of Technology Nuremberg, DLR, AIST, valeo.ai, the Technical University of Munich and the University of Tubingen. Press summaries of the release credit it to FunAI Lab. We could not find a separate FunAI Lab repository, so rely on the paper and the project page for the official links.

Limits and open questions

• Not like for like. The authors say training and inference setups differ across methods. Vision Banana is closed, so no one outside DeepMind can rerun its numbers.
• Instance segmentation lags. The paper says non-zero-shot variants remain further behind there.
• Test-time cost. The best scores use many passes: 20 to 90 generations per image on the cited results.
• Licenses. We have not verified the license of the FLUX.2 Klein base models or of the released OVB weights. Check the model card terms before shipping a product on any checkpoint. Our FLUX 3 Image Klein coverage covers a different model and its license.
• Early. The paper was posted on November 11, 2026 and has not been independently reproduced in anything we found.

How to try it

1. Open the project page and follow its links to the code repository and the model weights.
2. Start with a 4B checkpoint on a few images, single pass, and check the decoded mask visually.
3. Add legend ensembling with 5 to 10 passes and measure the gain against the extra time.
4. Compare against a specialist such as SAM 3 on your own images, not only on benchmark sets.
5. Read the ablation checkpoints if you want to train your own variant: they show the effect of each design choice.

For the cost side of vision inputs in language models, see our guide to the multimodal vision token budget.

Related reading

• FLUX.3 Image Klein open weights guide
• TailSteer and multi-turn image editing drift on FLUX.2 Klein
• Meta SAM 3 promptable concept segmentation
• NVIDIA MetricTrail metric depth for long video
• Multimodal vision token budget guide
• NVIDIA FLORA open robot data engine models

Details are accurate as of November 13, 2026, from the arXiv abstract and project page. Check the repository for updates.