Why the screenshot needs a text layer

If the screenshot preserves the screen, why send recognized text too? Because shrinking an image can make small characters harder to recover, while text alone discards the visual relationships around them. The useful payload is not one or the other. It is both, with the limits of each made clear.

Text can recover what downscaling blurs

At a 448×448 image budget, DocVLM's learned OCR modality lifted DocVQA accuracy from 56.0% to 86.6% on InternVL2 and from 84.4% to 91.2% on Qwen2-VL¹¹DocVLM: Make Your VLM an Efficient Reader Shpigel Nacson et al. (AWS AI Labs), 2024. The experiment integrated OCR into the model; it did not test Noru or the simple act of appending OCR to a prompt. It does show why the modalities are complementary: the image carries the page while another channel can restore characters the pixels no longer make legible.

Ordered text is useful; inferred text is fallible

The same work finds that layout-aware text can outperform a flat OCR string at the same budget¹. Noru's default text layer makes a more modest promise: recognized text is returned in reading order. It should not be mistaken for a complete geometric representation of the screen.

That distinction matters because OCR is an inference. It can miss a character, merge a label, or choose the wrong reading order. Noru labels measured capture facts separately from recognized text so the agent can treat the text as a useful reading aid, not as ground truth.

ScreenAI also reports gains from adding OCR to screen-, document-, and infographic-understanding tasks²²ScreenAI: A Vision-Language Model for UI and Infographics Understanding Baechler et al. (Google DeepMind), 2024. Pix2Struct, meanwhile, shows how much information lives in the visual arrangement of documents, interfaces, and charts³³Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding Lee et al., 2023. Neither result is a benchmark of Noru's payload. Together they explain why throwing away either channel is an avoidable loss.

What each channel contributes

The Noru decision

Noru pairs a model-sized rendering of the screen with recognized text in reading order. The full-resolution original remains local. The pairing is designed to preserve more of the evidence without pretending that OCR is exact.

The screenshot carries layout; the text layer makes recognized words directly addressable. That pairing reduces a known trade-off, but it neither recovers every character nor makes OCR ground truth.