Vision Framework

Detect text, faces, barcodes, objects, and body poses in images and video using on-device computer vision. Patterns target iOS 26+ with Swift 6.3, backward-compatible where noted.

See references/vision-requests.md for complete code patterns and references/visionkit-scanner.md for DataScannerViewController integration.

Contents

• Two API Generations
• Request Pattern (Modern API)
• Text Recognition (OCR)
• Face Detection
• Barcode Detection
• Document Scanning (iOS 26+)
• Image Segmentation
• Object Tracking
• Other Request Types
• Core ML Integration
• VisionKit: DataScannerViewController
• Common Mistakes
• Review Checklist
• References

Two API Generations

Vision has two distinct API layers. Prefer the modern API for new code.

The modern API uses the ImageProcessingRequest protocol. Each request type has a perform(on:orientation:) method that accepts CGImage, CIImage, CVPixelBuffer, CMSampleBuffer, Data, or URL. Most requests are structs; stateful requests for video tracking (e.g., TrackObjectRequest, TrackRectangleRequest, DetectTrajectoriesRequest) are final classes.

Request Pattern (Modern API)

All modern Vision requests follow the same pattern: create a request struct, call perform(on:), and handle the typed result.

import Vision

func recognizeText(in image: CGImage) async throws -> [String] {
    var request = RecognizeTextRequest()
    request.recognitionLevel = .accurate
    request.recognitionLanguages = [Locale.Language(identifier: "en-US")]

    let observations = try await request.perform(on: image)
    return observations.compactMap { observation in
        observation.topCandidates(1).first?.string
    }
}

Legacy Pattern (Pre-iOS 18)

Use VNImageRequestHandler with completion-based requests when targeting older deployment versions.

import Vision

func recognizeTextLegacy(in image: CGImage) throws -> [String] {
    var recognized: [String] = []
    let request = VNRecognizeTextRequest { request, error in
        guard let observations = request.results as? [VNRecognizedTextObservation] else { return }
        recognized = observations.compactMap { $0.topCandidates(1).first?.string }
    }
    request.recognitionLevel = .accurate

    let handler = VNImageRequestHandler(cgImage: image)
    try handler.perform([request])
    return recognized
}

Text Recognition (OCR)

Modern: RecognizeTextRequest (iOS 18+)

var request = RecognizeTextRequest()
request.recognitionLevel = .accurate       // .fast for real-time
request.recognitionLanguages = [
    Locale.Language(identifier: "en-US"),
    Locale.Language(identifier: "fr-FR"),
]
request.usesLanguageCorrection = true
request.customWords = ["SwiftUI", "Xcode"] // domain-specific terms

let observations = try await request.perform(on: cgImage)
for observation in observations {
    guard let candidate = observation.topCandidates(1).first else { continue }
    let text = candidate.string
    let confidence = candidate.confidence  // 0.0 ... 1.0
    let bounds = observation.boundingBox   // normalized coordinates
}

Legacy: VNRecognizeTextRequest

let request = VNRecognizeTextRequest()
request.recognitionLevel = .accurate
request.recognitionLanguages = ["en-US", "fr-FR"]
request.usesLanguageCorrection = true

Key differences: Modern API uses Locale.Language for languages; legacy uses string identifiers. Both support .accurate (best quality) and .fast (real-time suitable) recognition levels.

Face Detection

Detect face rectangles, landmarks (eyes, nose, mouth), and capture quality.

// Modern API
let faceRequest = DetectFaceRectanglesRequest()
let faces = try await faceRequest.perform(on: cgImage)

for face in faces {
    let boundingBox = face.boundingBox   // normalized CGRect
    let roll = face.roll                 // Measurement<UnitAngle>
    let yaw = face.yaw                  // Measurement<UnitAngle>
}

// Landmarks (eyes, nose, mouth contours)
var landmarkRequest = DetectFaceLandmarksRequest()
let landmarkFaces = try await landmarkRequest.perform(on: cgImage)
for face in landmarkFaces {
    let landmarks = face.landmarks
    let leftEye = landmarks?.leftEye?.normalizedPoints
    let nose = landmarks?.nose?.normalizedPoints
}

Coordinate System

Vision uses a normalized coordinate system with origin at the bottom-left. Convert to UIKit (top-left origin) before display:

func convertToUIKit(_ rect: CGRect, imageHeight: CGFloat) -> CGRect {
    CGRect(
        x: rect.origin.x,
        y: imageHeight - rect.origin.y - rect.height,
        width: rect.width,
        height: rect.height
    )
}

Barcode Detection

Detect 1D and 2D barcodes including QR codes.

var request = DetectBarcodesRequest()
request.symbologies = [.qr, .ean13, .code128, .pdf417]

let barcodes = try await request.perform(on: cgImage)