Forums

Hi folks!

Any way to get rid of the rendering? While playing with the "GetStartedSwift" project, I figured out how to initiate the engine and how to create the IINKEditor. But it seems the editor needs an IINKRenderer as well as an FontMetricsProvider (both seems very rendering specific). My app uses an own approach on rendering. So I like to provide only the data points. Any ideas?

Best regards

Chris

Hi Thomas,

I'm having the same use-case - I just want to put some strokes in and get a word out. My current implementation looks like this follows but I always get "" (empty string) in return:

FontMetricsProvider, minimal implementation, I don't want to render anything: 

public class FontMetricsProvider: NSObject, IINKIFontMetricsProvider {

    public func getCharacterBoundingBoxes(_ text: IINKText!, spans: [IINKTextSpan]!) -> [NSValue]! {
        return []
    }

    public func getFontSizePx(_ style: IINKStyle!) -> Float {
        return style.fontSize
    }

}

My custom engine initializer (I wrapped the static lib in a dynamic framework because I need it in multiple other frameworks):

extension IINKEngine {

    public convenience init?(certificateData: Data = Data(bytes: myCertificate.bytes, count: myCertificate.length), configurationPath: String = "/recognition-assets/conf") {
        self.init(certificate: certificateData)

        // Configure the iink runtime environment
        let configurationPath = Bundle(for: IINKEngine.self).bundlePath.appending(configurationPath)
        do {
            try configuration.setStringArray("configuration-manager.search-path", value: [configurationPath]) // Tells the engine where to load the recognition assets from.
        } catch {
            print("Should not happen, please check your resources assets : " + error.localizedDescription)
            return nil
        }

        // Set the temporary directory
        do {
            try configuration.setString("content-package.temp-folder", value: NSTemporaryDirectory().appending("/HandwritingRecognitionAssets"))
        } catch {
            print("Failed to set temporary folder: " + error.localizedDescription)
            return nil
        }
    }

}

This is the meat:  

public class HandwritingRecognizer {

    private let engine: IINKEngine? = IINKEngine()
    private lazy var editor: IINKEditor? = {
        let scaledDPI = Float(264 / UIScreen.main.scale)
        guard let renderer = self.engine?.createRenderer(withDpiX: scaledDPI, dpiY: scaledDPI, target: nil) else { return nil }
        let editor = self.engine?.createEditor(renderer)
        editor?.setFontMetricsProvider(FontMetricsProvider())
        return editor
    }()

    private lazy var package = self.createPackage(packageName: UUID().uuidString)

    private var currentPart: IINKContentPart?

    init() {}

    public func processStrokes() {
        guard let editor = self.editor else {
            return
        }

        editor.setViewSize(CGSize(width: 5, height: 10))

        addH()
        do {
            editor.part = try package?.getPart(currentPart!.identifier)
        } catch {
            print("Error while creating package : " + error.localizedDescription)
        }
        let text = try? editor.export_(nil, mimeType: .text)
        print(text)
    }

    private func addH() {
//        // left
//        let eventDown1 = IINKPointerEvent(eventType: .down, x: 0, y: 0,  t: 1, f: 1, pointerType: .pen, pointerId: 0)
//        let eventMove1 = IINKPointerEvent(eventType: .move, x: 0, y: 10, t: 2, f: 1, pointerType: .pen, pointerId: 0)
//        let eventUp1   = IINKPointerEvent(eventType: .up, x: 0,   y: 10, t: 3, f: 1, pointerType: .pen, pointerId: 0)
//
//        // right
//        let eventDown2 = IINKPointerEvent(eventType: .down, x: 5, y: 0,  t: 4, f: 1, pointerType: .pen, pointerId: 0)
//        let eventMove2 = IINKPointerEvent(eventType: .move, x: 5, y: 10, t: 5, f: 1, pointerType: .pen, pointerId: 0)
//        let eventUp2   = IINKPointerEvent(eventType: .up,   x: 5, y: 10, t: 6, f: 1, pointerType: .pen, pointerId: 0)
//
//        // middle
//        let eventDown3 = IINKPointerEvent(eventType: .down, x: 0, y: 5, t: 7, f: 1, pointerType: .pen, pointerId: 0)
//        let eventMove3 = IINKPointerEvent(eventType: .move, x: 5, y: 5, t: 8, f: 1, pointerType: .pen, pointerId: 0)
//        let eventUp3   = IINKPointerEvent(eventType: .up,   x: 5, y: 5, t: 9, f: 1, pointerType: .pen, pointerId: 0)
//
//        var hEvents = [eventDown1, eventMove1, eventUp1,
//                       eventDown2, eventMove2, eventUp2,
//                       eventDown3, eventMove3, eventUp3]
//        try! editor?.pointerEvents(&hEvents, count: hEvents.count, doProcessGestures: false)

        try! editor?.pointerDown(CGPoint(x: 0, y: 0),  at: 1, force: 1, type: .pen, pointerId: 0, error: nil)
        try! editor?.pointerMove(CGPoint(x: 0, y: 10), at: 2, force: 1, type: .pen, pointerId: 0)
        try! editor?.pointerUp(CGPoint(x: 0, y: 10), at: 3, force: 1, type: .pen, pointerId: 0)

        // right
        try! editor?.pointerDown(CGPoint(x: 5, y: 0), at: 4, force: 1, type: .pen, pointerId: 0, error: nil)
        try! editor?.pointerMove(CGPoint(x: 5, y: 10), at: 5, force: 1, type: .pen, pointerId: 0)
        try! editor?.pointerUp(CGPoint(x: 5, y: 10), at: 6, force: 1, type: .pen, pointerId: 0)

        // middle
        try! editor?.pointerDown(CGPoint(x: 0, y: 5), at: 7, force: 1, type: .pen, pointerId: 0, error: nil)
        try! editor?.pointerMove(CGPoint(x: 5, y: 5), at: 8, force: 1, type: .pen, pointerId: 0)
        try! editor?.pointerUp(CGPoint(x: 5, y: 5), at: 9, force: 1, type: .pen, pointerId: 0)
    }

    func createPackage(packageName: String) -> IINKContentPackage? {
        // Create a new content package with name
        let urlWithExtension = FileManager.default.temporaryDirectory
            .appendingPathComponent("com.company.product.handwriting", isDirectory: true)
            .appendingPathComponent(packageName, isDirectory: false)
            .appendingPathExtension("iink")

        try! FileManager.default.createDirectory(at: urlWithExtension.deletingLastPathComponent(), withIntermediateDirectories: true, attributes: nil)
        let fullPath = urlWithExtension.absoluteString.replacingOccurrences(of: "file://", with: "").decomposedStringWithCanonicalMapping
        guard let engine = engine else { return nil }
        var resultPackage: IINKContentPackage?
        do {
            resultPackage = try engine.createPackage(fullPath)
        } catch {
            do {
                resultPackage = try engine.openPackage(fullPath)
            } catch {
                resultPackage = nil
            }
        }
        do {
            currentPart = try resultPackage?.createPart("Text")
        } catch {
            currentPart = nil
        }
        return resultPackage
    }

}

  However, when processStrokes() gets called, it always prints "" (the empty string). I think I'm missing something but I don't know what.

Is my current understanding correct?

A "package" is a sorted collection of parts (acc. to the docs), i.e. assuming one uses documents it would be the whole document.

A "part" is whatever the user writes in one piece, e.g. a word or a complete sentence.

A "block" is some sub-thingy of a part, maybe a letter, maybe a number, maybe a part of a word (or a letter?), I usually don't care about this.

===

In the example code above a create a package for everything the engine should recognize (because I don't want the engine to have the context of the anything else), just feed some strokes and get a word (or list of candidates) out. I don't get any errors logged so I assume my setup is correct (it would log a message if it didn't have any recognition-assets (I use analyzer, en_US, math and shape), right?). I find it really hard to debug this because I don't see how I can get the points of an editor (to see if I fed them correctly into it), etc.

Any help is very much appreciated, thank you! 

Hi Thomas,

Thank you very much for your explanation and your pointers! After I adjusted the code according to your remarks it works perfectly fine! :)