iink SDK on Device

Answered

MyScript in SwiftUI

I am trying to add MyScript to an app which is built almost entirely in swiftui. I got the demo working and then I created my own app using swiftui. I built something similar to the demo. The first indication of a problem was that if I try to do addBlock it throws an error, which only means it doesn't put the text I am importing on the screen. Yellow text below is the point at which it fails. I did the same exact thing in the demo and it worked.

    func makeUIViewController(context: UIViewControllerRepresentableContext<HandwrittingViewController>) -> EditorViewController {

        let editorViewController = EditorViewController()

        

        if let appDelegate = UIApplication.shared.delegate as? AppDelegate {

            if (appDelegate.engine == nil)

            {

                print("Certificate error \(appDelegate.engineErrorMessage ?? "No ERROR")")

            }

            editorViewController.engine = appDelegate.engine

        }

        

        editorViewController.inputMode = .forcePen

        

        do {

            if let package = try createPackage(packageName: "New") {

                let myText = NSAttributedString(string: "Hello, MyScript")

                

                try editorViewController.editor.part = package.getPartAt(0)

                try editorViewController.editor.addBlock(CGPoint(x: 0,y: 0), type: "Text", mimeType: IINKMimeType.text, data: myText.string)

            }

        } catch {

            print("Error while creating package : " + error.localizedDescription)

        }

        

        return editorViewController

    }

image


When I go to write in the view the app crashes with lots of errors. 

I can share my project if you like. The purpose of my project it to prototype the functionality that will be incorporated into the final app.


Best Answer

Hello James,


According to your code snippet, I think the part creation is missing in your code.


As far as I understand your use case, either you want to use a 'Text' part or a 'Text Document' part. The main difference bewteen both of them is that a 'Text' part hosts a single text content block, wheras a 'Text Document' part can host several content blocks. (please check this section for further details)


But my guess is that you want to use a 'Text' part with a single text block.

So you should first create your Text part before adding the text with the import method.


            if let part = try resultPackage?.createPart("Text") /* Options are : "Diagram", "Drawing", "Math", "Text Document", "Text" */ {

                self.title = "Type: " + part.type

            }


Then you will be able to import your text with the import_ method.


Or if you want to use a 'Text Document' part, you would create your part 

            if let part = try  resultPackage?.createPart("Text Document") /* Options are : "Diagram", "Drawing",  "Math", "Text Document", "Text" */ {

                self.title = "Type: " + part.type

            }

Then you could use the addBlock method to add your text.


I recommand you reading this part of the documentation that describes how you can import text according to your part type.


Regards,


Gwenaëlle


Thank you for your update!

I forgot I modified the createPart call early in my test. This fixed it.
Answer

Hello James,


According to your code snippet, I think the part creation is missing in your code.


As far as I understand your use case, either you want to use a 'Text' part or a 'Text Document' part. The main difference bewteen both of them is that a 'Text' part hosts a single text content block, wheras a 'Text Document' part can host several content blocks. (please check this section for further details)


But my guess is that you want to use a 'Text' part with a single text block.

So you should first create your Text part before adding the text with the import method.


            if let part = try resultPackage?.createPart("Text") /* Options are : "Diagram", "Drawing", "Math", "Text Document", "Text" */ {

                self.title = "Type: " + part.type

            }


Then you will be able to import your text with the import_ method.


Or if you want to use a 'Text Document' part, you would create your part 

            if let part = try  resultPackage?.createPart("Text Document") /* Options are : "Diagram", "Drawing",  "Math", "Text Document", "Text" */ {

                self.title = "Type: " + part.type

            }

Then you could use the addBlock method to add your text.


I recommand you reading this part of the documentation that describes how you can import text according to your part type.


Regards,


Gwenaëlle

Thats it. It took me some effort to get the folders to be references. but it worked.


Now all that remains is to add an initial text block when the view is loaded.  The addBlock call as it is worked in the GetStartSwift example but it is failing here.



        editorViewController.inputMode = .forcePen

        

        do {

            if let package = try createPackage(packageName: "New") {

                let myText = NSAttributedString(string: "Hello, MyScript")

                

                

                try editorViewController.editor.part = package.getPartAt(0)

                try editorViewController.editor.addBlock(CGPoint(x: 0,y: 0), type: "Text", mimeType: IINKMimeType.text, data: myText.string)

            }

        } catch {

            print("Error while creating package : " + error.localizedDescription)

        }


2019-12-20 23:21:01.251341-0600 SwiftHandwritting[33229:2340956] Error, -[IINKEditor addBlock:type:mimeType:data:error:]::428 add block failed: could not create block at (x,y)

Error while creating package : The operation couldn’t be completed. Error, -[IINKEditor addBlock:type:mimeType:data:error:]::428 add block failed: could not create block at (x,y)

2019-12-20 23:21:01.486086-0600 SwiftHandwritting[33229:2341170] Metal API Validation Enabled

Dear James,

thank you for your ticket.

Based on the logs, you have the "CONFIGURATION_BUNDLE_NOT_FOUND" error, which means you are not able to load the en_US.conf file.

In order to solve this, can you please check the following:
-The myscript-iink-recognition-text-en_US.zip package was properly deployed, e.g. in the "recognition-assets" directory.
-The en_US.conf file should then be deployed into the "recognition-assets/conf" directory
-The en_US-ak-cur.res and en_US-lk-text.res files should then be deployed into the "recognition-assets/resources/en_US" directory.
-You shall then ensure the configurationPath and "configuration-manager.search-path" configuration are properly set.


In swift, this is done as follows:
// Configure the iink runtime environment
let configurationPath = Bundle.main.bundlePath.appending("/recognition-assets/conf")
do {
try engine.configuration.setStringArray([configurationPath], forKey:"configuration-manager.search-path") // Tells the engine where to load the recognition assets from.
}
catch {
print("Should not happen, please check your resources assets : " + error.localizedDescription)
return nil
}

On Android:
String confDir = "zip://" + getPackageCodePath() + "!/assets/conf";
conf.setStringArray("configuration-manager.search-path", new String[]{confDir});


After re-build, the en_US.conf file and assets (en_US-ak-cur.res and en_US-lk-text.res files) should be properly loaded.

Let us know if this doesn't work.

Best regards,

Olivier

https://developer-support.myscript.com/support/discussions/topics/16000027765 is the thread.  I've sent them a sample project to see if they can recreate the issue.

Can I get the link to your thread Rob?

It looks very similar to the issue I'm having James.  I think (and I could be wrong) that it's caused by a change implemented in the update from iOS11 to iOS12 or iOS13: earlier versions of iOS would let a background process update a foreground process (in my case, I had some functions that queried an SQL database and then updated a text field), but after an update iOS, I had to wrap the update into a DispatchQueue.Main.Async {} otherwise the app would be unable to update the text field and crash as Apple had stopped that happening.  I'm guessing that something similar needs done to interactiveInk, but I couldn't figure out where which is why I've popped a question in to them.  

If I compiled my app for an earlier version of iOS, it reverted to the earlier behaviour and ran, but that isn't really suitable for a long term fix going forward, which is why I could do with myscript doing something with their code.

Here are just a few of the errors:

2019-12-19 00:09:25.672909-0600 SwiftHandwritting[24731:1571487] onError: CONFIGURATION_BUNDLE_NOT_FOUND: error: no such configuration bundle en_US

2019-12-19 00:09:25.702539-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.704927-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.705257-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.707068-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.709137-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.710001-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.710187-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

2019-12-19 00:09:25.710383-0600 SwiftHandwritting[24731:1571487] [Assert] Cannot be called with asCopy = NO on non-main thread.

Hi James,

What errors are you seeing in the debugger when it crashes?  I'm in a similar situation in that I've built the demo app and it works fine, but I'm trying to prototype it for use in my own app, and it crashes as soon as I write anything on the screen. I've got a thread on here if you want more details.


Rob


Added one more line to mimic the Storyboard setup in the demo which allowed me to see drawing occur.  It still crashes after I write.



        let editorViewController = EditorViewController()

        editorViewController.view.isUserInteractionEnabled = true