iink SDK on Device

Answered

application freezes

Hello! When I use the batch recognition mode, the application freezes and the CPU occupies 100%, and no results appear. This is just an accidental phenomenon and not necessarily present.


Best Answer

Dear Hedy Jiang,

thank you for the update.

Currently, when the behavior occurs, is it on a specific device? Or specific Android release?

Is there enough memory left? What is the CPU usage before using the iink SDK?

Also, which version of the iink SDK are you using? The 1.3, the 1.4?

Indeed, with the below information, we are not able to investigate further. The API you are pointing us is what we call the batch-mode or non-interactive offscreen-mode. With this API, the strokes are not processed as soon as they are written but all at the same time. The consequence is that the more strokes are entered, the longer it takes to process.

Also, if "abnormal strokes" are entered (e.g. very long strokes that fill-in the screen, full page of drawing instead of Text...), then it can happen that the CPU reaches 100% usages for a while.

What we recommend to prevent from such use-cases is to chek the length of strokes, and reject strokes that are too long (e.g. strokes that are longer than the screen size...).

You can also use the incremental API instead of the batch API ; with the incremental API, the strokes are processed as soon as they are addeed, thus the result can be gotten pretty immediately. You can find more information about the incremental API in our documentation: https://developer.myscript.com/docs/interactive-ink/1.4/android/fundamentals/editing/#incremental-input

Also, as by default our code samples are using the incremental API, you can refer to the InputController.java file of the uireferenceimplementation. As you can see, when and ACTION_DOWN motion event occurs, the editor.pointerDown() function is called ; when an ACTION_UP the editor.pointerUp() is called , and for each ACTION_MOVE motionEvent, the editor.pointerMove() is called.

Filtering the strokes and using the incremental mode should prevent from bottlenecks that may occur.

Best regards,

Olivier


Answer

Dear Hedy Jiang,

thank you for the update.

Currently, when the behavior occurs, is it on a specific device? Or specific Android release?

Is there enough memory left? What is the CPU usage before using the iink SDK?

Also, which version of the iink SDK are you using? The 1.3, the 1.4?

Indeed, with the below information, we are not able to investigate further. The API you are pointing us is what we call the batch-mode or non-interactive offscreen-mode. With this API, the strokes are not processed as soon as they are written but all at the same time. The consequence is that the more strokes are entered, the longer it takes to process.

Also, if "abnormal strokes" are entered (e.g. very long strokes that fill-in the screen, full page of drawing instead of Text...), then it can happen that the CPU reaches 100% usages for a while.

What we recommend to prevent from such use-cases is to chek the length of strokes, and reject strokes that are too long (e.g. strokes that are longer than the screen size...).

You can also use the incremental API instead of the batch API ; with the incremental API, the strokes are processed as soon as they are addeed, thus the result can be gotten pretty immediately. You can find more information about the incremental API in our documentation: https://developer.myscript.com/docs/interactive-ink/1.4/android/fundamentals/editing/#incremental-input

Also, as by default our code samples are using the incremental API, you can refer to the InputController.java file of the uireferenceimplementation. As you can see, when and ACTION_DOWN motion event occurs, the editor.pointerDown() function is called ; when an ACTION_UP the editor.pointerUp() is called , and for each ACTION_MOVE motionEvent, the editor.pointerMove() is called.

Filtering the strokes and using the incremental mode should prevent from bottlenecks that may occur.

Best regards,

Olivier

Dear  Olivier @MyScript,

 

The identified code comes from the demo on your github. Secondly, this is an accidental phenomenon. I have tried many times but it can't be reproduced. At present, our products use myscript and there are more than 7000 users now. Only this case is found. What I mean is to let you check 'editor.pointerEvents(arrayListPoint.toArray(new PointerEvent[0]), false); editor.waitForIdle();' whether there are any abnormalities in the recognition library called here that cause an infinite loop. Find it and fix it.

Dear Hedy,


Indeed, there is nothing that looks wrong in the below code.


Can you please provide us with a project that allows to reproduce?


Thank you,


Best regards,


Olivier

Thanks, this is my key code:

 

String jiixString;
try {
    Log.i(TAG,"point size :"+arrayListPoint.size()+"   wait result");
    editor.pointerEvents(arrayListPoint.toArray(new PointerEvent[0]), false);
    editor.waitForIdle();
    jiixString = editor.export_(editor.getRootBlock(), MimeType.JIIX, exportParams);
    Log.i(TAG,"recognition end ");
} catch (Exception e) {
    e.printStackTrace();
    Log.i(TAG, "error!");
    return;
}

 I am pretty sure that the amount of data I entered is not large (about a dozen Chinese characters), and after waiting for more than two hours, the program neither output error log nor output 'recognition end', he should be stopped at waitFordle(). Woke up.

Dear Hedy,


thank you for contacting us and your question.


Currently, what has to be kept in mind is that the recognition process is a very demanding process, and the more strokes you add, the longer it takes. When processing ink, it is common to the CPU reaches 100%.


Now, when using the batch mode, in order to make sure the result is available, you shall call the waitForIdle() functin after the editor.pointerEvents(events.toArray(newPointerEvent[0]),false);


You shall then proceed as follows:

events.add(new PointerEvent().down(150.f, 126.f));
events.add(new PointerEvent().move(151.f, 126.f));
...
events.add(new PointerEvent().up(209.f, 128.f));
...
editor.pointerEvents(pointerEvents, false);
editor.export_(...);

Best regards,


Olivier