Forums

IInk becomes unresponsive when we do the following operations on Android:

We write Japanese text using complex characters and frequently use Undo instead of the Eraser to delete text, write more text, delete some of it again with undo, and repeat this for a while.

At some point, Iink becomes unresponsive and it is impossible to write strokes (no strokes are drawn when writing with the stylus) or save the document. The part type is a Text Document.

Did you ever see such a problem, or have any idea about what might be causing it and how it could be avoided?

Thank you!

Thank you for looking into this.

Unfortunately, the problem was experienced by our client and we were unable to reproduce it ourselves, so we don't have logs. They seem to be able to reproduce it quite quickly (often in less than 1 minuntes) but only when using an older device (Galaxy Tab S3). Regarding other actions, they do some document scrolling but it is not clear from the video they sent us

Hello Olivier,

We gathered more details about this issue where iink stops being responsive while we are writing.

- The problem can happen even when we don't perform Undo/Redo.

- The problem is more frequent with a low spec tablet. We tested with Galaxy Tab 3 and Galaxy Tab S6 Light and the problem was more frequent with Galaxy Tab 3.

- When we become unable to write, all iink functionality (saving for example) appears to stop working. Our app's buttons and other functionality unrelated to iink are still responding, so it seems like only iink is stuck.

- Even though iink is stuck, there is no exception or other error being output.

- We use iink's IEditorListener.contentChanged() to Export the contents of TextBlocks that were modified. They are exported using Editor.export_() using both the Text and the JIIX formats.

- We also have a background task saving the current document using  ContentPackage.saveToTemp() every 10 seconds.

The differences in the handwriting part between the GetStarted demo app and our app are that we export changes in contentChanged() and save regularly to temp.

Is it possible that if we save to temp or export a text block while at the same time inputting strokes, it can cause iink to become stuck? If that's the case, is it necessary to wait or block to avoid performing such operations at the same time?

Thank you for your help!

Thank you Olivier.

We understand that exporting JIIX data with a high frequency can slow down the application, but after implementing it, we find that the speed is bearable.

Our problem is that iink suddenly hangs. 1 second we are writing normally with no noticeable lag, and the next, strokes completely stop being drawn on the screen in response to stylus input. Even after waiting for more than 1 minute, there is no stroke output on the screen.

Is the hanging problem caused by the high load of exporting a lot of JIIX data?

Hi Olivier,

We investigated more about this problem where iink become unresponsive and understood a few things.

- If we try to zoom while iink is unresponsive, it throws an exception and the app crashes. This seems to be the same problem as described here:  https://developer-support.myscript.com/support/discussions/topics/16000024603

- There are cases where editor.pointerDown() is performed in InputController.handleOnTouchForPointer() but editor.pointerUp() or editor.pointerCancel() is not.

To fix the 2 problems above, we made the following changes to the reference implementation's InputController.java.

- We added a list to manage pointerIds.

- After editor.pointerDown() is called, we add the pointerId to the list.

- After editor.pointerUp() or editor.pointerCancel() is called, we delete the pointerId for the list.

- Before calling editor.pointerDown(), we check the list and if it contains the same pointerId we call editor.pointerCancel().

- When the event MotionEvent.ACTION_CANCEL comes, we call editor.pointerCancel() for every pointerId in the list.

The following shows the changes we made to the code:

--- InputController.java      2021-08-30 18:23:16.504268700 +0900
+++ InputController.java      2021-09-06 16:52:06.536356500 +0900
@@ -17,6 +17,7 @@ import com.myscript.iink.PointerType;
 import com.myscript.iink.graphics.Point;

 import java.util.EnumSet;
+import java.util.HashMap;

 public class InputController implements View.OnTouchListener, GestureDetector.OnGestureListener
 {
@@ -33,6 +34,7 @@ public class InputController implements
   private final GestureDetectorCompat gestureDetector;
   private IInputControllerListener _listener;
   private final long eventTimeOffset;
+  private HashMap<Integer, PointerType> pointerMap = new HashMap<>();

   public InputController(Context context, IRenderTarget renderTarget, Editor editor)
   {
@@ -107,10 +109,16 @@ public class InputController implements

     int historySize = event.getHistorySize();

+    boolean isPointerDown = pointerMap.containsKey(pointerId);
+
     switch (actionMask)
     {
       case MotionEvent.ACTION_POINTER_DOWN:
       case MotionEvent.ACTION_DOWN:
+        if(isPointerDown) {
+          editor.pointerCancel(pointerId);
+        }
+        pointerMap.put(pointerId, iinkPointerType);
         editor.pointerDown(event.getX(pointerIndex), event.getY(pointerIndex), eventTimeOffset + event.getEventTime(), event.getPressure(), iinkPointerType, pointerId);
         return true;

@@ -138,11 +146,17 @@ public class InputController implements
             pointerEvents[i] = new PointerEvent(PointerEventType.MOVE, event.getHistoricalX(pointerIndex, i), event.getHistoricalY(pointerIndex, i), eventTimeOffset + event.getHistoricalEventTime(i), event.getHistoricalPressure(pointerIndex, i), iinkPointerType, pointerId);
           editor.pointerEvents(pointerEvents, true);
         }
-        editor.pointerUp(event.getX(pointerIndex), event.getY(pointerIndex), eventTimeOffset + event.getEventTime(), event.getPressure(), iinkPointerType, pointerId);
+        if(isPointerDown) {
+          editor.pointerUp(event.getX(pointerIndex), event.getY(pointerIndex), eventTimeOffset + event.getEventTime(), event.getPressure(), iinkPointerType, pointerId);
+          pointerMap.remove(pointerId);
+        }
         return true;

       case MotionEvent.ACTION_CANCEL:
-        editor.pointerCancel(pointerId);
+        for(Integer key : pointerMap.keySet()) {
+          editor.pointerCancel(key);
+        }
+        pointerMap.clear();
         return true;

       default:

 With the changes above, we tested our app for 2 hours and did not experience the problem where iink becomes unresponsive.

However, we would like to check with you to know if the above changes could have a bad effect on the iink library.

Thank you!

We found a way to reproduce the problem with some reliability using the reference application "iink GetStarted" with a Samsung Galaxy Tab S6 Lite (Android 10). It might also be able to use this way with other devices that can detect when the stylus is close to the screen without touching it. 

1. Start the sample app "iink GetStarted"

2. Set the Mode to AUTO

3. Scroll the screen up and down using all 5 fingers.

4. Bring the stylus close to the screen while performing step 3.

5. Confirm that scrolling stops regardless of the movement of the fingers and then remove the fingers from the screen.

6. Write some text with the stylus.

7. Repeat steps 3-6 and after a few repetitions, it becomes impossible to write with the pen.

8. Continue to repeat steps 3-6 and eventually an exception will be thrown (see the attached image).

It also looks like if ACTION_CANCEL is called during a multi-touch editor.pointerCancel() is only called for 1 pointerId in the processing of ACTION_CANCEL in InputController.handleOnTouchForPointer(), but after that, the ACTION_UP / ACTION_CANCEL events are not called for the other pointerIds.

Would it be necessary to call editor.pointerCancel() for every pointerId in ACTION_CANCEL?

Thank you Olivier.

How about the problem with "iink GetStarted" we added above just before you replied?