General questions

Differentiating between gestures and text

I'm using both the gesture and text recognizers and am noticing that there are (legitimate) cases where both recognizers return results for a given stroke. For example, drawing a single top-bottom vertical stroke could be the number 1, letter l, or a top-bottom gesture.

I thought I might be able to recognize the difference by looking at strokes but it turns out the gesture and text recognizers return completely different stroke information, in particular, the ids do not correlate, which seems unfortunate. For example, the single vertical stroke returns the following gesture results:

  

{
 "version": "3",
 "id": "MainBlock",
 "type": "Gesture",
 "gestures": [ {
   "type": "top-bottom",
   "strokes": [ {
     "id": "ps0",
     "bounding-box": {
      "x": 91,
      "y": 79,
      "width": 8,
      "height": 100
     }
    } ],
   "bounding-box": {
    "x": 91,
    "y": 79,
    "width": 8,
    "height": 100
   }
  } ]
}

 And the text recognizer:

 

{
 "type": "Text",
 "bounding-box": {
  "x": 90,
  "y": 78,
  "width": 10,
  "height": 102
 },
 "label": "l",
 "words": [ {
   "label": "l",
   "candidates": [ "l", "I", "1", "/", "\\" ],
   "bounding-box": {
    "x": 90,
    "y": 78,
    "width": 10,
    "height": 102
   },
   "items": [ {
     "timestamp": "2024-09-12 16:26:58.951000",
     "X": [ 97, 96, 95, 95, 94, 94, 93, 93,
      93, 92, 92, 92, 91, 91, 91, 91,
      91, 91, 91, 91, 92, 92, 92, 92,
      93, 93, 93, 94, 94, 94, 94, 95,
      95, 95, 95, 96, 96, 96, 96, 97,
      97, 97, 98, 98, 98, 98, 98, 98,
      99, 99, 99, 99, 99, 99, 99, 99,
      99, 99, 99, 99, 98, 98, 98, 97,
      97, 96, 96 ],
     "Y": [ 81, 80, 80, 79, 79, 80, 80, 81,
      82, 83, 85, 86, 88, 90, 92, 95,
      97, 100, 103, 106, 109, 112, 115, 118,
      121, 124, 127, 130, 133, 136, 139, 141,
      144, 146, 149, 151, 153, 155, 157, 158,
      160, 161, 163, 164, 165, 166, 167, 168,
      169, 170, 171, 172, 173, 174, 175, 176,
      177, 178, 179, 178, 178, 177, 176, 176,
      175, 174, 173 ],
     "F": [ 1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1, 1, 1, 1, 1, 1,
      1, 1, 1 ],
     "T": [ 0, 8, 12, 13, 17, 24, 28, 30,
      34, 36, 37, 39, 41, 43, 45, 47,
      48, 50, 52, 54, 56, 58, 60, 61,
      63, 65, 67, 69, 71, 72, 74, 76,
      78, 80, 82, 84, 85, 87, 89, 91,
      93, 95, 96, 98, 100, 102, 104, 106,
      108, 109, 111, 113, 115, 119, 120, 124,
      128, 132, 139, 172, 176, 178, 181, 183,
      185, 189, 193 ],
     "type": "stroke",
     "id": "0000000001008500ff00"
    } ]
  } ],
 "version": "3",
 "id": "MainBlock"
}

So the stroke has an id of ps0 in the gesture results and an id of 0000000001008500ff00 in the text results.

So clearly I can't match the two via stroke id. Bounding box is the obvious choice, although it's not exactly the same either.

Are there any recommendations for how to best approach this?

Thank you!

 


Dear Shannonholl,


Thank you for your question.


The Gesture Recognizer is dedicated to the recognition of gestures without any text context.

So, in your case, Bounding box may be the obvious choice. You may refer to our "write to type" example:

Although it is an Android sample, it may help you in your integration.

Please, also note that depending on your use case, Recognizer objects may not be the most suitable objects to use: if you want to perform gesture recognition with text context, OffScreenEditor or Editor may be better choices.


Best regards,


Gwenaëlle

OK - thank you!

Login or Signup to post a comment