The export interface(editor.export_(…)) is very time-consuming in the case of writing a lot of data.
H
Hedy Jiang
started a topic
over 2 years ago
The export interface(editor.export_(…)) is very time-consuming in the case of writing a lot of data.
In the video, I used
incremental recognition in the following way. When the recognition has been
completed, add a word. Through the log, it is found that the recognition takes
very little (about one second), but the program will stay in the export
interface for a long time.
/
*Append each data of stroke to myscrpit.
* When the pen is raised, a stroke will be generated, and then this method will be called.
* @param trail: Equivalent to a stroke, it contains the points and stroke types that make up the stroke
*/
private void processStroke(JniGetTrails trail) {
//get stroke type. 4:erase other:pen
int type = trail.getRecognTrailType();
//all the points that make up the stroke.
List<JniRecognData> list = trail.getRecognData();
//set pointerType.
PointerType pointerType = type == 4 ? PointerType.ERASER : PointerType.PEN;
//down
JniRecognData p = list.get(0);
editor.pointerDown(p.X, p.Y, -1, 0.0f, pointerType, -1);
//move
PointerEvent[] pointerEvents = new PointerEvent[list.size() - 2];
for (int i = 1; i < list.size() - 1; ++i) {
p = list.get(i);
PointerEvent pointerEvent = new PointerEvent(PointerEventType.MOVE, p.X, p.Y, -1, 0.0f, pointerType, -1);
pointerEvents[i - 1] = pointerEvent;
}
editor.pointerEvents(pointerEvents, false);
//up
p = list.get(list.size() - 1);
editor.pointerUp(p.X, p.Y, -1, 0.0f, pointerType, -1);
}
/**
* Called when the user clicks to get the recognition result
* @return result
*/
private String convert() {
Log.e(TAG, "convert");
editor.waitForIdle();
Log.e(TAG, "waitForIdle end,recognition is over");
String res = "";
JiixRawContent jiixRawContent;
try {
/**
*If I have written a lot of data on this page, it will take longer to call this method, why?
*/
String jiixString = editor.export_(editor.getRootBlock(), MimeType.JIIX);
if (jiixString.equals("")) {
Log.e(TAG, "jiixString is null");
return null;
}
jiixRawContent = new Gson().fromJson(jiixString, JiixRawContent.class);
res = new Gson().toJson(jiixRawContent);
} catch (IOException e) {
Log.e(TAG, "exportToText error");
e.printStackTrace();
}
return res;
}
Best Answer
O
Olivier @MyScript
said
over 2 years ago
Dear Hedy,
thank you for the update.
To answer your questions: 1. Is there a way for Text to continue to use stroke erasers?
In a Text part, you shall ensure the following: -You MUST provide the proper timestamp information (the one you get for your device when getting a given pixel) to the editor.pointerDown(...), new PointerEvent(PointerEventType.MOVE...) and the editor.pointerUp(...) functions -you set the ProcessGestures parameter of editor.pointerEvents to true: editor.pointerEvents(pointerEvents, true); Otherwise, our solution will not ne able to detect gestures (whatever the contentPart you are using).
If you want to convert a point from milimeters to pixels, this is done as follows: Transform tr= editor->getRenderer()->getViewTransform(); Point myPoint(x,y); //coordinates in milimeters tr.apply(myPoint); //coordinates in pixels
You can then use this information to highlight the word or term you are searching for.
Let me know if this helps.
Best regards,
Olivier
NB: please find an example of a JIIX export, which as you can see can be parsed as a JSON
Based on the videos, I see you are using an Eink screen. Which one exactly are you using? Indeed, as you know, our solution is rather demanding, and depending on the hardware, processing time can vary a lot.
Now, I have the following questions: -Which part are you using? Text, Text Document, Raw Content...? -When requesting convert, do you simply "waitForIdle" and export? I am also a bit puzzled when you say the export takes time? Usually, the export is straightforward, while the waitForIdle is the function that can take long time. -How do you trigger your "processStroke" function? At each pen up, to add the new stroke? Or you add again all the strokes to the editor? -Are you always reproducing this behavior when you have a full page of notes? Or is it an isolated case? -Also, do you have the possibility to add the points as soon as they are written, rather than adding per stroke (i.e. do you have the possibility to proceed as we do in the handleOnTouchForPointer function of the InputController.java file)? In our case, the handleOnTouchForPointer is triggered at each motion event.
Last, if this doesn't help, would it be possible to have more code (ideally your project), so that we may better understand and provide with a more accurate answer.
Thank you,
Best regards,
Olivier
H
Hedy Jiang
said
over 2 years ago
-Which part are you using? Text, Text Document, Raw Content...? Raw Content,Because I need to support search, export text, and also need to support stroke eraser.
just takes time to export, because the incremental recognition used just adds a word, now we first make sure that the recognition has been completed, and then repeat the call to export, I will print out the time-consuming information, the screenshot is as follows :
cost time:
H
Hedy Jiang
said
over 2 years ago
How do you trigger your "processStroke" function? At each pen up, to add the new stroke? Or you add again all the strokes to the editor?
Due to the particularity of the scene, I have no way to transmit each point to the recognizer in real time like InputController.java. I can only use the stroke as the smallest unit to obtain all points of each stroke in real time when writing notes, and then append them to the recognizer. I have verified this point, it is the same as the real-time transmission point efficiency，because both are incremental recognition, the former adds each point in real time, and the latter adds all points of the stroke in real time.
O
Olivier @MyScript
said
over 2 years ago
Dear Hedy,
thank you for the precision.
Currently, the reasin simply adding "this" takes 16 seconds is because you are using a "Raw Content" part.
Indeed, the Raw Content, such as the Digram part have a 2 steps process:
-First, the iink runs a "Text/non-text" extraction process
-This process allows to have Text parts (on which the iink will the Text recognition) and Shapes or other parts
The problem is that the more ink is entered, the longer the text/non-text takes ; even-though you simply add again few strokes, it can take a lot of time, which is the current case.
From this, our recommendation is that you use a Text part in the current case, so that the Text/non-text process will not run again when simply adding the "this" ink.
If you decide to remain in a Raw-Content part, nothing can be done to speed up the process.
Best regards,
Olivier
H
Hedy Jiang
said
over 2 years ago
Thanks! I know that Text is faster than RAW export, so I have two questions, 1. Is there a way for Text to continue to use stroke erasers?; 2. Whether Text can achieve search?
O
Olivier @MyScript
said
over 2 years ago
Answer
Dear Hedy,
thank you for the update.
To answer your questions: 1. Is there a way for Text to continue to use stroke erasers?
In a Text part, you shall ensure the following: -You MUST provide the proper timestamp information (the one you get for your device when getting a given pixel) to the editor.pointerDown(...), new PointerEvent(PointerEventType.MOVE...) and the editor.pointerUp(...) functions -you set the ProcessGestures parameter of editor.pointerEvents to true: editor.pointerEvents(pointerEvents, true); Otherwise, our solution will not ne able to detect gestures (whatever the contentPart you are using).
If you want to convert a point from milimeters to pixels, this is done as follows: Transform tr= editor->getRenderer()->getViewTransform(); Point myPoint(x,y); //coordinates in milimeters tr.apply(myPoint); //coordinates in pixels
You can then use this information to highlight the word or term you are searching for.
Let me know if this helps.
Best regards,
Olivier
NB: please find an example of a JIIX export, which as you can see can be parsed as a JSON
Hedy Jiang
The export interface(editor.export_(…)) is very time-consuming in the case of writing a lot of data.
In the video, I used incremental recognition in the following way. When the recognition has been completed, add a word. Through the log, it is found that the recognition takes very little (about one second), but the program will stay in the export interface for a long time.
/ *Append each data of stroke to myscrpit. * When the pen is raised, a stroke will be generated, and then this method will be called. * @param trail: Equivalent to a stroke, it contains the points and stroke types that make up the stroke */ private void processStroke(JniGetTrails trail) { //get stroke type. 4:erase other:pen int type = trail.getRecognTrailType(); //all the points that make up the stroke. List<JniRecognData> list = trail.getRecognData(); //set pointerType. PointerType pointerType = type == 4 ? PointerType.ERASER : PointerType.PEN; //down JniRecognData p = list.get(0); editor.pointerDown(p.X, p.Y, -1, 0.0f, pointerType, -1); //move PointerEvent[] pointerEvents = new PointerEvent[list.size() - 2]; for (int i = 1; i < list.size() - 1; ++i) { p = list.get(i); PointerEvent pointerEvent = new PointerEvent(PointerEventType.MOVE, p.X, p.Y, -1, 0.0f, pointerType, -1); pointerEvents[i - 1] = pointerEvent; } editor.pointerEvents(pointerEvents, false); //up p = list.get(list.size() - 1); editor.pointerUp(p.X, p.Y, -1, 0.0f, pointerType, -1); } /** * Called when the user clicks to get the recognition result * @return result */ private String convert() { Log.e(TAG, "convert"); editor.waitForIdle(); Log.e(TAG, "waitForIdle end,recognition is over"); String res = ""; JiixRawContent jiixRawContent; try { /** *If I have written a lot of data on this page, it will take longer to call this method, why? */ String jiixString = editor.export_(editor.getRootBlock(), MimeType.JIIX); if (jiixString.equals("")) { Log.e(TAG, "jiixString is null"); return null; } jiixRawContent = new Gson().fromJson(jiixString, JiixRawContent.class); res = new Gson().toJson(jiixRawContent); } catch (IOException e) { Log.e(TAG, "exportToText error"); e.printStackTrace(); } return res; }Dear Hedy,
thank you for the update.
To answer your questions:
1. Is there a way for Text to continue to use stroke erasers?
In a Text part, you shall ensure the following:
-You MUST provide the proper timestamp information (the one you get for your device when getting a given pixel) to the editor.pointerDown(...), new PointerEvent(PointerEventType.MOVE...) and the editor.pointerUp(...) functions
-you set the ProcessGestures parameter of editor.pointerEvents to true: editor.pointerEvents(pointerEvents, true);
Otherwise, our solution will not ne able to detect gestures (whatever the contentPart you are using).
2. Whether Text can achieve search?
What is the exact purpose of your Search functionality? Do you want to highlight a word you are searching for?
If so, you can proceed as follows:
-You export as JIIX
-You can then get the "label" (i.e. the full text result)
-Then, for each "words", you browse the JIIX to search for the label (or the candidates)
-For each word, you can get the "bounding-box" in milimeters (in order to convert it in Pixels), you shall use the transform matrix: https://developer.myscript.com/docs/interactive-ink/latest/android/fundamentals/zooming-and-scrolling/#view-transformation-matrix
-And the API: https://developer.myscript.com/refguides/interactive-ink/android/2.0/com/myscript/iink/graphics/Transform.html
If you want to convert a point from milimeters to pixels, this is done as follows:
Transform tr= editor->getRenderer()->getViewTransform();
Point myPoint(x,y); //coordinates in milimeters
tr.apply(myPoint); //coordinates in pixels
You can then use this information to highlight the word or term you are searching for.
Let me know if this helps.
Best regards,
Olivier
NB: please find an example of a JIIX export, which as you can see can be parsed as a JSON
{
"type": "Text",
"bounding-box": {
"x": 42.232914,
"y": 26.9399986,
"width": 135.773331,
"height": 16.1816673
},
"label": "hello, this is a test",
"words": [ {
"label": "hello",
"candidates": [ "hello", "fello", "hellor", "hella", "tello" ],
"bounding-box": {
"x": 42.232914,
"y": 26.9399986,
"width": 42.6400032,
"height": 14.9116669
},
"items": [ {
"timestamp": "2022-09-06 08:49:11.598000",
"X": [ 43.232914, ...],
"Y": [ 36.1949997, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000000001005701ff00"
}, {
"timestamp": "2022-09-06 08:49:13.601000",
"X": [ 66.3045807, ...],
"Y": [ 30.4799976, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "000001000100a100ff00"
}, {
"timestamp": "2022-09-06 08:49:14.615000",
"X": [ 76.2529144, ...],
"Y": [ 37.0416641, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000020001006900ff00"
} ]
}, {
"label": ",",
"candidates": [ ",", ";", "/", "i", "I" ],
"bounding-box": {
"x": 85.2012482,
"y": 36.253334,
"width": 5.17499542,
"height": 6.86833191
},
"items": [ {
"timestamp": "2022-09-06 08:49:16.478000",
"X": [ 89.3762436, ...],
"Y": [ 37.253334, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000030001001d00ff00"
} ]
}, {
"label": " "
}, {
"label": "this",
"candidates": [ "this", "This", "thir", "thies", "thits" ],
"bounding-box": {
"x": 97.6895828,
"y": 27.3633308,
"width": 23.5899963,
"height": 12.1600037
},
"items": [ {
"timestamp": "2022-09-06 08:49:17.829000",
"X": [ 98.6895828, ...],
"Y": [ 38.5233345, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000040001007700ff00"
}, {
"timestamp": "2022-09-06 08:49:18.649000",
"X": [ 109.061249, ...],
"Y": [ 28.3633308, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "000005000100bf00ff00"
}, {
"timestamp": "2022-09-06 08:49:20.063000",
"X": [ 115.622917, 115.622917, 115.622917 ],
"Y": [ 31.3266659, 31.1149979, 31.1149979 ],
"F": [ 0, 0, 0 ],
"T": [ 0, 32, 109 ],
"type": "stroke",
"id": "0000060001000500ff00"
}, {
"timestamp": "2022-09-06 08:49:20.760000",
"X": [ 101.652916, ...],
"Y": [ 33.6549988, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000070001001b00ff00"
} ]
}, {
"label": " "
}, {
"label": "is",
"candidates": [ "is", "in", "ir", "Is", "as" ],
"bounding-box": {
"x": 127.534576,
"y": 30.75,
"width": 11.1016693,
"height": 9.40833282
},
...
} ],
"version": "3",
"id": "MainBlock"
}
- Oldest First
- Popular
- Newest First
Sorted by Oldest FirstHedy Jiang
The link to the test video is below：
https://drive.google.com/drive/folders/12YVrobCSyevnHVLsQmLi5jjhR4tU43Eg?usp=sharing
Olivier @MyScript
Dear Hedy,
thank you for contacting us and your question.
Based on the videos, I see you are using an Eink screen. Which one exactly are you using? Indeed, as you know, our solution is rather demanding, and depending on the hardware, processing time can vary a lot.
Now, I have the following questions:
-Which part are you using? Text, Text Document, Raw Content...?
-When requesting convert, do you simply "waitForIdle" and export? I am also a bit puzzled when you say the export takes time? Usually, the export is straightforward, while the waitForIdle is the function that can take long time.
-How do you trigger your "processStroke" function? At each pen up, to add the new stroke? Or you add again all the strokes to the editor?
-Are you always reproducing this behavior when you have a full page of notes? Or is it an isolated case?
-Also, do you have the possibility to add the points as soon as they are written, rather than adding per stroke (i.e. do you have the possibility to proceed as we do in the handleOnTouchForPointer function of the InputController.java file)? In our case, the handleOnTouchForPointer is triggered at each motion event.
Last, if this doesn't help, would it be possible to have more code (ideally your project), so that we may better understand and provide with a more accurate answer.
Thank you,
Best regards,
Olivier
Hedy Jiang
-Which part are you using? Text, Text Document, Raw Content...?
Raw Content,Because I need to support search, export text, and also need to support stroke eraser.
just takes time to export, because the incremental recognition used just adds a word, now we first make sure that the recognition has been completed, and then repeat the call to export, I will print out the time-consuming information, the screenshot is as follows :
cost time:
Hedy Jiang
How do you trigger your "processStroke" function? At each pen up, to add the new stroke? Or you add again all the strokes to the editor?
Due to the particularity of the scene, I have no way to transmit each point to the recognizer in real time like InputController.java. I can only use the stroke as the smallest unit to obtain all points of each stroke in real time when writing notes, and then append them to the recognizer. I have verified this point, it is the same as the real-time transmission point efficiency，because both are incremental recognition, the former adds each point in real time, and the latter adds all points of the stroke in real time.
Olivier @MyScript
Dear Hedy,
thank you for the precision.
Currently, the reasin simply adding "this" takes 16 seconds is because you are using a "Raw Content" part.
Indeed, the Raw Content, such as the Digram part have a 2 steps process:
-First, the iink runs a "Text/non-text" extraction process
-This process allows to have Text parts (on which the iink will the Text recognition) and Shapes or other parts
The problem is that the more ink is entered, the longer the text/non-text takes ; even-though you simply add again few strokes, it can take a lot of time, which is the current case.
From this, our recommendation is that you use a Text part in the current case, so that the Text/non-text process will not run again when simply adding the "this" ink.
If you decide to remain in a Raw-Content part, nothing can be done to speed up the process.
Best regards,
Olivier
Hedy Jiang
Thanks! I know that Text is faster than RAW export, so I have two questions, 1. Is there a way for Text to continue to use stroke erasers?; 2. Whether Text can achieve search?
Olivier @MyScript
Dear Hedy,
thank you for the update.
To answer your questions:
1. Is there a way for Text to continue to use stroke erasers?
In a Text part, you shall ensure the following:
-You MUST provide the proper timestamp information (the one you get for your device when getting a given pixel) to the editor.pointerDown(...), new PointerEvent(PointerEventType.MOVE...) and the editor.pointerUp(...) functions
-you set the ProcessGestures parameter of editor.pointerEvents to true: editor.pointerEvents(pointerEvents, true);
Otherwise, our solution will not ne able to detect gestures (whatever the contentPart you are using).
2. Whether Text can achieve search?
What is the exact purpose of your Search functionality? Do you want to highlight a word you are searching for?
If so, you can proceed as follows:
-You export as JIIX
-You can then get the "label" (i.e. the full text result)
-Then, for each "words", you browse the JIIX to search for the label (or the candidates)
-For each word, you can get the "bounding-box" in milimeters (in order to convert it in Pixels), you shall use the transform matrix: https://developer.myscript.com/docs/interactive-ink/latest/android/fundamentals/zooming-and-scrolling/#view-transformation-matrix
-And the API: https://developer.myscript.com/refguides/interactive-ink/android/2.0/com/myscript/iink/graphics/Transform.html
If you want to convert a point from milimeters to pixels, this is done as follows:
Transform tr= editor->getRenderer()->getViewTransform();
Point myPoint(x,y); //coordinates in milimeters
tr.apply(myPoint); //coordinates in pixels
You can then use this information to highlight the word or term you are searching for.
Let me know if this helps.
Best regards,
Olivier
NB: please find an example of a JIIX export, which as you can see can be parsed as a JSON
{
"type": "Text",
"bounding-box": {
"x": 42.232914,
"y": 26.9399986,
"width": 135.773331,
"height": 16.1816673
},
"label": "hello, this is a test",
"words": [ {
"label": "hello",
"candidates": [ "hello", "fello", "hellor", "hella", "tello" ],
"bounding-box": {
"x": 42.232914,
"y": 26.9399986,
"width": 42.6400032,
"height": 14.9116669
},
"items": [ {
"timestamp": "2022-09-06 08:49:11.598000",
"X": [ 43.232914, ...],
"Y": [ 36.1949997, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000000001005701ff00"
}, {
"timestamp": "2022-09-06 08:49:13.601000",
"X": [ 66.3045807, ...],
"Y": [ 30.4799976, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "000001000100a100ff00"
}, {
"timestamp": "2022-09-06 08:49:14.615000",
"X": [ 76.2529144, ...],
"Y": [ 37.0416641, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000020001006900ff00"
} ]
}, {
"label": ",",
"candidates": [ ",", ";", "/", "i", "I" ],
"bounding-box": {
"x": 85.2012482,
"y": 36.253334,
"width": 5.17499542,
"height": 6.86833191
},
"items": [ {
"timestamp": "2022-09-06 08:49:16.478000",
"X": [ 89.3762436, ...],
"Y": [ 37.253334, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000030001001d00ff00"
} ]
}, {
"label": " "
}, {
"label": "this",
"candidates": [ "this", "This", "thir", "thies", "thits" ],
"bounding-box": {
"x": 97.6895828,
"y": 27.3633308,
"width": 23.5899963,
"height": 12.1600037
},
"items": [ {
"timestamp": "2022-09-06 08:49:17.829000",
"X": [ 98.6895828, ...],
"Y": [ 38.5233345, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000040001007700ff00"
}, {
"timestamp": "2022-09-06 08:49:18.649000",
"X": [ 109.061249, ...],
"Y": [ 28.3633308, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "000005000100bf00ff00"
}, {
"timestamp": "2022-09-06 08:49:20.063000",
"X": [ 115.622917, 115.622917, 115.622917 ],
"Y": [ 31.3266659, 31.1149979, 31.1149979 ],
"F": [ 0, 0, 0 ],
"T": [ 0, 32, 109 ],
"type": "stroke",
"id": "0000060001000500ff00"
}, {
"timestamp": "2022-09-06 08:49:20.760000",
"X": [ 101.652916, ...],
"Y": [ 33.6549988, ...],
"F": [ 0, ...],
"T": [ 0, ...],
"type": "stroke",
"id": "0000070001001b00ff00"
} ]
}, {
"label": " "
}, {
"label": "is",
"candidates": [ "is", "in", "ir", "Is", "as" ],
"bounding-box": {
"x": 127.534576,
"y": 30.75,
"width": 11.1016693,
"height": 9.40833282
},
...
} ],
"version": "3",
"id": "MainBlock"
}