Forums
Using API to Auto-Classify Text from Math
Attached the images since one of them broke in the original post.
Dear Mohammadjdiab,
Thank you for your question.
The automatic classification of text and math with Raw Content that you are referring to is only available with our Raw Content Recognizer on native APIs.
For more details on Raw Content Recognizer, you can refer to https://developer.myscript.com/docs/interactive-ink/3.2/android/advanced/recognizers/#raw-content-recognizer
The API "https://cloud.myscript.com/api/v4.0/iink/batch" that you test is the Web API.
We are currently working on porting automatic classification of text and math with Raw Content on our Web APIs too: we plan to release this feature in the first half of next year.
Best regards,
Gwenaëlle
Thanks Gwenaelle, I will give the native API a try :) I look forward to the web API supporting this feature in the future!
Dear Mohammadjdiab,
Thanks for your feedback.
We will keep you updated as soon as we release the web API supporting this feature!
In the meantime, please feel free to share any updates on your use of Raw Content Recognizer!
Best regards,
Gwenaëlle
Hey Gwenaëlle, I tried out the windows .NET SDK and I am running into some issues with Raw Content again:
- Context: I want to understand MyScript's capability to automatically classify math from text on a digital canvas.
- The UWP Github includes Demo and GetStarted projects
- I tried running both:
- for "Text" it classifies and recognizes everything perfectly, see attached screenshot.
- for "Math" it also classifies and recognizes everything perfectly, see attached screenshot.
- for "Raw Content", the convert button does nothing, see below screenshot:
- Likewise on "Text" only or "Math" only it works perfectly.
- On "Raw Content" the conversion works, and it seems to do good with text, however math is inaccurate, see below screenshot:
Dear Mohammadjdiab,
Thanks for your feedback.
The Raw Content part is very flexible and its behavior depends on its configuration.
The difference between the GetStarted and the Demo is the configuration that is applied to the Raw Content part.
This is the reason why when you try to convert some text in the GetStarted, it does nothing.
In the Demo the configuration is contained into the ink\examples\configurations\interacitivy.json
For further details please refer to:
https://developer.myscript.com/docs/interactive-ink/3.2/reference/configuration/#raw-content
Regarding your math question in the Demo, for the moment the auto classification is not available for math in Raw Content, so you must manually select the math strokes (for instance with the lasso), then you must call the SetSelectionType method to force the recognition of these strokes as math. For more details, please refer to:
Best regards,
Gwenaëlle
Dear Gwenaëlle,
Thank you for your responses and for taking the time to assist me. I appreciate your explanations regarding the Raw Content Recognizer and its capabilities.
However, I’m seeking some clarification to ensure I fully understand the functionality. Initially, it was mentioned that automatic classification of text and math is available with the Raw Content Recognizer on native APIs. However, your latest response seems to suggest that auto-classification is not available for math in Raw Content and that strokes must be manually selected and classified using the SetSelectionType method.
Could you please confirm if automatic classification of math versus text without user intervention is possible at all—whether through the native API, the web API, or any other approach within your SDK? Automatic classification is a critical feature for my use case, and understanding the SDK’s current capabilities will help me determine how best to proceed.
Thank you again for your support and clarification. I look forward to hearing from you.
Hello,
Thank you for your update!
Automatic classification of math is available with Raw Content Recognizers, whereas our Demo relies on Editor objects.
The difference between both objects is the following one:
The recognizers aim at performing real-time incremental recognition on transient ink.
They are intended to address use cases like input-method writing solutions based on iink SDK, where your application performs the strokes capture and rendering. They simplify iink SDK integration when the ink is simply an input medium for digital content. For more details, please refer to https://developer.myscript.com/docs/interactive-ink/3.2/overview/recognizers/
The editor object allows you to perform rendering driven interactivity: The principle of rendering-driven interactivity is to delegate the display management to MyScript iink SDK so that it can handle the entire rendering canvas and perform all interactivity functions for you. For more details, please refer to https://developer.myscript.com/docs/interactive-ink/3.2/concepts/ink-capture/
Today the Editor object only supports manual classification of math within raw content with the SetSelectionType method.
Best regards,
Gwenaëlle
That makes sense, thanks Gwenaëlle!
How would you recommend I get started with coding the raw content recognizer? does MyScript have example code I can use as reference? I don't mind moving to android or another SDK if those have example code or easier integration.
We don't have a code sample based on Raw Content Recognizer, but we do have the write to type example based on a Gesture Recognizer combined with a Text Recognizer.
For more details, please refer to:
https://github.com/MyScript/interactive-ink-additional-examples-android
Best regards,
Gwenaëlle
Mohammadjdiab
Hello, I am trying to use the API "https://cloud.myscript.com/api/v4.0/iink/batch" to auto classify and transcribe some of my notes, below is my test case:
I am using the "Raw Content" "contentType" since I have both math and text, hoping to get the API to auto-classify and know which strokes are Math and which are Text. Attached are my input to the API request and the output out of it in the format "application/vnd.myscript.jiix". If I plot the outputted bounding boxes with their classification I notice that it has a single big bounding box and it classifies the math part as well as the text part as "Text" instead of separating the Math from the Text:Am I doing something wrong? how can I have the API auto-classify Text from Math?