Application Tool Kit

Answered

Neosmartpen v2 integration

 Hi, i'm trying to integrate the neosmartpen into your handwriting recognition, i wondered if there was some examples available for windows.


Best Answer
Dear Manuel,

thank you for contacting us.

Currently, we do not have any code sample to use the ATK with the neosmartpen.

To my knowledge, you should be able to find a SDK to extract the strokes (x and y coordinates) from the pen. Then, you can use the "addStroke" API to provide the ATK with these strokes (AddStroke should be called for each stroke).

Let us know if you have further questions.

Best regards,

Olivier

Dear Manuel,

currently, if the candidate you expect is not listed, there is no other alternative than erase and re-write.

Normally, if writing in handprinted, even out of lexicon terms are likely to be recognized with a pretty good accuracy.

Best regards,

Olivier
Dear Olivier,
Thanks again for the support.

If i want to pick an alternate word, how should i do?
I've set my WordCandidatesList to 5, so that i have some alternatives to choose from, then i set my widget text to the word i want to change (let's say it's "ciao" for now), then i call the WordCandidates with 0 as index (i only want one word at the time to be changed, not the whole phrase, so i tought that 0 as an index should be fine).
I peeked into the CandidateInfo and i get the following:
completition:
labels: ciao (wich is the content of the widget)
sorces: it_IT etc.


How should i proceed?

Dear Manuel,

Thank you for the update.

Regarding the way our technology works, actually, each language is trained based on many ink samples collected in the corresponding language. We do not have a continuous learning, based on each user. Indeed, we tried such continuous learning, and it did not bring significative improvement.

Nevertheless, indeed, if the user writes a word not belonging to our lexicons (e.g. personal abreviations), adding these to a lexicon will help increase the accuracy.

Best regards,

Olivier
Dear Olivier,

I tried with the pen methods, but the recognizing didn't seem accurate as with the addStroke method and when i tried to switch from a field to another the widget crashes with an unhandled exception, i had some guesses why it did that, but i haven't tested yet.
I've actualy kept the addStroke method for the pen inputs, when i switch field i store the text and then set the field to the old value + the value of ontextchanged listener, now it works fine.

I have a few questions about the recognizer learning:
How does it learns?
If my user selects another word candidates, will it learn from the choice my user did? After a Clear(), if i set widget.text to something i recognized earlier(wich would probably be a string) and then i select the alternative candidates, what should i expect? Is it possible to make post-recognize choise affect the learner?
Let's say my user doesn't find the alternative candidate it wants, can i add the user custon word into the alternative list for future recognition session? Or should i add to an userlexicon, even if it's an alredy known word by the default lexicon? Does the userlexicon saves himself after a recognition, or should i store it and load when i configure the widget?

Dear Manuel,


currently, such behavior rather looks like the addStroke is not properly done, or not properly working.


Instead, can you please try using the penDown, penMove and penUp API, as explained to Yang Jingwei above?


Best regards,


Olivier

Dear Olivier,

It doesn't seems to recognize correctly, nearly everything i write get recognized as a dot, or an a at best, if i set the widget visible and i track how it works visualy it doens't seem to scroll.
I guess that when i call the scrollto(index) function (with index equals to widget.text.lenght), the widget should visually scroll until the last letter is on the borded left, but it doesn't, the only noticeable thing is a cursor that appear at the first letter but nothing more.
I tought that could be wrong the index, so i tried with a +1 or a -1, but i get the same result no matter what.
Dear Manuel,

currently, the latest way you proceed is the solution we would recommend.

When you say you you have weird stuff, van you please explain further?

Thank you,

Best regards,

Olivier
Dear Olivier,

I have multiple fields to write in, i can choose wich field is to write in based on my position on paper, but i have some problems.
I actualy have only one widget, wich i clear everytime i start to write in another field.

If i start to write in the first field, then in the second it goes all smoothly, but if i want to write again in the first field i have some issues:
If i just add the recognized word in the "ontextchanged" it adds every step the recognizer does.
ontextchanged(widget w, string text, bool intermediate){
txt = text;
field.text += txt;
}

If i try to add the previous text into the widget, it takes my writing in between the previous text
if(oldfield != newfield){
widget.clear();
widget.text = field.text;
}

i tought i could scroll the widget, sending the old text to the left and start writing in a "blank" space, but it doesn't seems to work, making the recognize do weird stuff:
if(oldfield != newfield){
widget.clear();
widget.text = field.text;
widget.scrollTo(widget.text.lenght);
}
Dear Manuel,

thank you for the update.

Feel free to contact us if you have any other question.

Best regards,

Olivier
Dear Olivier,
Thanks as always for the support, i've actualy resolved this issue (wich was more of a misunderstanding on how sltw works, i think) by checking the "ontextchanged" function.

The widget in the xaml is now set to "not visible" (i don't need to see the pen input on sceen, for that i have the physical paper, or a canvas), then i've added an output wich shows everytime what the recognized text is.
Now for everystroke i add, the output change accordingly to what the widget recognize, with a nice "realtime" recognize effect.
Dear Manuel,

Currently, how do you implement the widget?

Indeed, normally, the recognition starts each time a stroke is added, there is not API to tell to start the recognition?

For this reason, I am facing difficulties figuring out your use-case. Indeed, you should need to do a click to start the recognition.

Best regards,

Olivier

 Dear Olivier,

Once i've got all my strokes and added to the widget, how can i stop the acquiring and recognize it? Right now i need to click into the widget (wich obviusly makes a dot), then it start recognizing the whole script. Is possible to make the widget recognize one word at time (for example, if the pen is up and it's passed a set amount of time, i want to recognize the word)?

 

Dear Yang Jinwei,

we do not have an ATK version for Windows 7. It only runs on Windows 10 UWP.

Best regards,

Olivier

您好Olivier:

 非常感谢您的回复,问题已经解决了,请问一下Windows版本的MathWidget是否有Win7版本的ATK?如果有能否提供win7版的ATK呢?

Dear Yang Jingwei,

currently, the AddStroke function API is not working properly for the MathWidget on Win10.

Instead, you should use the PenDown, PenMove and PenUp API (it behaves exactly the same way than AddStroke).

This can be done as follows in your code:

private void Button_Click(object sender, RoutedEventArgs e) {
MyMathWidget.PenDown(new MyScript.Atk.Core.CaptureInfo(100, 100, 0.5f, 0, 0));
MyMathWidget.PenMove(new MyScript.Atk.Core.CaptureInfo(150, 100, 0.5f, 0, 8));
...
MyMathWidget.PenMove(new MyScript.Atk.Core.CaptureInfo(300, 300, 0.5f, 0, 120));
MyMathWidget.PenUP(new MyScript.Atk.Core.CaptureInfo(350, 300, 0.5f, 0, 128));
}

Let us know if this helps.

Best regards,

Olivier


1 person likes this