Forums

I have been trying to convert x-y coordinates obtained from the wacom clipboard my handwritten text-

i have converted it to PointerEvents as-

follows as how i obtained pointer events array from clipboard strokes

pointerEvents2[k] = new PointerEvent(PointerEventType.CANCEL, saved_strokes.get(i)[j].x, saved_strokes.get(i)[j].y, saved_strokes.get(i)[j].timestampInMillis, 1.0f, PointerType.PEN, 1);

editor.pointerEvents(pointerEvents2,false);
editor.waitForIdle();
try {
    editor.export_(null, MimeType.TEXT);
} catch (IOException e) {
    e.printStackTrace();
}

*Editor have been initialised and all but i still get error at editor.pointerEvents(pointerEvents2,false);

Please help me out where it got wrong.

I also tried same text points conversion on MyScriptJs but it isn't accurate with words its doing fine with the characters-

https://myscript.github.io/MyScriptJS/examples/v4/pointer_events.html

Dear Olivier

Thanks for your time, I did converted the pointer events as you suggested but it results the same let me post the error too if it might help.

above is pointerEvents with down, move and up actions

File file = new File(getFilesDir(), packageName);

                ContentPackage pkg = engine.createPackage(file);

                ContentPart part = pkg.createPart("Text");

                editor.setPart(part);

                editor.pointerEvents(pointerEvents2, false);

                editor.waitForIdle();

                try {

                    editor.export_(null, MimeType.TEXT);

                } catch (IOException e) {

                    e.printStackTrace();

                } 

but i still get the error at NativeFunctions.pointerEvents

here is error screen-

Also at the MyScriptJS

https://myscript.github.io/MyScriptJS/examples/v4/pointer_events.html

if we feed  pointer events stroke by stroke it converts right but all strokes are fed via pointerEvents it converts with inaccuracy as i mentioned earlier so i need to know if there is a way to feed it with complete pointerEvents array and it provide accurate corresponding text

Thanks with regards

Jai Singh

Hello Olivier,

I am attaching the class and few snapshots of data in arrays .

below are the savedstroke data for reference

// Copyright MyScript. All rights reserved.

package com.hcflab.mbroadcast.factory.myscript.iink.getstarted;

import android.media.Image;
import android.os.Bundle;
import android.os.Environment;
import android.support.annotation.Nullable;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.view.View;
import android.widget.ImageButton;
import android.widget.ImageView;

import com.hcflab.mbroadcast.factory.AppGlobals;
import com.hcflab.mbroadcast.factory.R;
import com.myscript.iink.Configuration;
import com.myscript.iink.ContentPackage;
import com.myscript.iink.ContentPart;
import com.myscript.iink.ConversionState;
import com.myscript.iink.Editor;
import com.myscript.iink.Engine;
import com.myscript.iink.IEditorListener;
import com.myscript.iink.MimeType;
import com.myscript.iink.PointerEvent;
import com.myscript.iink.PointerEventType;
import com.myscript.iink.PointerType;
import com.myscript.iink.uireferenceimplementation.EditorView;
import com.myscript.iink.uireferenceimplementation.InputController;
import com.wacom.cdlcore.RawPoint;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.util.ArrayList;

public class MainActivity extends AppCompatActivity implements View.OnClickListener
{
  private static final String TAG = "MainActivity";

  private Engine engine;
  private ContentPackage contentPackage;
  private ContentPart contentPart;
  private EditorView editorView;
  ImageView convert, convert2;
  DocumentController documentController;
    ArrayList<RawPoint[]> saved_strokes = new ArrayList<RawPoint[]>();
  @Override
  protected void onCreate(@Nullable Bundle savedInstanceState)
  {
    super.onCreate(savedInstanceState);

    ErrorActivity.installHandler(this);

    engine = IInkApplication.getEngine();

    // configure recognition
    Configuration conf = engine.getConfiguration();
    String confDir = "zip://" + getPackageCodePath() + "!/assets/conf";
    conf.setStringArray("configuration-manager.search-path", new String[]{confDir});
    String tempDir = getFilesDir().getPath() + File.separator + "tmp";
    conf.setString("content-package.temp-folder", tempDir);

    setContentView(R.layout.myscript_main);

    editorView = findViewById(R.id.editor_view);
    convert = findViewById(R.id.convert);
    convert2 = findViewById(R.id.convert2);
    editorView.setEngine(engine);

    /*
      o implement, the preview, you should get the text result from the jiix file, as follows:
  -first, ensure you have an iink Editor properly initialized, such as in the code samples
          -Create a content package: ContentPackage pkg = engine.createPackage(« myfile.iink »)
          -Create a text content part: ContentPart part = pkg.createPart(« Text »)
          -Set this part to the editor: Editor.setPart(part)
          -Feed the editor with your ink using editor.pointerEvents(eventArray, false), where eventArray is your ink converted into pointer events (pointerDown, pointerMove+, pointerUp) and false means the gestures where deactivated
          -wait for the recognition to end: editor.waitForIdle()
          -export in JIIX format: editor.export_(MimeType.JIIX)

          => You can then get words with corresponding input range. This way, to a give ink, you can match the corresponding words.
    */



    convert2.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            Editor editor = editorView.getEditor();
            final File path =
                    Environment.getExternalStoragePublicDirectory
                            (
                                    File.separator + getResources().getString(R.string.app_name) + File.separator + "myscript" + File.separator
                            );

            // Make sure the path directory exists.
            if(!path.exists())
            {
                // Make it, if it doesn't exit
                path.mkdirs();
            }
            final File file2 = new File(path, "conversion.txt");

            saved_strokes.clear();
            int k = 0;
            saved_strokes =AppGlobals.saved_strokes2;
            int size=0, size2=0;
            for (int z=0;z<saved_strokes.size();z++){
                size+=saved_strokes.get(z).length;
            }

            String json ="";
            PointerEvent[] pointerEvents2 = new PointerEvent[size + 1];
            for (int i=0; i<saved_strokes.size(); i++){
                for (int j=0; j<saved_strokes.get(i).length; j++){

                    if (j==0) {
                        pointerEvents2[k] = new PointerEvent(PointerEventType.DOWN, saved_strokes.get(i)[j].x, saved_strokes.get(i)[j].y, saved_strokes.get(i)[j].timestampInMillis, 1.0f, PointerType.PEN, 1);
                    }else if (j==(saved_strokes.get(i).length-1)) {
                        pointerEvents2[k] = new PointerEvent(PointerEventType.UP, saved_strokes.get(i)[j].x, saved_strokes.get(i)[j].y, saved_strokes.get(i)[j].timestampInMillis, 1.0f, PointerType.PEN, 1);
                    }else {
                        pointerEvents2[k] = new PointerEvent(PointerEventType.MOVE, saved_strokes.get(i)[j].x, saved_strokes.get(i)[j].y, saved_strokes.get(i)[j].timestampInMillis, 1.0f, PointerType.PEN, 1);
                    }
                    k+=1;
                }
            }

            try {

                ContentPackage pkg = engine.createPackage(file2);
                ContentPart part = pkg.createPart("Text");
                editor.setPart(part);
                editor.pointerEvents(pointerEvents2, false);
                editor.waitForIdle();
                try {
                    editor.export_(null, MimeType.TEXT);
                } catch (IOException e) {
                    e.printStackTrace();
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    });

    final Editor editor = editorView.getEditor();
    editor.addListener(new IEditorListener()
    {
      @Override
      public void partChanging(Editor editor, ContentPart oldPart, ContentPart newPart)
      {
        // no-op
      }

      @Override
      public void partChanged(Editor editor)
      {
        invalidateOptionsMenu();
        invalidateIconButtons();
      }

      @Override
      public void contentChanged(Editor editor, String[] blockIds)
      {
        invalidateOptionsMenu();
        invalidateIconButtons();
      }

      @Override
      public void onError(Editor editor, String blockId, String message)
      {
        Log.e(TAG, "Failed to edit block \"" + blockId + "\"" + message);
      }
    });

    setInputMode(InputController.INPUT_MODE_FORCE_PEN); // If using an active pen, put INPUT_MODE_AUTO here

    String packageName = "File2.iink";
    File file = new File(getFilesDir(), packageName);
    try
    {
      contentPackage = engine.createPackage(file);
      contentPart = contentPackage.createPart("Diagram"); // Choose type of content (possible values are: "Text Document", "Text", "Diagram", "Math", and "Drawing")
    }
    catch (IOException e)
    {
      Log.e(TAG, "Failed to open package \"" + packageName + "\"", e);
    }
    catch (IllegalArgumentException e)
    {
      Log.e(TAG, "Failed to open package \"" + packageName + "\"", e);
    }

    setTitle("Type: " + contentPart.getType());

    // wait for view size initialization before setting part
    editorView.post(new Runnable()
    {
      @Override
      public void run()
      {
        editorView.getRenderer().setViewOffset(0, 0);
        editorView.getRenderer().setViewScale(1);
        editorView.setVisibility(View.VISIBLE);
        editor.setPart(contentPart);
      }
    });

    findViewById(R.id.button_input_mode_forcePen).setOnClickListener(this);
    findViewById(R.id.button_input_mode_forceTouch).setOnClickListener(this);
    findViewById(R.id.button_input_mode_auto).setOnClickListener(this);
    findViewById(R.id.button_undo).setOnClickListener(this);
    findViewById(R.id.button_redo).setOnClickListener(this);
    findViewById(R.id.button_clear).setOnClickListener(this);

    invalidateIconButtons();
  }

    public void writeToFile2(String data)
    {
        // Get the directory for the user's public pictures directory.
        final File path =
                Environment.getExternalStoragePublicDirectory
                        (
                                File.separator + getResources().getString(R.string.app_name) + File.separator + "myscript" + File.separator
                        );

        // Make sure the path directory exists.
        if(!path.exists())
        {
            // Make it, if it doesn't exit
            path.mkdirs();
        }
        final File file2 = new File(path, "points.txt");
        if (file2.exists()){
            file2.delete();
        }
        final File file = new File(path, "points.txt");


        // Save your stream, don't forget to flush() it before closing it.

        try
        {
            file.createNewFile();
            FileOutputStream fOut = new FileOutputStream(file,true);
            OutputStreamWriter myOutWriter = new OutputStreamWriter(fOut);
            myOutWriter.append(data);

            myOutWriter.close();

            fOut.flush();
            fOut.close();
        }
        catch (IOException e)
        {
            Log.e("Exception", "File write failed: " + e.toString());
        }
    }

  public String to_string(int [] z){
      if (z.length > 0) {
          StringBuilder nameBuilder = new StringBuilder();

          for (int n : z) {
              nameBuilder.append("'").append(n).append("',");
//              nameBuilder.append("'").append(n.replace("'", "\\'")).append("',");
              // can also do the following
              // nameBuilder.append("'").append(n.replace("'", "''")).append("',");
          }

          nameBuilder.deleteCharAt(nameBuilder.length() - 1);

          return nameBuilder.toString();
      }else{
          return "";
      }
  }


  public String to_stringl(long [] x){
      if (x.length > 0) {
          StringBuilder nameBuilder = new StringBuilder();

          for (long n : x) {
              nameBuilder.append("'").append(n).append("',");
//              nameBuilder.append("'").append(n.replace("'", "\\'")).append("',");
              // can also do the following
              // nameBuilder.append("'").append(n.replace("'", "''")).append("',");
          }

          nameBuilder.deleteCharAt(nameBuilder.length() - 1);

          return nameBuilder.toString();
      }else{
          return "";
      }
  }

  @Override
  protected void onDestroy()
  {
    editorView.setOnTouchListener(null);
    editorView.close();

    if (contentPart != null)
    {
      contentPart.close();
      contentPart = null;
    }
    if (contentPackage != null)
    {
      contentPackage.close();
      contentPackage = null;
    }


    // IInkApplication has the ownership, do not close here
    engine = null;

    super.onDestroy();
  }

  @Override
  public boolean onCreateOptionsMenu(Menu menu)
  {
    getMenuInflater().inflate(R.menu.myscript_menu, menu);

    MenuItem convertMenuItem = menu.findItem(R.id.menu_convert);
    convertMenuItem.setEnabled(true);

    return super.onCreateOptionsMenu(menu);
  }

  @Override
  public boolean onOptionsItemSelected(MenuItem item)
  {
    switch (item.getItemId())
    {
      case R.id.menu_convert:
      {
        Editor editor = editorView.getEditor();
        ConversionState[] supportedStates = editor.getSupportedTargetConversionStates(null);
        if (supportedStates.length > 0)
          editor.convert(null, supportedStates[0]);
        return true;
      }
      default:
      {
        return super.onOptionsItemSelected(item);
      }
    }
  }

  @Override
  public void onClick(View v)
  {
    switch (v.getId())
    {
      case R.id.button_input_mode_forcePen:
        setInputMode(InputController.INPUT_MODE_FORCE_PEN);
        break;
      case R.id.button_input_mode_forceTouch:
        setInputMode(InputController.INPUT_MODE_FORCE_TOUCH);
        break;
      case R.id.button_input_mode_auto:
        setInputMode(InputController.INPUT_MODE_AUTO);
        break;
      case R.id.button_undo:
        editorView.getEditor().undo();
        break;
      case R.id.button_redo:
        editorView.getEditor().redo();
        break;
      case R.id.button_clear:
        editorView.getEditor().clear();
        break;
      default:
        Log.e(TAG, "Failed to handle click event");
        break;
    }
  }

  private void setInputMode(int inputMode)
  {
    editorView.setInputMode(inputMode);
    findViewById(R.id.button_input_mode_forcePen).setEnabled(inputMode != InputController.INPUT_MODE_FORCE_PEN);
    findViewById(R.id.button_input_mode_forceTouch).setEnabled(inputMode != InputController.INPUT_MODE_FORCE_TOUCH);
    findViewById(R.id.button_input_mode_auto).setEnabled(inputMode != InputController.INPUT_MODE_AUTO);
  }

  private void invalidateIconButtons()
  {
    Editor editor = editorView.getEditor();
    final boolean canUndo = editor.canUndo();
    final boolean canRedo = editor.canRedo();
    runOnUiThread(new Runnable()
    {
      @Override
      public void run()
      {
        ImageButton imageButtonUndo = (ImageButton) findViewById(R.id.button_undo);
        imageButtonUndo.setEnabled(canUndo);
        ImageButton imageButtonRedo = (ImageButton) findViewById(R.id.button_redo);
        imageButtonRedo.setEnabled(canRedo);
        ImageButton imageButtonClear = (ImageButton) findViewById(R.id.button_clear);
        imageButtonClear.setEnabled(contentPart != null);
      }
    });
  }
}

Dear Olivier,

thanks for your solution on MyScriptJS text.guides.enable to false did solved my problem, now conversion is quite accurate.

>I am nevertheless puzzled by your "saved_strokes" variable. What does it look like? 

and on java saved_strokes is the array_list of strokes array

>When doing "size+=saved_strokes.get(z).length;" what is the value of the size variable?

here each item represent a stroke and each stroke contains a collection of x-y coordinates which you can see here 98, 112, 46.. so so size+=saved_strokes.get(z).length it give sum of all x-y points of all stroke just to initialise the pointerEvents array so size for above case will be 540 as instance.

>When filling your pointerEvents2 array, is it properly filled? Indeed, I am thinking it may not be properly filled. Can you please check it carefully?

pointsEvents2 array is accurate according to me as the size also equals the 540 equal to above points and same data now fed on MyScriptJS it does return accurate conversion, so i am not sure what is wrong here.

I do appreciate your time and effort please help this crash solution

Regards 

Jai Singh Hada

Hello Olivier,

I also want  to ask how to get multi line text as recognition result as for now it returns single line text regardless input strokes were multi line.

Is there any way to make inputTextType to multi_line_text kind of something as i couldn't find for MyScriptJS.

is textInputMode i found related can it work?

Thanks and regards

Jai Singh Hada

Dear Olivier,

Thanks for all your prompt support it does solved all my issues.

Thanks with regards,

Jai Singh Hada

Hello,

I am trying the same but text conversion is not accurate for single character and also for a word.

ex- if i write P on wacom bamboo writing pad then it recognise it as either ' s ' or ' g '.

need your help to short out this.

Thanks & Regards

Pankaj Tyagi