How to integrate TensorflowJs and Unity by Creating a simple guessing game

This is a Tensorflow.js tutorial for integrating a javascript library with Unity, above is the vid and below you will find some of the steps followed.

1. 1.

   Pre-reqs

   • Have Unity installed
   • Have node.js installed
2. 2.

   Create the game using the Unity editor

   
3. 3.

   Integrate unity with a javascript library like tensorflow.js - Calling a javascript library from unity

   • Create a class in unity with the methods that you already have in javascript

     • > Real "GetPrediction" method for using in webgl

         
       [DllImport("__Internal")]
       public static extern int GetPrediction(int valueToPredict);
         
       

     • > Debug "GetPrediction" method for using inside the editor

         
       #if UNITY_EDITOR
           public static int GetPrediction(int valueToPredict) {
               Debug.Log("Called GetPrediction");
       
               return 1;
           }
       #else
           //Real Method here
       #endif
         
       

   • Create a jslib file inside Assets/Plugins/WebGL with the "GetPrediction" method to be called

       
     mergeInto(LibraryManager.library, {
       //...
       GetPrediction: function (valueToPredict) {
         //Your logic here
         return window.PublicInterface.getPrediction(valueToPredict);
       }
     });
       
     

4. 4.

   Integrate unity with a javascript library like tensorflow.js - Calling a unity C# method from javascript

   • Create a jslib file inside Assets/Plugins/WebGL with the "SendMessage" call referencing the name for the game object inside the scene and the method to be called

       
     mergeInto(LibraryManager.library, {
       //...
     
       //SendMessage('Game object name on scene', 'Method inside one of the components');
       SendMessage('PublicInterface', 'TrainingDone');
     
       //...
     });
       
     

   • Create a class in unity with the methods that you already have in javascript

     • > "TrainingDone" C# method for calling from javascript

         
       public class UnityInterfaceController : MonoBehaviour {
           public void TrainingDone() {
               //More code
           }
       }
         
       

   • Have a game object on the scene with the UnityInterfaceController monobehavior component attached to it
5. 5.

   Create a node.js application to handle the integration with tensorflow.js

   • Change to the directory where the npm project is

       
     cd ./Assets/WebGLTemplates/TensorflowJsProjects/1.SimpleSequence
       
     

   • Install all the dependencies

       
     npm install
       
     

   • Start the compilation process

       
     npm run dev
       
     

   • Modify any .ts file and find the generated .js file inside ./Builds/WebGL/js/
6. 6.

   Resources

   Github link for the code

   Unity + Tensorflow.js Simple Guessing Game page

   Better Unity Webgl Template

   WebGL: Interacting with browser scripting

   Unity

   What is Unity

   Node.js