AI machine learning for Void Expanse Drones of the 9sMODs series.

[ninekorn]

I am working on a version of AI learning/machine Learning for Void Expanse so that I can use that on the drones to make their movement better in game and also for their decision making. I am barely getting started in this though. I am currently working on a version of it based on the series by Sebastian Lague here : https://www.youtube.com/watch?v=bVQUSndDllU
He built his tutorial in Python, although that wasn't why I decided to learn a bit of Python a couple of days ago, it was actually for exporting TONS of objects and build XML files for Void Expanse all at the same time. I've decided to "try" and translate his Python tutorial of Episode 1 and 2 mathematical equations to c# inside of Unity and started playing with that. But I am encountering some "obstacles". For instance, when Sebastian Lague explains the mathematical equations in episode 1 and 2, there is something that I am missing. I don't know how he is using the "output" of the Neural Network. Is he "re-feeding" it into the neural network as inputs in order for the neural network to complete a certain number of iterations until it reaches the goal?

If anyone has any idea of how Machine Learning works, and has really good explanations that can be understood for a beginner, I would be really interested in asking questions. here is what I have so far.. Although the script is able to get to the goal whether it is closer to 1 or closer to 0 which is going to be perfect for me when using the Dot product for the movement in Void Expanse, I have no clue if this qualifies as a Neural Network.

Code: [Select]

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;

public class SC_Neural_Network : MonoBehaviour
{
    double[] input;
    double[] weightsOne;
    double[] hiddenLayer;
    double[] lastHiddenLayer;
    double[] biasValueOne;
    double[] biasValueTwo;
    double[] output;
    double[] lastOutput;

    double[] currentOutputError = new double[2];
    double[] lastOutputError = new double[2];

    double[] currentHiddenLayerOutputError = new double[3];
    double[] lastHiddenLayerOutputError = new double[3];
    double[] lastHiddenPercentChanged = new double[3];
    double[] currentHiddenPercentChanged = new double[3];

    double[] weightsTwo;
    System.Random rand = new System.Random();

    double[] lastPercentChanged = new double[2];
    double[] currentPercentChanged = new double[2];

    public void Start()
    {
        input = new double[2];

        weightsOne = new double[6];
        hiddenLayer = new double[3];
        lastHiddenLayer = new double[3];
        biasValueOne = new double[3];
        biasValueTwo = new double[2];
        output = new double[2];
        weightsTwo = new double[6];
        lastOutput = new double[2];


        neuralNet(0.001, 100); // right now, can only use 0.001 or 0.999
    }

    double oriGoal = 0;

    int starter = 0;
    int finisher = 0;
    int someCounter = 0;

    //right now only working for input of 2 and hidden layer of 3 and output of 2 and biasOne 3 and biasTwo 2 and WeightsOne 6 and WeightsTwo 6
    private void neuralNet(double goal,int counterMax)
    {
        oriGoal = goal;

        while (someCounter < counterMax && finisher == 0)
        {
            if (starter == 0)
            {
                input[0] = rand.NextDouble();
                input[1] = rand.NextDouble();
                for (int c1 = 0; c1 < weightsOne.Length; c1++)
                {
                    weightsOne[c1] = rand.NextDouble();
                    weightsTwo[c1] = rand.NextDouble();
                }
                for (int c1 = 0; c1 < biasValueOne.Length; c1++)
                {
                    biasValueOne[c1] = rand.NextDouble();
                }
                for (int c1 = 0; c1 < biasValueTwo.Length; c1++)
                {
                    biasValueTwo[c1] = rand.NextDouble();
                }
                starter = 1;
            }
            else
            {
                input[0] = output[0];
                input[1] = output[1];
            }

            for (int c = 0; c < hiddenLayer.Length; c++)
            {
                hiddenLayer[c] = ActivationFunction(input[0] * weightsOne[c * 2 + 0] + input[1] * weightsOne[c * 2 + 1] + biasValueOne[c]);
            }

            for (int c = 0; c < output.Length; c++)
            {
                output[c] = ActivationFunction(hiddenLayer[0] * weightsTwo[c * 3 + 0] + hiddenLayer[1] * weightsTwo[c * 3 + 1] + hiddenLayer[2] * weightsTwo[c * 3 + 2] + biasValueTwo[c]);
       
                if (output[c] <= goal && goal >= 0.999)
                {
                    currentOutputError[c] = 1 - output[c];
                    lastOutputError[c] = 1 - lastOutput[c];

                    var totalDiffInError = Math.Abs(currentOutputError[c] - lastOutputError[c]);
                    currentPercentChanged[c] = totalDiffInError / currentOutputError[c];
                    var diffToGoal = Math.Abs(goal - currentOutputError[c]);

                    if (currentOutputError[c] >= lastOutputError[c])
                    {
                        for (int c1 = 0; c1 < weightsOne.Length; c1++)
                        {
                            currentPercentChanged[c] *= 1.19;

                            double someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            weightsOne[c1] = weightsOne[c1] - someTest;

                            someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            currentPercentChanged[c] *= 1.19;
                            weightsTwo[c1] = weightsTwo[c1] - someTest;
                        }
                    }
                    else if (currentOutputError[c] < lastOutputError[c])
                    {
                        for (int c1 = 0; c1 < weightsOne.Length; c1++)
                        {
                            currentPercentChanged[c] *= 1.19;

                            double someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            weightsOne[c1] = weightsOne[c1] + someTest;

                            someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            currentPercentChanged[c] *= 1.19;
                            weightsTwo[c1] = weightsTwo[c1] + someTest;
                        }
                    }
                }
                else if(output[c] >= goal && goal <= 0.001)
                {
                    //Debug.Log("test00");
                    currentOutputError[c] = 1 - output[c];
                    lastOutputError[c] = 1 - lastOutput[c];

                    var totalDiffInError = Math.Abs(currentOutputError[c] - lastOutputError[c]);
                    currentPercentChanged[c] = totalDiffInError / currentOutputError[c];

                    var diffToGoal = Math.Abs(goal - currentOutputError[c]);


                    if (currentOutputError[c] >= lastOutputError[c])
                    {
                        for (int c1 = 0; c1 < weightsOne.Length; c1++)
                        {
                            currentPercentChanged[c] *= 1.33;
                            double someTest = (currentOutputError[c] * currentPercentChanged[c]);

                            weightsOne[c1] = weightsOne[c1] + someTest;

                            currentPercentChanged[c] *= 1.33;
                            someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            weightsTwo[c1] = weightsTwo[c1] + someTest;
                        }
                    }
                    else if (currentOutputError[c] < lastOutputError[c])
                    {
                        for (int c1 = 0; c1 < weightsOne.Length; c1++)
                        {
                            currentPercentChanged[c] *= 1.33;
                            double someTest = (currentOutputError[c] * currentPercentChanged[c]);
                       
                            weightsOne[c1] = weightsOne[c1] - someTest;

                            currentPercentChanged[c] *= 1.33;
                            someTest = (currentOutputError[c] * currentPercentChanged[c]);
                            weightsTwo[c1] = weightsTwo[c1] - someTest;
                        }
                    }
                    /*else //not used but if the above variable 1.33 is less than 1.19, it breaks the script and i gotta implement this part
                    {

                    }*/
                }
                else if(output[c] <= goal && goal <= 0.001 || output[c] >= goal && goal >= 0.999)
                {
                    finisher = 1;
                    Debug.Log("FINISHED AND REACHED GOAL");
                }

                lastPercentChanged[c] = currentPercentChanged[c];
                lastOutput[c] = output[c];
               
                Debug.Log(output[c]);
            }
            someCounter++;
        }
    }

    double ActivationFunction(double x)
    {
        return (1 / (1 + Mathf.Exp((float)-x)));
    }
}



EDIT: this can hardly be qualified as a neural network except for the variable names in the script. I am far from the goal. Will keep learning and searching for more info on how to implement a working version in c# or javascript.

[ninekorn]

I have a working example for Perceptron and i used it in Unity3D in order for a "perceptron" to learn whether to turn right or turn left. I just can't wait to bring that in Void Expanse. It's a really EASY to understand Perceptron program, i believe for ANY beginners working on AILearning. In fact, the best one i came across even if no tutorial. It's just that simple:

Source:

https://github.com/Brollof/SimplePerceptron

I use a modifiable array of 360 length (360 degrees modifiable to 720 degrees for half degrees and so on) and whatever the position of the waypoint is, the perceptron learns fast enough and keeps its weights and data in memory so it will be possible with Global Variables to keep in memory the "learned" things for the drones, which means it will be safe for when server "debug_reinit" happens.

[ninekorn]

my first git project. https://github.com/ninekorn/SCCSCompass .  A machine learning compass. it can also be a "direction bullseye" for turrets to aim at and mix that with the asset "Projectile Helper" of Unity3D and you've got a very nasty turret defense system for any games.

using the https://github.com/Brollof/SimplePerceptron

my compass is not perfect but it is very good still in it's current state. i am currently modifying my compass in order to have drones using machine learning in order to get into formation. it's getting there slowly but surely.

[ninekorn]

English: using Brollof's linear Simple Perceptron found here https://github.com/Brollof/SimplePerceptron , i have made a gimbal machine learning system for the x/y/z axis.
Français: J'utilise le Perceptron Brollof Simple et linéaire que vous pouvez trouver ici https://github.com/Brollof/SimplePerceptron et j'ai construit un systèmes de gimballes "machine learning" pour les axes x/y/z.

my machine learning compass and gimbal projects are available here: https://github.com/ninekorn/SCCSCompass
mon projet de machine learning boussole et gimballes sont disponibles ici: https://github.com/ninekorn/SCCSCompass



current known issues: the dot product by itself to slow down high speed rotating gimbal needles is not enough. i will soon lerp this or use both dot product and lerp.

it will be coming in my 9sMODs series for Void Expanse for the ship formations and hopefully more. it's already "working" for the rotation left/right and thrust front/back but i've got the wrong settings and i am missing some very important variables so it's currently failing. i need more time to work on this.



//not available yet: other project using parts of this system:
English: A planet type "voxel minecraft" shooter game and a machine learning gimbal that follows the players movements🙂. i built this game. not available anywhere else. work in progress
Français: Un shooter de planètes type "voxel minecraft" ainsi qu'un système de machine learning gimballes🙂. c'est moi qui a construit ce jeu. présentement non disponible ailleurs. en développement .