diff --git a/README.md b/README.md index 8c2d9d9bb3..104ce2d6c3 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,8 @@ If you run into any problems using the ML-Agents toolkit, [submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and make sure to include as much detail as possible. +Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to [let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454). + For any other questions or feedback, connect directly with the ML-Agents team at ml-agents@unity3d.com. diff --git a/docs/Background-Unity.md b/docs/Background-Unity.md index 5a452fbb1d..3884bf7740 100644 --- a/docs/Background-Unity.md +++ b/docs/Background-Unity.md @@ -17,3 +17,5 @@ with the ML-Agents toolkit: * [Physics](https://docs.unity3d.com/Manual/PhysicsSection.html) * [Ordering of event functions](https://docs.unity3d.com/Manual/ExecutionOrder.html) (e.g. FixedUpdate, Update) +* [Prefabs](https://docs.unity3d.com/Manual/Prefabs.html) + diff --git a/docs/Basic-Guide.md b/docs/Basic-Guide.md index 2105fa8c09..5a8f3273f7 100644 --- a/docs/Basic-Guide.md +++ b/docs/Basic-Guide.md @@ -1,7 +1,7 @@ # Basic Guide This guide will show you how to use a pre-trained model in an example Unity -environment, and show you how to train the model yourself. +environment (3D Ball) and show you how to train the model yourself. If you are not familiar with the [Unity Engine](https://unity3d.com/unity), we highly recommend the [Roll-a-ball @@ -42,33 +42,42 @@ TensorFlow files in the Project window under **Assets** > **ML-Agents** > to the `Scripting Define Symbols` for each type of device you want to use (**`PC, Mac and Linux Standalone`**, **`iOS`** or **`Android`**). -![Project Settings](images/project-settings.png) + ![Project Settings](images/project-settings.png) -**Note**: If you don't see anything under **Assets**, drag the -`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window. + **Note**: If you don't see anything under **Assets**, drag the + `UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window. -![Imported TensorFlowsharp](images/imported-tensorflowsharp.png) + ![Imported TensorFlowsharp](images/imported-tensorflowsharp.png) ## Running a Pre-trained Model +We've included pre-trained models for the 3D Ball example. -1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder +1. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Scenes` folder and open the `3DBall` scene file. -2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder - and select the `Game/Platform` prefab. -3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** Brain located in - `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of - the `Ball 3D Agent`. -4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`. - __Note__ : You can modify multiple game objects in a scene by selecting them all at +2. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder. + Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window. + + **Note**: The platforms in the `3DBall` scene were created using the `Platform` prefab. Instead of updating all 12 platforms individually, you can update the `Platform` prefab instead. + + ![Platform Prefab](images/platform_prefab.png) + +3. In the **Project** window, drag the **3DBallLearning** Brain located in + `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window. + + ![3dball learning brain](images/3dball_learning_brain.png) + +4. You should notice that each `Platform` under each `Game` in the **Hierarchy** windows now contains **3DBallLearning** as `Brain`. __Note__ : You can modify multiple game objects in a scene by selecting them all at once using the search bar in the Scene Hierarchy. -5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels` +5. In the **Project** window, click on the **3DBallLearning** Brain located in + `Assets/ML-Agents/Examples/3DBall/Brains`. You should see the properties in the **Inspector** window. +6. In the **Project** window, open the `Assets/ML-Agents/Examples/3DBall/TFModels` folder. -6. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels` - folder to the **Model** field of the **3DBallLearning** Brain. -7. Click the **Play** button and you will see the platforms balance the balls +7. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels` + folder to the **Model** field of the **3DBallLearning** Brain in the **Inspector** window. __Note__ : All of the brains should now have `3DBallLearning` as the TensorFlow model in the `Model` property +8. Click the **Play** button and you will see the platforms balance the balls using the pretrained model. -![Running a pretrained model](images/running-a-pretrained-model.gif) + ![Running a pretrained model](images/running-a-pretrained-model.gif) ## Using the Basics Jupyter Notebook @@ -84,30 +93,26 @@ More information and documentation is provided in the ## Training the Brain with Reinforcement Learning -### Adding a Brain to the training session +### Setting up the enviornment for training To set up the environment for training, you will need to specify which agents are contributing to the training and which Brain is being trained. You can only perform training with a `Learning Brain`. -1. Assign the **3DBallLearning** Brain to the agents you would like to train. - __Note:__ You can assign the same Brain to multiple agents at once : To do so, you can - use the prefab system. When an agent is created from a prefab, modifying the prefab - will modify the agent as well. If the agent does not synchronize with the prefab, you - can hit the Revert button on top of the Inspector. - Alternatively, you can select multiple agents in the scene and modify their `Brain` - property all at once. -2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain - is in the Broadcast Hub. In order to train, you need to toggle the - `Control` checkbox. - -__Note:__ Assigning a Brain to an agent (dragging a Brain into the `Brain` property of +1. Each platform agent needs an assigned `Learning Brain`. In this example, each platform agent was created using a prefab. To update all of the brains in each platform agent at once, you only need to update the platform agent prefab. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder. Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window. In the **Project** window, drag the **3DBallLearning** Brain located in `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window. + + **Note**: The Unity prefab system will modify all instances of the agent properties in your scene. If the agent does not synchronize automatically with the prefab, you can hit the Revert button in the top of the **Inspector** window. + +2. In the **Hierarchy** window, select `Ball3DAcademy`. +3. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Brains` folder and drag the **3DBallLearning** Brain to the `Brains` property under `Braodcast Hub` in the `Ball3DAcademy` object in the **Inspector** window. In order to train, make sure the `Control` checkbox is selected. + + **Note:** Assigning a Brain to an agent (dragging a Brain into the `Brain` property of the agent) means that the Brain will be making decision for that agent. Whereas dragging a Brain into the Broadcast Hub means that the Brain will be exposed to the Python process. The `Control` checkbox means that in addition to being exposed to Python, the Brain will be controlled by the Python process (required for training). -![Set Brain to External](images/mlagents-SetBrainToTrain.png) + ![Set Brain to External](images/mlagents-SetBrainToTrain.png) ### Training the environment @@ -134,7 +139,7 @@ be controlled by the Python process (required for training). Editor"_ is displayed on the screen, you can press the :arrow_forward: button in Unity to start training in the Editor. -**Note**: Alternatively, you can use an executable rather than the Editor to + **Note**: Alternatively, you can use an executable rather than the Editor to perform training. Please refer to [this page](Learning-Environment-Executable.md) for instructions on how to build and use an executable. diff --git a/docs/FAQ.md b/docs/FAQ.md index 614f120289..2a326f2540 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -71,7 +71,7 @@ There may be a number of possible causes: _Solution_: Look into the [log files](https://docs.unity3d.com/Manual/LogFiles.html) generated by the Unity Environment to figure what error happened. -# _Cause_: You have assigned HTTP_PROXY and HTTPS_PROXY values in your +* _Cause_: You have assigned HTTP_PROXY and HTTPS_PROXY values in your environment variables. _Solution_: Remove these values and try again. ## Communication port {} still in use diff --git a/docs/Getting-Started-with-Balance-Ball.md b/docs/Getting-Started-with-Balance-Ball.md index 6c80a3acf4..756aaec8ab 100644 --- a/docs/Getting-Started-with-Balance-Ball.md +++ b/docs/Getting-Started-with-Balance-Ball.md @@ -28,7 +28,7 @@ Let's get started! In order to install and set up the ML-Agents toolkit, the Python dependencies and Unity, see the [installation instructions](Installation.md). -## Understanding a Unity Environment (3D Balance Ball) +## Understanding the Unity Environment (3D Balance Ball) An agent is an autonomous actor that observes and interacts with an _environment_. In the context of Unity, an environment is a scene containing an @@ -53,18 +53,18 @@ to speed up training since all twelve agents contribute to training in parallel. The Academy object for the scene is placed on the Ball3DAcademy GameObject. When you look at an Academy component in the inspector, you can see several properties that control how the environment works. -The **Broadcast Hub** keeps track of which Brains will send data during training, -If a Brain is added to the hub, his data will be sent to the external training +The **Broadcast Hub** keeps track of which Brains will send data during training. +If a Brain is added to the hub, the data from this Brain will be sent to the external training process. If the `Control` checkbox is checked, the training process will be able to -control the agents linked to the Brain to train them. -The **Training** and **Inference Configuration** properties +control and train the agents linked to the Brain. +The **Training Configuration** and **Inference Configuration** properties set the graphics and timescale properties for the Unity application. The Academy uses the **Training Configuration** during training and the **Inference Configuration** when not training. (*Inference* means that the Agent is using a trained model or heuristics or direct control — in other words, whenever **not** training.) -Typically, you set low graphics quality and a high time scale for the **Training -configuration** and a high graphics quality and the timescale to `1.0` for the +Typically, you would set a low graphics quality and timescale to greater `1.0` for the **Training +Configuration** and a high graphics quality and timescale to `1.0` for the **Inference Configuration** . **Note:** if you want to observe the environment during training, you can adjust @@ -72,7 +72,7 @@ the **Training Configuration** settings to use a larger window and a timescale closer to 1:1. Be sure to set these parameters back when training in earnest; otherwise, training can take a very long time. -Another aspect of an environment to look at is the Academy implementation. Since +Another aspect of an environment is the Academy implementation. Since the base Academy class is abstract, you must always define a subclass. There are three functions you can implement, though they are all optional: @@ -88,28 +88,24 @@ environment around the Agents. ### Brain -Brains are assets that exist in your project folder. The Ball3DAgents are connected -to a Brain, for example : the **3DBallLearning**. -A Brain doesn't store any information about an Agent, it just +As of v0.6, a Brain is a Unity asset and exists within the `UnitySDK` folder. These brains (ex. **3DBallLearning.asset**) are loaded into each Agent object (ex. **Ball3DAgents**). A Brain doesn't store any information about an Agent, it just routes the Agent's collected observations to the decision making process and -returns the chosen action to the Agent. Thus, all Agents can share the same -Brain, but act independently. The Brain settings tell you quite a bit about how +returns the chosen action to the Agent. All Agents can share the same +Brain, but would act independently. The Brain settings tell you quite a bit about how an Agent works. -You can create Brain objects by selecting `Assets -> -Create -> ML-Agents -> Brain`. There are 3 kinds of Brains : -The **Learning Brain** is a Brain that uses a Neural Network to take decisions. -When the Brain is checked as `Control` in the Academy **Broadcast Hub**, the -external process will be taking decisions for the agents -and generate a neural network when the training is over. You can also use the +You can create new Brain assets by selecting `Assets -> +Create -> ML-Agents -> Brain`. There are 3 types of Brains. +The **Learning Brain** is a Brain that uses a trained neural network to make decisions. +When the `Control` box is checked in the Brains property under the **Broadcast Hub** in the Academy, the external process that is training the neural network will take over decision making for the agents +and ultimately generate a trained neural network. You can also use the **Learning Brain** with a pre-trained model. -The **Heuristic** Brain allows you to hand-code the Agent's logic by extending +The **Heuristic** Brain allows you to hand-code the Agent logic by extending the Decision class. Finally, the **Player** Brain lets you map keyboard commands to actions, which -can be useful when testing your agents and environment. If none of these types -of Brains do what you need, you can implement your own Brain. +can be useful when testing your agents and environment. You can also implement your own type of Brain. -In this tutorial, you will use a **Learning Brain** for training. +In this tutorial, you will use the **Learning Brain** for training. #### Vector Observation Space diff --git a/docs/Glossary.md b/docs/Glossary.md index d22e3ec61b..35afa1f915 100644 --- a/docs/Glossary.md +++ b/docs/Glossary.md @@ -7,7 +7,7 @@ * **Agent** - Unity Component which produces observations and takes actions in the environment. Agents actions are determined by decisions produced by a linked Brain. -* **Brain** - Unity Component which makes decisions for the agents linked to it. +* **Brain** - Unity Asset which makes decisions for the agents linked to it. * **Decision** - The specification produced by a Brain for an action to be carried out given an observation. * **Editor** - The Unity Editor, which may include any pane (e.g. Hierarchy, diff --git a/docs/Training-Imitation-Learning.md b/docs/Training-Imitation-Learning.md index 7789398b77..2d02a4d703 100644 --- a/docs/Training-Imitation-Learning.md +++ b/docs/Training-Imitation-Learning.md @@ -4,7 +4,7 @@ It is often more intuitive to simply demonstrate the behavior we want an agent to perform, rather than attempting to have it learn via trial-and-error methods. Consider our [running example](ML-Agents-Overview.md#running-example-training-npc-behaviors) -of training a medic NPC : instead of indirectly training a medic with the help +of training a medic NPC. Instead of indirectly training a medic with the help of a reward function, we can give the medic real world examples of observations from the game and actions from a game controller to guide the medic's behavior. Imitation Learning uses pairs of observations and actions from diff --git a/docs/images/3dball_learning_brain.png b/docs/images/3dball_learning_brain.png new file mode 100644 index 0000000000..1f4a4440ed Binary files /dev/null and b/docs/images/3dball_learning_brain.png differ diff --git a/docs/images/platform_prefab.png b/docs/images/platform_prefab.png new file mode 100644 index 0000000000..9eed9e3c1d Binary files /dev/null and b/docs/images/platform_prefab.png differ