Skip to content
Merged
10 changes: 5 additions & 5 deletions docs/Background-TensorFlow.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ performing computations using data flow graphs, the underlying representation of
deep learning models. It facilitates training and inference on CPUs and GPUs in
a desktop, server, or mobile device. Within the ML-Agents toolkit, when you
train the behavior of an agent, the output is a TensorFlow model (.bytes) file
that you can then embed within an Internal Brain. Unless you implement a new
that you can then embed within a Learning Brain. Unless you implement a new
algorithm, the use of TensorFlow is mostly abstracted away and behind the
scenes.

Expand All @@ -39,15 +39,15 @@ recommend this
## TensorFlowSharp

One of the drawbacks of TensorFlow is that it does not provide a native C# API.
This means that the Internal Brain is not natively supported since Unity scripts
are written in C#. Consequently, to enable the Internal Brain, we leverage a
This means that the Learning Brain is not natively supported since Unity scripts
are written in C#. Consequently, to enable the Learning Brain, we leverage a
third-party library
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) which
provides .NET bindings to TensorFlow. Thus, when a Unity environment that
contains an Internal Brain is built, inference is performed via TensorFlowSharp.
contains a Learning Brain is built, inference is performed via TensorFlowSharp.
We provide an additional in-depth overview of how to leverage
[TensorFlowSharp within Unity](Using-TensorFlow-Sharp-in-Unity.md)
which will become more
relevant once you install and start training behaviors within the ML-Agents
toolkit. Given the reliance on TensorFlowSharp, the Internal Brain is currently
toolkit. Given the reliance on TensorFlowSharp, the Learning Brain is currently
marked as experimental.
46 changes: 24 additions & 22 deletions docs/Basic-Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,17 +45,20 @@ TensorFlow files in the Project window under **Assets** > **ML-Agents** >

## Running a Pre-trained Model

1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall` folder
1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder
and open the `3DBall` scene file.
2. In the **Hierarchy** window, select the **Ball3DBrain** child under the
**Ball3DAcademy** GameObject to view its properties in the Inspector window.
3. On the **Ball3DBrain** object's **Brain** component, change the **Brain
Type** to **Internal**.
4. In the **Project** window, locate the
`Assets/ML-Agents/Examples/3DBall/TFModels` folder.
5. Drag the `3DBall` model file from the `TFModels` folder to the **Graph
Model** field of the **Ball3DBrain** object's **Brain** component.
6. Click the **Play** button and you will see the platforms balance the balls
2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder
and select the `Game/Platform` prefab.
3. In the `Ball 3D Agent` Component: Drag the **Ball3DBrain** located into
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of
the `Ball 3D Agent`.
4. Make sure that all of the Agents in the Scene now have **Ball3DBrain** as `Brain`.
__Note__ : You can modify multiple game objects in a scene by selecting them all at once using the search bar in the Scene Hierarchy.
5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder.
6. Drag the `3DBall` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **Ball3DBrain**.
7. Click the **Play** button and you will see the platforms balance the balls
using the pretrained model.

![Running a pretrained model](images/running-a-pretrained-model.gif)
Expand All @@ -74,18 +77,18 @@ More information and documentation is provided in the

## Training the Brain with Reinforcement Learning

### Setting the Brain to External
### Adding a Brain to the training session

Since we are going to build this environment to conduct training, we need to set
the Brain used by the Agents to **External**. This allows the Agents to
communicate with the external training process when making their decisions.
Since we are going to build this environment to conduct training, we need to add
the Brain to the training session. This allows the Agents linked to that Brain
to communicate with the external training process when making their decisions.

1. In the **Scene** window, click the triangle icon next to the Ball3DAcademy
object.
2. Select its child object **Ball3DBrain**.
3. In the Inspector window, set **Brain Type** to **External**.
1. Assign the **Ball3DBrain** to the agents you would like to train.
__Note:__ You can only perform training with an `Learning Brain`.
2. Select the **Ball3DAcademy** GameObject and add the **Ball3DBrain**
to the Broadcast Hub and toggle the `Control` checkbox.

![Set Brain to External](images/mlagents-SetExternalBrain.png)
![Set Brain to External](images/mlagents-SetBrainToTrain.png)

### Training the environment

Expand Down Expand Up @@ -216,15 +219,14 @@ You can press Ctrl+C to stop the training, and your trained model will be at
`models/<run-identifier>/editor_<academy_name>_<run-identifier>.bytes` where
`<academy_name>` is the name of the Academy GameObject in the current scene.
This file corresponds to your model's latest checkpoint. You can now embed this
trained model into your Internal Brain by following the steps below, which is
trained model into your Learning Brain by following the steps below, which is
similar to the steps described
[above](#play-an-example-environment-using-pretrained-model).

1. Move your model file into
`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **Ball3DBrain** object from the Scene hierarchy.
4. Change the **Type of Brain** to **Internal**.
3. Select the **Ball3DBrain** Learning Brain from the Scene hierarchy.
5. Drag the `<env_name>_<run-identifier>.bytes` file from the Project window of
the Editor to the **Graph Model** placeholder in the **Ball3DBrain**
inspector window.
Expand Down
6 changes: 3 additions & 3 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ ENABLE_TENSORFLOW flag for your scripting define symbols, you will see the
following error message:

```console
You need to install and enable the TensorFlowSharp plugin in order to use the Internal Brain.
You need to install and enable the TensorFlowSharp plugin in order to use the Learning Brain.
```

This error message occurs because the TensorFlowSharp plugin won't be usage
Expand Down Expand Up @@ -69,8 +69,8 @@ On Windows, you can find
If you are able to launch the environment from `UnityEnvironment` but then
receive a timeout error, there may be a number of possible causes.

* _Cause_: There may be no Brains in your environment which are set to
`External`. In this case, the environment will not attempt to communicate
* _Cause_: There may be no Brains the `Broadcast Hub` of the Academy.
In this case, the environment will not attempt to communicate
with python. _Solution_: Set the Brains(s) you wish to externally control
through the Python API to `External` from the Unity Editor, and rebuild the
environment.
Expand Down
56 changes: 31 additions & 25 deletions docs/Getting-Started-with-Balance-Ball.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,17 @@ to speed up training since all twelve agents contribute to training in parallel.

The Academy object for the scene is placed on the Ball3DAcademy GameObject. When
you look at an Academy component in the inspector, you can see several
properties that control how the environment works. For example, the **Training**
and **Inference Configuration** properties set the graphics and timescale
properties for the Unity application. The Academy uses the **Training
Configuration** during training and the **Inference Configuration** when not
training. (*Inference* means that the Agent is using a trained model or
heuristics or direct control — in other words, whenever **not** training.)
properties that control how the environment works.
The **Broadcast Hub** keeps track of which brains will send data during training,
If a brain is added to the hub, his data will be sent to the external training
process. If the `Control` checkbox is checked, the training process will be able to
control the agents linked to the brain to train them.
The **Training** and **Inference Configuration** properties
set the graphics and timescale properties for the Unity application.
The Academy uses the **Training Configuration** during training and the
**Inference Configuration** when not training. (*Inference* means that the
Agent is using a trained model or heuristics or direct control — in other
words, whenever **not** training.)
Typically, you set low graphics quality and a high time scale for the **Training
configuration** and a high graphics quality and the timescale to `1.0` for the
**Inference Configuration** .
Expand All @@ -83,27 +88,28 @@ environment around the Agents.

### Brain

The Ball3DBrain GameObject in the scene, which contains a Brain component, is a
child of the Academy object. (All Brain objects in a scene must be children of
the Academy.) All the Agents in the 3D Balance Ball environment use the same
Brain instance. A Brain doesn't store any information about an Agent, it just
Brains are assets that exist in your project folder. The Ball3DAgents are connected
to a brain, for example : the **Ball3DBrain**.
A Brain doesn't store any information about an Agent, it just
routes the Agent's collected observations to the decision making process and
returns the chosen action to the Agent. Thus, all Agents can share the same
Brain, but act independently. The Brain settings tell you quite a bit about how
an Agent works.

The **Brain Type** determines how an Agent makes its decisions. The **External**
and **Internal** types work together — use **External** when training your
Agents; use **Internal** when using the trained model. The **Heuristic** Brain
allows you to hand-code the Agent's logic by extending the Decision class.
You can create brain objects by selecting `Assets ->
Create -> ML-Agents -> Brain`. There are 3 kinds of brains :
The **Learning Brain** is a brain that uses a Neural Network to take decisions.
When the Brain is checked as `Control` in the Academy **Broadcast Hub**, the
external process will be taking decisions for the agents
and generate a neural network when the training is over. You can also use the
**Learning Brain** with a pre-trained model.
The **Heuristic** Brain allows you to hand-code the Agent's logic by extending
the Decision class.
Finally, the **Player** Brain lets you map keyboard commands to actions, which
can be useful when testing your agents and environment. If none of these types
of Brains do what you need, you can implement your own CoreBrain to create your
own type.
of Brains do what you need, you can implement your own Brain.

In this tutorial, you will set the **Brain Type** to **External** for training;
when you embed the trained model in the Unity application, you will change the
**Brain Type** to **Internal**.
In this tutorial, you will use a **Learning Brain** for training.

#### Vector Observation Space

Expand Down Expand Up @@ -264,17 +270,17 @@ From TensorBoard, you will see the summary statistics:

Once the training process completes, and the training process saves the model
(denoted by the `Saved Model` message) you can add it to the Unity project and
use it with Agents having an **Internal** Brain type. **Note:** Do not just
close the Unity Window once the `Saved Model` message appears. Either wait for
the training process to close the window or press Ctrl+C at the command-line
prompt. If you simply close the window manually, the .bytes file containing the
trained model is not exported into the ml-agents folder.
use it with Agents having a **Learning Brain**.
__Note:__ Do not just close the Unity Window once the `Saved Model` message appears.
Either wait for the training process to close the window or press Ctrl+C at the
command-line prompt. If you simply close the window manually, the .bytes file
containing the trained model is not exported into the ml-agents folder.

### Setting up TensorFlowSharp Support

Because TensorFlowSharp support is still experimental, it is disabled by
default. In order to enable it, you must follow these steps. Please note that
the `Internal` Brain mode will only be available once completing these steps.
the `Learning` Brain mode will only be available once completing these steps.

To set up the TensorFlowSharp Support, follow [Setting up ML-Agents Toolkit
within Unity](Basic-Guide.md#setting-up-ml-agents-within-unity) section. of the
Expand Down
4 changes: 2 additions & 2 deletions docs/Glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,5 @@
logic should not be placed here.
* **External Coordinator** - ML-Agents class responsible for communication with
outside processes (in this case, the Python API).
* **Trainer** - Python class which is responsible for training a given External
Brain. Contains TensorFlow graph which makes decisions for External Brain.
* **Trainer** - Python class which is responsible for training a given
Brain. Contains TensorFlow graph which makes decisions for Learning Brain.
Loading