Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
0055e1a
minor update to add 3D ball
shihzy Dec 13, 2018
ee9318c
included survey link
shihzy Dec 13, 2018
a2312d0
included clarification on updating prefab
shihzy Dec 13, 2018
244ed59
uploading platform prefab screenshot
shihzy Dec 13, 2018
a0f7b28
included screenshot of the platform prefab
shihzy Dec 13, 2018
a01fbe9
replacing the old screenshot
shihzy Dec 13, 2018
e8fec6f
updated to clarify the learningbrain step
shihzy Dec 13, 2018
863a3a4
upload screenshot for learning brain
shihzy Dec 13, 2018
e00be6b
added screenshot for learning brain in prefab
shihzy Dec 13, 2018
43ccbcc
replace 3dball learning brain screenshot
shihzy Dec 13, 2018
7e12d29
replace screenshot 2nd attempt
shihzy Dec 13, 2018
cce6061
Delete 3dball_learning_brain.png
shihzy Dec 13, 2018
e5642b7
trying again for screenshot
shihzy Dec 13, 2018
89df439
resized
shihzy Dec 13, 2018
6999242
replace again
shihzy Dec 13, 2018
bf4ef27
replace
shihzy Dec 13, 2018
46a2cd8
trying to replace again
shihzy Dec 13, 2018
ee61988
Delete 3dball_learning_brain.png
shihzy Dec 13, 2018
21bd015
Add files via upload
shihzy Dec 13, 2018
760f1bf
cleanup
shihzy Dec 13, 2018
bd174d9
more cleanup
shihzy Dec 13, 2018
88e37d2
more cleanup
shihzy Dec 13, 2018
ca924c4
clarified instructions for dragging TF model
shihzy Dec 13, 2018
9aaf59b
cleanup
shihzy Dec 13, 2018
63d349f
cleanup
shihzy Dec 13, 2018
4f1d4e4
changes to the adding brain steps
shihzy Dec 14, 2018
323c104
clean up
shihzy Dec 14, 2018
ded2d3e
Added background link to prefabs
shihzy Dec 14, 2018
6c02eaf
cleanup
shihzy Dec 14, 2018
c1b7c18
clean up
shihzy Dec 14, 2018
2ebe150
cleanup
shihzy Dec 14, 2018
67853e2
clean up
shihzy Dec 14, 2018
486b0fb
cleanup
shihzy Dec 14, 2018
6ddd0b1
clean up
shihzy Dec 14, 2018
43fad41
remove erroneous #
shihzy Dec 14, 2018
53073ca
typo
shihzy Dec 14, 2018
347a6ac
cleaned up the language in the academy
shihzy Dec 14, 2018
af3317b
typo
shihzy Dec 14, 2018
a89d4ac
Update to brain section
shihzy Dec 14, 2018
e874e81
clean up
shihzy Dec 14, 2018
f09a236
clean up
shihzy Dec 14, 2018
cc1ecae
clean up
shihzy Dec 14, 2018
c5c3144
clean up
shihzy Dec 14, 2018
5a4c1e5
clean up
shihzy Dec 14, 2018
58664ac
typo
shihzy Dec 14, 2018
85e4676
update definition of brain
shihzy Dec 14, 2018
f8dc198
updates from comments in PR
shihzy Dec 14, 2018
35ba6d6
minor typo
shihzy Dec 14, 2018
b106e22
minor tweak
shihzy Dec 14, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ If you run into any problems using the ML-Agents toolkit,
[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
make sure to include as much detail as possible.

Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to [let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).

For any other questions or feedback, connect directly with the ML-Agents
team at [email protected].

Expand Down
2 changes: 2 additions & 0 deletions docs/Background-Unity.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ with the ML-Agents toolkit:
* [Physics](https://docs.unity3d.com/Manual/PhysicsSection.html)
* [Ordering of event functions](https://docs.unity3d.com/Manual/ExecutionOrder.html)
(e.g. FixedUpdate, Update)
* [Prefabs](https://docs.unity3d.com/Manual/Prefabs.html)

71 changes: 38 additions & 33 deletions docs/Basic-Guide.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Basic Guide

This guide will show you how to use a pre-trained model in an example Unity
environment, and show you how to train the model yourself.
environment (3D Ball) and show you how to train the model yourself.

If you are not familiar with the [Unity Engine](https://unity3d.com/unity), we
highly recommend the [Roll-a-ball
Expand Down Expand Up @@ -42,33 +42,42 @@ TensorFlow files in the Project window under **Assets** > **ML-Agents** >
to the `Scripting Define Symbols` for each type of device you want to use
(**`PC, Mac and Linux Standalone`**, **`iOS`** or **`Android`**).

![Project Settings](images/project-settings.png)
![Project Settings](images/project-settings.png)

**Note**: If you don't see anything under **Assets**, drag the
`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window.
**Note**: If you don't see anything under **Assets**, drag the
`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window.

![Imported TensorFlowsharp](images/imported-tensorflowsharp.png)
![Imported TensorFlowsharp](images/imported-tensorflowsharp.png)

## Running a Pre-trained Model
We've included pre-trained models for the 3D Ball example.

1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder
1. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Scenes` folder
and open the `3DBall` scene file.
2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder
and select the `Game/Platform` prefab.
3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of
the `Ball 3D Agent`.
4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`.
__Note__ : You can modify multiple game objects in a scene by selecting them all at
2. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder.
Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window.

**Note**: The platforms in the `3DBall` scene were created using the `Platform` prefab. Instead of updating all 12 platforms individually, you can update the `Platform` prefab instead.

![Platform Prefab](images/platform_prefab.png)

3. In the **Project** window, drag the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.

![3dball learning brain](images/3dball_learning_brain.png)

4. You should notice that each `Platform` under each `Game` in the **Hierarchy** windows now contains **3DBallLearning** as `Brain`. __Note__ : You can modify multiple game objects in a scene by selecting them all at
once using the search bar in the Scene Hierarchy.
5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels`
5. In the **Project** window, click on the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains`. You should see the properties in the **Inspector** window.
6. In the **Project** window, open the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder.
6. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **3DBallLearning** Brain.
7. Click the **Play** button and you will see the platforms balance the balls
7. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **3DBallLearning** Brain in the **Inspector** window. __Note__ : All of the brains should now have `3DBallLearning` as the TensorFlow model in the `Model` property
8. Click the **Play** button and you will see the platforms balance the balls
using the pretrained model.

![Running a pretrained model](images/running-a-pretrained-model.gif)
![Running a pretrained model](images/running-a-pretrained-model.gif)

## Using the Basics Jupyter Notebook

Expand All @@ -84,30 +93,26 @@ More information and documentation is provided in the

## Training the Brain with Reinforcement Learning

### Adding a Brain to the training session
### Setting up the enviornment for training

To set up the environment for training, you will need to specify which agents are contributing
to the training and which Brain is being trained. You can only perform training with
a `Learning Brain`.

1. Assign the **3DBallLearning** Brain to the agents you would like to train.
__Note:__ You can assign the same Brain to multiple agents at once : To do so, you can
use the prefab system. When an agent is created from a prefab, modifying the prefab
will modify the agent as well. If the agent does not synchronize with the prefab, you
can hit the Revert button on top of the Inspector.
Alternatively, you can select multiple agents in the scene and modify their `Brain`
property all at once.
2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain
is in the Broadcast Hub. In order to train, you need to toggle the
`Control` checkbox.

__Note:__ Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
1. Each platform agent needs an assigned `Learning Brain`. In this example, each platform agent was created using a prefab. To update all of the brains in each platform agent at once, you only need to update the platform agent prefab. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder. Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window. In the **Project** window, drag the **3DBallLearning** Brain located in `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.

**Note**: The Unity prefab system will modify all instances of the agent properties in your scene. If the agent does not synchronize automatically with the prefab, you can hit the Revert button in the top of the **Inspector** window.

2. In the **Hierarchy** window, select `Ball3DAcademy`.
3. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Brains` folder and drag the **3DBallLearning** Brain to the `Brains` property under `Braodcast Hub` in the `Ball3DAcademy` object in the **Inspector** window. In order to train, make sure the `Control` checkbox is selected.

**Note:** Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
the agent) means that the Brain will be making decision for that agent. Whereas dragging
a Brain into the Broadcast Hub means that the Brain will be exposed to the Python process.
The `Control` checkbox means that in addition to being exposed to Python, the Brain will
be controlled by the Python process (required for training).

![Set Brain to External](images/mlagents-SetBrainToTrain.png)
![Set Brain to External](images/mlagents-SetBrainToTrain.png)

### Training the environment

Expand All @@ -134,7 +139,7 @@ be controlled by the Python process (required for training).
Editor"_ is displayed on the screen, you can press the :arrow_forward: button
in Unity to start training in the Editor.

**Note**: Alternatively, you can use an executable rather than the Editor to
**Note**: Alternatively, you can use an executable rather than the Editor to
perform training. Please refer to [this
page](Learning-Environment-Executable.md) for instructions on how to build and
use an executable.
Expand Down
2 changes: 1 addition & 1 deletion docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ There may be a number of possible causes:
_Solution_: Look into the [log
files](https://docs.unity3d.com/Manual/LogFiles.html) generated by the Unity
Environment to figure what error happened.
# _Cause_: You have assigned HTTP_PROXY and HTTPS_PROXY values in your
* _Cause_: You have assigned HTTP_PROXY and HTTPS_PROXY values in your
environment variables. _Solution_: Remove these values and try again.

## Communication port {} still in use
Expand Down
42 changes: 19 additions & 23 deletions docs/Getting-Started-with-Balance-Ball.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Let's get started!
In order to install and set up the ML-Agents toolkit, the Python dependencies
and Unity, see the [installation instructions](Installation.md).

## Understanding a Unity Environment (3D Balance Ball)
## Understanding the Unity Environment (3D Balance Ball)

An agent is an autonomous actor that observes and interacts with an
_environment_. In the context of Unity, an environment is a scene containing an
Expand All @@ -53,26 +53,26 @@ to speed up training since all twelve agents contribute to training in parallel.
The Academy object for the scene is placed on the Ball3DAcademy GameObject. When
you look at an Academy component in the inspector, you can see several
properties that control how the environment works.
The **Broadcast Hub** keeps track of which Brains will send data during training,
If a Brain is added to the hub, his data will be sent to the external training
The **Broadcast Hub** keeps track of which Brains will send data during training.
If a Brain is added to the hub, the data from this Brain will be sent to the external training
process. If the `Control` checkbox is checked, the training process will be able to
control the agents linked to the Brain to train them.
The **Training** and **Inference Configuration** properties
control and train the agents linked to the Brain.
The **Training Configuration** and **Inference Configuration** properties
set the graphics and timescale properties for the Unity application.
The Academy uses the **Training Configuration** during training and the
**Inference Configuration** when not training. (*Inference* means that the
Agent is using a trained model or heuristics or direct control — in other
words, whenever **not** training.)
Typically, you set low graphics quality and a high time scale for the **Training
configuration** and a high graphics quality and the timescale to `1.0` for the
Typically, you would set a low graphics quality and timescale to greater `1.0` for the **Training
Configuration** and a high graphics quality and timescale to `1.0` for the
**Inference Configuration** .

**Note:** if you want to observe the environment during training, you can adjust
the **Training Configuration** settings to use a larger window and a timescale
closer to 1:1. Be sure to set these parameters back when training in earnest;
otherwise, training can take a very long time.

Another aspect of an environment to look at is the Academy implementation. Since
Another aspect of an environment is the Academy implementation. Since
the base Academy class is abstract, you must always define a subclass. There are
three functions you can implement, though they are all optional:

Expand All @@ -88,28 +88,24 @@ environment around the Agents.

### Brain

Brains are assets that exist in your project folder. The Ball3DAgents are connected
to a Brain, for example : the **3DBallLearning**.
A Brain doesn't store any information about an Agent, it just
As of v0.6, a Brain is a Unity asset and exists within the `UnitySDK` folder. These brains (ex. **3DBallLearning.asset**) are loaded into each Agent object (ex. **Ball3DAgents**). A Brain doesn't store any information about an Agent, it just
routes the Agent's collected observations to the decision making process and
returns the chosen action to the Agent. Thus, all Agents can share the same
Brain, but act independently. The Brain settings tell you quite a bit about how
returns the chosen action to the Agent. All Agents can share the same
Brain, but would act independently. The Brain settings tell you quite a bit about how
an Agent works.

You can create Brain objects by selecting `Assets ->
Create -> ML-Agents -> Brain`. There are 3 kinds of Brains :
The **Learning Brain** is a Brain that uses a Neural Network to take decisions.
When the Brain is checked as `Control` in the Academy **Broadcast Hub**, the
external process will be taking decisions for the agents
and generate a neural network when the training is over. You can also use the
You can create new Brain assets by selecting `Assets ->
Create -> ML-Agents -> Brain`. There are 3 types of Brains.
The **Learning Brain** is a Brain that uses a trained neural network to make decisions.
When the `Control` box is checked in the Brains property under the **Broadcast Hub** in the Academy, the external process that is training the neural network will take over decision making for the agents
and ultimately generate a trained neural network. You can also use the
**Learning Brain** with a pre-trained model.
The **Heuristic** Brain allows you to hand-code the Agent's logic by extending
The **Heuristic** Brain allows you to hand-code the Agent logic by extending
the Decision class.
Finally, the **Player** Brain lets you map keyboard commands to actions, which
can be useful when testing your agents and environment. If none of these types
of Brains do what you need, you can implement your own Brain.
can be useful when testing your agents and environment. You can also implement your own type of Brain.

In this tutorial, you will use a **Learning Brain** for training.
In this tutorial, you will use the **Learning Brain** for training.

#### Vector Observation Space

Expand Down
2 changes: 1 addition & 1 deletion docs/Glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* **Agent** - Unity Component which produces observations and takes actions in
the environment. Agents actions are determined by decisions produced by a
linked Brain.
* **Brain** - Unity Component which makes decisions for the agents linked to it.
* **Brain** - Unity Asset which makes decisions for the agents linked to it.
* **Decision** - The specification produced by a Brain for an action to be
carried out given an observation.
* **Editor** - The Unity Editor, which may include any pane (e.g. Hierarchy,
Expand Down
2 changes: 1 addition & 1 deletion docs/Training-Imitation-Learning.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ It is often more intuitive to simply demonstrate the behavior we want an agent
to perform, rather than attempting to have it learn via trial-and-error methods.
Consider our
[running example](ML-Agents-Overview.md#running-example-training-npc-behaviors)
of training a medic NPC : instead of indirectly training a medic with the help
of training a medic NPC. Instead of indirectly training a medic with the help
of a reward function, we can give the medic real world examples of observations
from the game and actions from a game controller to guide the medic's behavior.
Imitation Learning uses pairs of observations and actions from
Expand Down
Binary file added docs/images/3dball_learning_brain.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/platform_prefab.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.