Remove graph scope

[vincentpierre]

Feature : Remove Graph Scopes

Abstract

As part of our work to make inference within the Unity Engine easier to use, we propose to generate one graph per brain when training with multi-brain rather than a single graph with multiple graphs scopes.

Background

A key project we want to integrate in v0.6 is around inference within Unity. As such, there are several pieces that are required in order to make inference in Unity more intuitive. The first step is to ensure the graphs we generate through our training process are always formatted the same way. This is currently not the case when a model is trained through multi-brain since the graph generated contains multiple inference sub-graphs with different graph scopes.

Goals

• Make each model a single inference graph (only one action node per .bytes file)
• Generate only one bytes file per brain when training with multi-brain
• No disruption of the current workflow

Non-Goals

• Create new trainers / policies

Plan

The graph generated with multi-brain goes under : models/<run-id>/<env-name>_<run-id>.bytes with brain-name for graph scope
We propose to genetate multiple graphs without graph scopes under models/<run-id>/<brain-name>.bytes.
The policy will be responsible for serializing, saving and loading the models and will be called from the trainer_controller through trainer base class method calls.

Acceptance criteria

Single model per brain when training with multi-brain
Design aligns with long term plan

Milestones

• Implement a POC for saving sub-graphs into different
• Review functionality
• Review Design (@awjuliani)
• Fix the pytest
• Write documentation PR here
• Remove Graph Scope property from Internal Brain PR here

[awjuliani]

Changes look good. Just remove dead code and comments.

[mmattar]

@vincentpierre - I thought we were aiming to remove TF dependency from the Policy class?