diff --git a/docs/_static/object_diagram.svg b/docs/_static/object_diagram.svg
new file mode 100644
index 0000000..ba2ae47
--- /dev/null
+++ b/docs/_static/object_diagram.svg
@@ -0,0 +1,4 @@
+
+
+
+
diff --git a/docs/_static/wrapper_lifecycle.svg b/docs/_static/wrapper_lifecycle.svg
new file mode 100644
index 0000000..e6294c9
--- /dev/null
+++ b/docs/_static/wrapper_lifecycle.svg
@@ -0,0 +1,4 @@
+
+
+
+
diff --git a/docs/api_reference.md b/docs/api_reference.md
index b091b11..60718a1 100644
--- a/docs/api_reference.md
+++ b/docs/api_reference.md
@@ -163,3 +163,24 @@ to instantiate such class by yourself**, and thus doing so is unsupported.
.. autofunction:: simphony_osp.tools.relationships_between
```
+
+## Development
+
+```{eval-rst}
+.. autoclass:: simphony_osp.development.Wrapper
+ :members: open, populate, commit, compute, close, load, save, delete, add, remove, triples
+ :special-members: __init__
+
+.. autoclass:: simphony_osp.development.BufferType
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
+.. autoclass:: simphony_osp.development.Operations
+ :members:
+ :undoc-members:
+ :show-inheritance:
+ :special-members: __init__
+
+.. autofunction:: simphony_osp.development.find_operations
+```
diff --git a/docs/conf.py b/docs/conf.py
index 352c672..009799c 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -68,7 +68,9 @@
def setup(app):
+ import simphony_osp.development
import simphony_osp.ontology
+ import simphony_osp.session
# Override names and modules of members of "simphony_osp.ontology" in the
# API reference.
@@ -83,3 +85,10 @@ def setup(app):
item = getattr(simphony_osp.session, obj)
setattr(item, "__name__", obj)
setattr(item, "__module__", simphony_osp.session.__name__)
+
+ # Override names and modules of members of "simphony_osp.development" in
+ # the API reference.
+ for obj in simphony_osp.development.__all__:
+ item = getattr(simphony_osp.development, obj)
+ setattr(item, "__name__", obj)
+ setattr(item, "__module__", simphony_osp.development.__name__)
diff --git a/docs/wrappers/wrapper_development_tutorial.ipynb b/docs/developers/wrapper_development_tutorial.ipynb
similarity index 100%
rename from docs/wrappers/wrapper_development_tutorial.ipynb
rename to docs/developers/wrapper_development_tutorial.ipynb
diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md
new file mode 100644
index 0000000..cd34a17
--- /dev/null
+++ b/docs/developers/wrappers.md
@@ -0,0 +1,283 @@
+# Wrapper development
+
+SimPhoNy Wrappers are software components that transform data from the
+assertional-knowledge form to the data structures of
+other software and back. Wrappers are abstracted to users as
+[sessions](../usage/sessions/introduction.ipynb), which may be viewed
+as "boxes" where ontology individuals can be stored.
+
+Sessions work in a way similar to databases. To start using them, one first
+has to “open” or “connect” to them. After that, changes can be made on the
+data they contain, but such changes are not permanent until a “commit” is
+performed. When one finishes working with them, the connection should be
+“closed”. Unconfirmed changes are lost when the connection is “closed”.
+
+Therefore, developing a wrapper involves crafting:
+
+- An abstraction of the concepts handled by the software as terminological
+ knowledge, that can then be used to represent the information as assertional
+ knowledge.
+- A database-like interface that is used by SimPhoNy to communicate with the
+ software.
+
+For the latter, SimPhoNy defines the _Wrapper API_, that must be implemented by
+the developer.
+
+## `Wrapper` abstract class
+
+The database-like interface used by SimPhoNy to communicate with the software,
+called _Wrapper API_, is defined by the `simphony_osp.development.Wrapper`
+abstract class. Objects belonging to the `Wrapper` class are indirectly
+controlled by the interactions between the user and session objects, as the
+diagram below shows.
+
+
+
+
+
+
+
+_[UML object diagram](https://www.uml-diagrams.org/class-diagrams-overview.html#object-diagram)
+showing the objects involved in the SimPhoNy wrapper mechanism that are
+relevant from a developer's perspective._
+
+
+
+
+
+SimPhoNy makes use of the [RDFLib](https://github.com/RDFLib/rdflib) library to
+handle [RDF](https://www.w3.org/TR/rdf-concepts/) data. Thus, the session is in
+fact an interface to an
+[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html).
+As the user interacts with the session, triples from the underlying graph are
+queried, added or removed.
+The library also provides a further abstraction, the
+[store](https://rdflib.readthedocs.io/en/stable/_modules/rdflib/store.html).
+Stores abstract [triplestores](https://en.wikipedia.org/wiki/Triplestore), a
+kind of database that can store collections of RDF graphs in the form of
+triples. SimPhoNy implements a special kind of store that is designed to
+communicate with SimPhoNy wrapper objects, the final pieces of the chain.
+
+The **wrapper object's interface is what the developer must implement** in
+order to make the software interoperable with SimPhoNy. As pointed out in the
+diagram, there are several RDF Graph objects, session objects and lists (of
+[ontology individual objects](../usage/assertional_knowledge.ipynb#Ontology-individual-objects))
+that are accessible from the wrapper object. Those offer several ways for the
+developer to retrieve the inputs from the user and translate them into inputs
+for the software, or conversely, reflect the outputs from the software into the
+user's session.
+
+Perhaps the most important of all is the **base graph**. The base graph is an
+[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html)
+that is accessible from the wrapper object, and must be kept in sync with the
+software's data structures at all times, as it constitutes their
+**RDF representation**. The goal of the SimPhoNy Wrapper API is to facilitate
+this task to the developer.
+
+```{note}
+If an
+[RDFLib store plug-in](https://rdflib.readthedocs.io/en/stable/plugin_stores.html)
+already exists for a specific software, then it can be trivially reused to
+implement a SimPhoNy wrapper. Just create a graph using the plug-in, and set it
+as the base graph for the SimPhoNy wrapper object.
+```
+
+## API Overview
+
+The [flowchart](https://en.wikipedia.org/wiki/Flowchart) below illustrates the
+**lifecycle** of a session connected to a wrapper object: from its creation to
+the moment it is closed.
+
+A sequence of method calls is executed as a
+consequence of each possible action the user can take. Each sequence is
+represented using a different color, and the action that triggers it is written
+next to its accompanying arrow, that points to the first method call in the
+sequence.
+
+
+
+
+
+
+
+_Flowchart showing the catalogue of possible user actions and how they
+translate to calls to the methods of the wrapper class, that the wrapper
+developer must implement._
+
+
+
+
+
+The `Wrapper` object is spawned when the user opens the session. The `open` and
+`populate` methods are then subsequently called in order to gain access to the
+resources needed by the software and pre-populate the session with ontology
+individuals if necessary. After that, the session is ready to be used. The user
+may then access or modify the assertional knowledge (triggering the optional,
+low-level RDF manipulation methods), access files associated to ontology
+individuals belonging to the _File_ class (triggering the `load` method),
+add or change files, commit the changes or request the software to compute new
+results. When the user is done, the session is closed.
+
+## API Specification
+
+This section describes the expected behavior of all methods from the Wrapper
+API.
+
+### Main methods
+
+#### `open`
+
+Secure access to the resources used by your software. For example:
+
+- spawn simulation engine objects
+- open a connection to a database
+- open files that need to be accessed
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.open
+```
+
+#### `populate`
+
+Populate the base session so that it represents the initial state of the
+software. For example:
+
+- given a path to a simulation settings file,
+ populate the session with entities
+ descibing the contents of the file
+- populate the session with dataset
+ individuals after connecting to a data
+ repository
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.populate
+```
+
+#### `commit`
+
+Reflect user's changes on the session in the software's data structure. For
+example:
+
+- configure a simulation's settings
+- update an SQL table
+- modify a file
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.commit
+```
+
+The difference with respect to the `compute` method is that this method should
+not update the content's of the session itself.
+
+#### `close`
+
+Release the resources used by your software. For example:
+
+- terminate the running process
+- close the connection to a database
+- close files that are not needed
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.close
+```
+
+#### `compute` _(optional)_
+
+Perform a computation using the data currently present on the software's data
+structures and update the base session to reflect the changes afterwards.
+For example:
+
+- run a simulation
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.compute
+```
+
+#### `__init__` _(optional)_
+
+The `__init__` method allows the user to provide extra
+parameters in the form of JSON-serializable keyword arguments.
+
+```{note}
+This method does not appear on the flowchart for simplicity, but is executed
+before the `open` method.
+```
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.__init__
+```
+
+### File manipulation methods
+
+#### `load` _(optional)_
+
+Receive an identifier of a file individual and retrieve the corresponding file.
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.load
+```
+
+#### `save` _(optional)_
+
+Receive an identifier of a file individual and a file handle and save the
+contents somewhere.
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.save
+```
+
+#### `delete` _(optional)_
+
+Receive the identifier of a file individual and delete the stored file.
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.delete
+```
+
+### RDF manipulation methods
+
+These methods operate at the RDF triple level. When a triple (or pattern) is
+added or removed, they can intercept the operation and decide whether the
+triple should go to the base graph or not when a commit operation is executed.
+The intercepted triples get stored in a buffer that is accessible during the
+commit operation.
+
+They are useful when one wants to prevent storing the same information twice
+(in the base graph and in the software's data structure).
+
+#### `add` _(optional)_
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.add
+```
+
+#### `remove` _(optional)_
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.remove
+```
+
+#### `triples` _(optional)_
+
+```{eval-rst}
+.. autofunction:: simphony_osp.development.Wrapper.triples
+```
+
+## Packaging template
+
+This page is meant to offer a mid-level view on what SimPhoNy Wrappers are
+and how do they work. If you are interested in developing one, you may find a
+template for building and packaging a wrapper in the
+[wrapper development repository](https://github.com/simphony/wrapper-development).
+
+```{warning}
+You are reading the documentation of a release candidate version of
+SimPhoNy.
+
+Some details have not yet been fully documented. In particular, the
+contents of the
+[wrapper development repository](https://github.com/simphony/wrapper-development)
+are still outdated. Consider using the
+[SimLAMMPS wrapper code](https://github.com/simphony/simlammps)
+as a template instead until the repository is updated.
+```
diff --git a/docs/index.md b/docs/index.md
index ca6f244..78c5d94 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -2,8 +2,7 @@
```{note}
⚠️ You are reading the documentation of a release candidate version of
-SimPhoNy. This version has not yet been thoroughly tested, and its
-functionality is not yet fully documented. The documentation for the latest
+SimPhoNy. The documentation for the latest
stable version of SimPhoNy can be found
[here](https://simphony.readthedocs.io/en/latest/).
```
@@ -49,6 +48,15 @@ SimPhoNy enables:
:text: To the API reference
:classes: btn-outline-primary stretched-link
+ ---
+ **Developer's documentation**
+
+ Wrapper development, SimPhoNy operations
+
+ ```{link-button} developers/wrappers.html
+ :text: To the developer's documentation
+ :classes: btn-outline-primary stretched-link
+
---
**Additional Information**
@@ -86,6 +94,14 @@ usage/wrappers/index
usage/visualization
```
+```{toctree}
+:hidden: true
+:caption: Developer's documentation
+:maxdepth: 2
+
+developers/wrappers
+```
+
```{toctree}
:hidden: true
:caption: Additional Info
diff --git a/docs/wrappers/wrapper_development.md b/docs/wrappers/wrapper_development.md
deleted file mode 100644
index 8f1d1d4..0000000
--- a/docs/wrappers/wrapper_development.md
+++ /dev/null
@@ -1,340 +0,0 @@
-# Wrapper development
-
-For an skeleton structure of a wrapper, you can visit the [wrapper development repo](https://github.com/simphony/wrapper-development).
-For a tutorial on creating a simple wrapper, there is a [jupyter notebook](wrapper_development_tutorial.ipynb) available.
-
-## Ontology
-
-The end goal is to build one, unique and standard ontology with all the relevant entities and relationships.
-This ontology could use modules where the entities regarding a certain domain are present.
-
-However, for the development of a wrapper, it is usually more practical to create a minimal temporary ontology
-with the entities required by a wrapper. Once the development is in a more stable stage, the development and merge
-of a correct ontology can be done, and should not require major changes in the code.
-
-These are the requirements for a minimal wrapper ontology:
-
-- Should contain an entity representing the wrapper.
- Said entity should inherit from (subclass, is_a) `cuba.Wrapper`.
-- All attributes should subclass `cuba.attribute`.
-- Top level entities should subclass `cuba.Entity`
-- Active relationships should subclass `cuba.ActiveRelationship`
-- Passive relationships should subclass `cuba.PassiveRelationship`
-
-
- Dummy ontology sample
-
-```yaml
----
-version: "M.m"
-
-author: Parmenides
-
-namespace: some_new_wrapper_ontology
-
-ontology:
- aRelationship:
- description: "default relationship"
- subclass_of:
- - cuba.activeRelationship
- inverse: some_new_wrapper_ontology.pihsnoitalerA
- default_rel: true
-
- pihsnoitalerA:
- description: "inverse of the default relationship"
- subclass_of:
- - cuba.passiveRelationship
- inverse: some_new_wrapper_ontology.aRelationship
-
- ################
-
- SomeNewWrapper:
- subclass_of:
- - cuba.Wrapper
-
- value:
- subclass_of:
- - cuba.attribute
-
- SomeEntity:
- subclass_of:
- - cuba.Entity
-```
-
-
-
-## Coding
-
-An advantage of the
-[3-layered-design](../introduction/general_architecture.md) that we follow is the modularity and conceptual separation.
-The closer to the user, the higher the abstraction.
-
-This allows us to group and clearly define which components should and which ones should not be modified when creating a new wrapper.
-
-- [Semantic layer](../detailed_design.md#semantic-layer):
- Requires no work.
- As presented in the previous section, only an entity representing the wrapper has to be present in the ontology.
-
-- [Interoperability layer](../detailed_design.md#interoperability-layer):
-
- - [Session class](../detailed_design.md#session):
- Represents the bulk of the work that a wrapper developer needs to do.
- A new class inheriting from the appropriate Session Abstract Base Class must be coded.
- It should at least implement all the inherited abstract methods.
- - `__str__(self)`: String representation of the wrapper.
- - `_apply_added(self, root_obj, buffer)`: Add all the elements in `buffer` to the engine.
- - `_apply_updated(self, root_obj, buffer)`: Update all the elements in `buffer` in the engine.
- - `_apply_deleted(self, root_obj, buffer)`: Remove all the elements in `buffer` from the engine.
- - `_load_from_backend(self, uids, expired=None)`: Loads the given uids (and the dependant entities)
- with the latest information from the backend.
- Only loads the directly related information, not all the children recursively.
- This method _must_ be implemented for a wrapper to work.
- - Specific for a simulation:
- - `_run(self, root_cuds_object)`: Call the run method on the simulation.
- - Specific for a database:
- - `_initialize(self)`: Initialise the database.
- - `_load_first_level(self)`: Load the first level of children from the root from the database.
- - `_init_transaction(self)`: Start the transaction.
- - `_rollback_transaction(self)`: Rollback the transaction.
- - `close(self)`: Close the connection.
-
-- [Syntactic layer](../detailed_design.md#syntactic-layer):
- If none is available, one must be developed.
-
-To facilitate the creation of the session class on the interoperability layer,
-there are several session abstract base classes that you can make your session
-inherit from, which already include some additional generic functions common to a few
-typical applications: databases, triplestores and simulation engines.
-
-On the diagram below, you may observe a simplified session inheritance scheme
-for OSP-core. As a wrapper developer, you will most probably want to inherit
-from one of following abstract classes: `WrapperSession`, `DbWrapperSession`,
-`TripleStoreWrapperSession`, `SqlWrapperSession`, or `SimWrapperSession`.
-Your new wrapper session would be located of the OSP-core box,
-together among other wrapper sessions like the Simlammps, Sqlite or SqlAlchemy
-sessions.
-
-```{uml}
- :caption: Simplified session inheritance scheme
- :align: center
-
- skinparam {
- Shadowing false
- BackgroundColor transparent
- ClassBackgroundColor #E3E3E3
- ClassBorderColor black
- PackageBorderColor black
- PackageBackgroundColor #9FC6DE
- ArrowColor #179c7d
- NoteBackgroundColor transparent
- NoteBorderColor black
- }
-
- rectangle "OSP-core" as OSP {
- abstract class Session {
- }
-
- class CoreSession implements Session {
- }
-
- abstract class WrapperSession extends Session {
- }
-
- class TransportSession implements WrapperSession {
- }
-
- abstract class DbWrapperSession extends WrapperSession {
- commit()
- }
-
- abstract class TripleStoreWrapperSession extends DbWrapperSession {
- }
-
- abstract class SqlWrapperSession extends TripleStoreWrapperSession {
- }
-
- abstract class SimWrapperSession extends WrapperSession {
- run()
- }
- }
-
- rectangle "Sqlite wrapper" as sqlite {
- class SqliteWrapperSession implements SqlWrapperSession {
- }
- }
-
- rectangle "SqlAlchemy wrapper" as sqlalchemy {
- class SqlAlchemyWrapperSession implements SqlWrapperSession {
- }
- }
-
- rectangle "Your wrapper" as yourwrapper {
- class YourSession{
- }
- }
-
- rectangle "Simlammps wrapper" as simlammps {
- class SimlammpsSession implements SimWrapperSession {
- }
- }
-
- ' -----------------------
- ' -------- NOTES --------
- ' -----------------------
- note as OSP.note_core
- The CoreSession is the default
- shared session for all Python objects
- end note
- OSP.note_core .. CoreSession
-
- note as note_db
- The db changes are persisted via
- cuds_object.session.commit()
- end note
- note_db .. DbWrapperSession
-
- note as OSP.note_sim
- The simulation is run by calling
- cuds_object.session.run()
- end note
- OSP.note_sim .. SimWrapperSession
-```
-
-## Engine installation
-
-Most engines will require some sort of compilation or installation before being able to use them through Python.
-
-To facilitate the installation of the backend to the end users, a shell script with the necessary commands should be made available.
-It is also recommended to split the installation of the engine from the installation of the engine requirements.
-
-
- Sample install_engine_requirements.sh
-
-```shell
- #!/bin/bash
- #
- # Author: Ada Lovelace
- #
- # Description: This script install the requirements for some engine
- # Used as part of the installation for SomeWrapper.
- #
- # Run Information: This script is called by install_engine.sh
-
- echo "Installing necessary requirements for the engine"
- platform=$(python3 -mplatform)
-
- case $platform in
- *"Ubuntu"*)
- sudo apt-get update
- sudo apt-get install cmake
- ;;
- *"centos"*)
- sudo yum update
- sudo yum install make -y
- sudo yum install cmake -y
- ;;
- # Add other platforms here
- esac
-
-```
-
-
-
-
- Sample install_engine.sh
-
-```shell
- #!/bin/bash
- #
- # Author: Ada Lovelace
- #
- # Description: This script installs SomeEngine and its Python binding
- # Used as part of the installation for SomeWrapper.
- #
- # Run Information: This script is run manually.
-
- ###################################
- ### Install engine requirements ###
- ###################################
- ./install_engine_requirements.sh
-
- ################################
- ### Download necessary files ###
- ################################
- echo "Checking out a recent stable version"
- git clone some-repo.com/some-engine.git
- cd some-engine
-
- ############################
- ### Perform installation ###
- ############################
- cmake cmake
- make install
-
- #########################
- ### Test installation ###
- #########################
- {
- python3 -c 'from someEngine import engine; engine.test()'
- } || {
- echo "There was an error with the installation."
- echo "Please, try again or contact the developer."
- }
-
-```
-
-
-
-When the implementation of the wrapper is done, the user should be able to install all the necessary components via:
-
-```shell
-(env) user@computer:~/some_wrapper$ ./install_engine.sh
-(env) user@computer:~/some_wrapper$ python setup.py install
-```
-
-### Dockerfile with the engine
-
-Apart from a system installation, we highly recommend providing a `Dockerfile` with the engine
-and other minimal requirements, in case the system installation is not possible or desired.
-
-Similar to how OSP-core is the structure on top of which the wrappers are made,
-we designed a schema of Docker images where OSP-core is used as a base image.
-
-Thus, OSP-core has an image (currently using Ubuntu) that should be tagged `simphony/osp-core:`.
-The Dockerfile of a wrapper will have that image in the `FROM` statement at the top,
-and take care of installing the engine requirements (and the wrapper itself).
-
-To fix the tagging of the images and the versioning compatibility,
-the `Dockerfile` should be installed via the provided `docker_install.sh` script.
-It will tag the OSP-core image and call the Dockerfile in the root of the wrapper accordingly.
-
-In terms of implementation, a wrapper developer needs to take care of the `Dockerfile`,
-making sure to leave the first two lines as they are in the [wrapper development repo](https://github.com/simphony/wrapper-development/blob/master/Dockerfile).
-`docker_install.sh` will only have to be modified with the proper tag for the wrapper image.
-
-## Continuous Integration
-
-GitLab provides Continuous Integration via the `.gitlab-ci.yml` file.
-This should be used for checking both the style and the functionality of the code automatically after each commit.
-
-If the wrapper requires the installation of an engine, it would probably be best to install it in a Docker image
-and push the image to Gitlab Container Registry so that the CI jobs use that image as the base system in which to run.
-
-The `Dockerfile` for the Container Registry image will be very similar to the one used for installing the engine.
-However, here it might be useful to install other libraries like flake8 for style checks.
-
-## Utility functions for wrapper development
-
-We have developed some functions that will probably come in handy when developing a wrapper.
-You can find them in [osp.core.utils.wrapper_development](https://github.com/simphony/osp-core/blob/master/osp/core/utils/wrapper_development.py).
-
-## Wrapper Examples
-
-Some wrappers we are developing are:
-
-- [SQLAlchemy](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/sqlalchemy-wrapper)
-- [SQLite](https://github.com/simphony/osp-core/tree/master/osp/wrappers/sqlite)
-- [SimLammps](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simlammps)
-- [SimGromacs](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simgromacs)
-- [SimOpenFoam](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simopenfoam)
-- [Quantum Espresso](https://github.com/simphony/quantum-espresso-wrapper)