From 137e73d5cb6a295800d9c41bdcae5878762350bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Fri, 23 Sep 2022 17:29:02 +0200 Subject: [PATCH 1/8] Initial wrapper mechanism documentation for SimPhoNy v4 --- .../wrapper_development_tutorial.ipynb | 0 docs/developers/wrappers.md | 133 +++++++ docs/index.md | 17 + docs/wrappers/wrapper_development.md | 340 ------------------ 4 files changed, 150 insertions(+), 340 deletions(-) rename docs/{wrappers => developers}/wrapper_development_tutorial.ipynb (100%) create mode 100644 docs/developers/wrappers.md delete mode 100644 docs/wrappers/wrapper_development.md diff --git a/docs/wrappers/wrapper_development_tutorial.ipynb b/docs/developers/wrapper_development_tutorial.ipynb similarity index 100% rename from docs/wrappers/wrapper_development_tutorial.ipynb rename to docs/developers/wrapper_development_tutorial.ipynb diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md new file mode 100644 index 0000000..64fe826 --- /dev/null +++ b/docs/developers/wrappers.md @@ -0,0 +1,133 @@ +# Wrapper development + +SimPhoNy Wrappers are software components that transform data from the +assertional-knowledge form that they take in SimPhoNy's +[sessions](../usage/sessions/introduction.ipynb) to the data structures of +other software and back. Wrappers are abstracted to users as +[sessions](../usage/sessions/introduction.ipynb), which may be viewed +as "boxes" where ontology individuals can be placed + +Sessions work in a way similar to databases. To start using them, one first +has to “open” or “connect” to them. After that, changes can be performed on the +data they contain, but such changes are not made permanent until a “commit” is +performed. When one finishes working with them, the connection should be +“closed”. Unconfirmed changes are lost when the connection is “closed”. + +The development of a wrapper involves implementing: + +- An abstraction of the concepts handled by the software as terminological + knowledge, that can then be used to represent the information as assertional + knowledge. +- A database-like interface that is used by SimPhoNy to communicate with the + software. + +For the latter, SimPhoNy defines a _Wrapper API_ that can be implemented by the +developer. + +## Static perspective + +The static view on the SimPhoNy's wrapper API highlights the software +components that the user and wrapper developer interact with, and how they +are connected. + +As shown in the diagram below, the user sees a SimPhoNy session object, +which abstracts all the complexity, appearing just as a simple "box" +that can store ontology individuals and where relevant +operations and queries can be run. + +However, the session object is an interface to an +[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html). +As the user interacts with the session, triples from the underlying graph are +being queried, added or removed. + +
+ +![Static perspective](../_static/object_diagram.svg) + +
+ +_[UML object diagram](https://www.uml-diagrams.org/class-diagrams-overview.html#object-diagram) +showing the objects involved in the SimPhoNy wrapper mechanism that are +relevant from a developer's perspective._ + +
+ +
+ +The [RDFLib](https://github.com/RDFLib/rdflib) library provides a further +abstraction, the +[store](https://rdflib.readthedocs.io/en/stable/_modules/rdflib/store.html). +Stores abstract [triplestores](https://en.wikipedia.org/wiki/Triplestore), a +kind of database that can store collections of RDF graphs in the form of +triples. SimPhoNy implements a special kind of store that is designed to +communicate with SimPhoNy wrapper objects, the final pieces of the chain. + +The **wrapper object's interface is what the developer must implement** in +order to make the software interoperable with SimPhoNy. As pointed out in the +diagram, there are several RDF Graph objects, session objects and lists (of +[ontology individual objects](../usage/assertional_knowledge.ipynb#Ontology-individual-objects)) +that are accessible from the wrapper object. Those offer several ways for the +developer to retrieve the inputs from the user and translate them into inputs +for the software, or conversely, reflect the outputs from the software into the +user's session. + +Perhaps the most important of all is the **base graph**. The base graph is an +[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html) +that is accessible from the wrapper object, and must be kept in sync with the +software's data structures at all times, as it constitutes their +**RDF representation**. + +```{note} +If an +[RDFLib store plug-in](https://rdflib.readthedocs.io/en/stable/plugin_stores.html) +already exists for a specific software, then it can be trivially reused to +implement a SimPhoNy wrapper. Just create a graph using the plug-in, and set it +as the base graph for the SimPhoNy wrapper object. +``` + +The goal of the SimPhoNy wrapper API is to facilitate this task to +the developer. + +## Dynamic perspective + +The dynamic view on the SimPhoNy's wrapper API offers an insight on how the +interactions between the user and the session object translate into calls to +the methods of the wrapper object. The diagram below illustrates the +**lifecycle** of a session connected to a wrapper object: from its creation +to the moment it is closed. + +
+ +![Wrapper session lifecycle](../_static/wrapper_lifecycle.svg) + +
+ +_Flowchart showing the actions a user can perform and how they translate +to calls to the methods of the wrapper object, that the wrapper developer must +implement._ + +
+ +
+ +## API Specification + +TODO + +## Writing your own wrapper + +This page is meant to offer a mid-level view on what is a SimPhoNy wrapper +and how do they work. If you are interested in developing one, please check the +[wrapper development repository](https://github.com/simphony/wrapper-development), +where a template for building and packaging a wrapper can be found. + +```{warning} +You are reading the documentation of a release candidate version of +SimPhoNy. This version has not yet been fully documented. + +In particular, the contents of the +[wrapper development repository](https://github.com/simphony/wrapper-development) +are still outdated. Consider using the +[SimLAMMPS wrapper code](https://github.com/simphony/simlammps) as +a template instead until the repository is updated. +``` diff --git a/docs/index.md b/docs/index.md index ca6f244..660c6d2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -49,6 +49,15 @@ SimPhoNy enables: :text: To the API reference :classes: btn-outline-primary stretched-link + --- + **Developer's documentation** + + Wrapper development, SimPhoNy operations + + ```{link-button} developers/wrappers.html + :text: To the developer's documentation + :classes: btn-outline-primary stretched-link + --- **Additional Information** @@ -86,6 +95,14 @@ usage/wrappers/index usage/visualization ``` +```{toctree} +:hidden: true +:caption: Developer's documentation +:maxdepth: 2 + +developers/wrappers +``` + ```{toctree} :hidden: true :caption: Additional Info diff --git a/docs/wrappers/wrapper_development.md b/docs/wrappers/wrapper_development.md deleted file mode 100644 index 8f1d1d4..0000000 --- a/docs/wrappers/wrapper_development.md +++ /dev/null @@ -1,340 +0,0 @@ -# Wrapper development - -For an skeleton structure of a wrapper, you can visit the [wrapper development repo](https://github.com/simphony/wrapper-development). -For a tutorial on creating a simple wrapper, there is a [jupyter notebook](wrapper_development_tutorial.ipynb) available. - -## Ontology - -The end goal is to build one, unique and standard ontology with all the relevant entities and relationships. -This ontology could use modules where the entities regarding a certain domain are present. - -However, for the development of a wrapper, it is usually more practical to create a minimal temporary ontology -with the entities required by a wrapper. Once the development is in a more stable stage, the development and merge -of a correct ontology can be done, and should not require major changes in the code. - -These are the requirements for a minimal wrapper ontology: - -- Should contain an entity representing the wrapper. - Said entity should inherit from (subclass, is_a) `cuba.Wrapper`. -- All attributes should subclass `cuba.attribute`. -- Top level entities should subclass `cuba.Entity` -- Active relationships should subclass `cuba.ActiveRelationship` -- Passive relationships should subclass `cuba.PassiveRelationship` - -
- Dummy ontology sample - -```yaml ---- -version: "M.m" - -author: Parmenides - -namespace: some_new_wrapper_ontology - -ontology: - aRelationship: - description: "default relationship" - subclass_of: - - cuba.activeRelationship - inverse: some_new_wrapper_ontology.pihsnoitalerA - default_rel: true - - pihsnoitalerA: - description: "inverse of the default relationship" - subclass_of: - - cuba.passiveRelationship - inverse: some_new_wrapper_ontology.aRelationship - - ################ - - SomeNewWrapper: - subclass_of: - - cuba.Wrapper - - value: - subclass_of: - - cuba.attribute - - SomeEntity: - subclass_of: - - cuba.Entity -``` - -
- -## Coding - -An advantage of the -[3-layered-design](../introduction/general_architecture.md) that we follow is the modularity and conceptual separation. -The closer to the user, the higher the abstraction. - -This allows us to group and clearly define which components should and which ones should not be modified when creating a new wrapper. - -- [Semantic layer](../detailed_design.md#semantic-layer): - Requires no work. - As presented in the previous section, only an entity representing the wrapper has to be present in the ontology. - -- [Interoperability layer](../detailed_design.md#interoperability-layer): - - - [Session class](../detailed_design.md#session): - Represents the bulk of the work that a wrapper developer needs to do. - A new class inheriting from the appropriate Session Abstract Base Class must be coded. - It should at least implement all the inherited abstract methods. - - `__str__(self)`: String representation of the wrapper. - - `_apply_added(self, root_obj, buffer)`: Add all the elements in `buffer` to the engine. - - `_apply_updated(self, root_obj, buffer)`: Update all the elements in `buffer` in the engine. - - `_apply_deleted(self, root_obj, buffer)`: Remove all the elements in `buffer` from the engine. - - `_load_from_backend(self, uids, expired=None)`: Loads the given uids (and the dependant entities) - with the latest information from the backend. - Only loads the directly related information, not all the children recursively. - This method _must_ be implemented for a wrapper to work. - - Specific for a simulation: - - `_run(self, root_cuds_object)`: Call the run method on the simulation. - - Specific for a database: - - `_initialize(self)`: Initialise the database. - - `_load_first_level(self)`: Load the first level of children from the root from the database. - - `_init_transaction(self)`: Start the transaction. - - `_rollback_transaction(self)`: Rollback the transaction. - - `close(self)`: Close the connection. - -- [Syntactic layer](../detailed_design.md#syntactic-layer): - If none is available, one must be developed. - -To facilitate the creation of the session class on the interoperability layer, -there are several session abstract base classes that you can make your session -inherit from, which already include some additional generic functions common to a few -typical applications: databases, triplestores and simulation engines. - -On the diagram below, you may observe a simplified session inheritance scheme -for OSP-core. As a wrapper developer, you will most probably want to inherit -from one of following abstract classes: `WrapperSession`, `DbWrapperSession`, -`TripleStoreWrapperSession`, `SqlWrapperSession`, or `SimWrapperSession`. -Your new wrapper session would be located of the OSP-core box, -together among other wrapper sessions like the Simlammps, Sqlite or SqlAlchemy -sessions. - -```{uml} - :caption: Simplified session inheritance scheme - :align: center - - skinparam { - Shadowing false - BackgroundColor transparent - ClassBackgroundColor #E3E3E3 - ClassBorderColor black - PackageBorderColor black - PackageBackgroundColor #9FC6DE - ArrowColor #179c7d - NoteBackgroundColor transparent - NoteBorderColor black - } - - rectangle "OSP-core" as OSP { - abstract class Session { - } - - class CoreSession implements Session { - } - - abstract class WrapperSession extends Session { - } - - class TransportSession implements WrapperSession { - } - - abstract class DbWrapperSession extends WrapperSession { - commit() - } - - abstract class TripleStoreWrapperSession extends DbWrapperSession { - } - - abstract class SqlWrapperSession extends TripleStoreWrapperSession { - } - - abstract class SimWrapperSession extends WrapperSession { - run() - } - } - - rectangle "Sqlite wrapper" as sqlite { - class SqliteWrapperSession implements SqlWrapperSession { - } - } - - rectangle "SqlAlchemy wrapper" as sqlalchemy { - class SqlAlchemyWrapperSession implements SqlWrapperSession { - } - } - - rectangle "Your wrapper" as yourwrapper { - class YourSession{ - } - } - - rectangle "Simlammps wrapper" as simlammps { - class SimlammpsSession implements SimWrapperSession { - } - } - - ' ----------------------- - ' -------- NOTES -------- - ' ----------------------- - note as OSP.note_core - The CoreSession is the default - shared session for all Python objects - end note - OSP.note_core .. CoreSession - - note as note_db - The db changes are persisted via - cuds_object.session.commit() - end note - note_db .. DbWrapperSession - - note as OSP.note_sim - The simulation is run by calling - cuds_object.session.run() - end note - OSP.note_sim .. SimWrapperSession -``` - -## Engine installation - -Most engines will require some sort of compilation or installation before being able to use them through Python. - -To facilitate the installation of the backend to the end users, a shell script with the necessary commands should be made available. -It is also recommended to split the installation of the engine from the installation of the engine requirements. - -
- Sample install_engine_requirements.sh - -```shell - #!/bin/bash - # - # Author: Ada Lovelace - # - # Description: This script install the requirements for some engine - # Used as part of the installation for SomeWrapper. - # - # Run Information: This script is called by install_engine.sh - - echo "Installing necessary requirements for the engine" - platform=$(python3 -mplatform) - - case $platform in - *"Ubuntu"*) - sudo apt-get update - sudo apt-get install cmake - ;; - *"centos"*) - sudo yum update - sudo yum install make -y - sudo yum install cmake -y - ;; - # Add other platforms here - esac - -``` - -
- -
- Sample install_engine.sh - -```shell - #!/bin/bash - # - # Author: Ada Lovelace - # - # Description: This script installs SomeEngine and its Python binding - # Used as part of the installation for SomeWrapper. - # - # Run Information: This script is run manually. - - ################################### - ### Install engine requirements ### - ################################### - ./install_engine_requirements.sh - - ################################ - ### Download necessary files ### - ################################ - echo "Checking out a recent stable version" - git clone some-repo.com/some-engine.git - cd some-engine - - ############################ - ### Perform installation ### - ############################ - cmake cmake - make install - - ######################### - ### Test installation ### - ######################### - { - python3 -c 'from someEngine import engine; engine.test()' - } || { - echo "There was an error with the installation." - echo "Please, try again or contact the developer." - } - -``` - -
- -When the implementation of the wrapper is done, the user should be able to install all the necessary components via: - -```shell -(env) user@computer:~/some_wrapper$ ./install_engine.sh -(env) user@computer:~/some_wrapper$ python setup.py install -``` - -### Dockerfile with the engine - -Apart from a system installation, we highly recommend providing a `Dockerfile` with the engine -and other minimal requirements, in case the system installation is not possible or desired. - -Similar to how OSP-core is the structure on top of which the wrappers are made, -we designed a schema of Docker images where OSP-core is used as a base image. - -Thus, OSP-core has an image (currently using Ubuntu) that should be tagged `simphony/osp-core:`. -The Dockerfile of a wrapper will have that image in the `FROM` statement at the top, -and take care of installing the engine requirements (and the wrapper itself). - -To fix the tagging of the images and the versioning compatibility, -the `Dockerfile` should be installed via the provided `docker_install.sh` script. -It will tag the OSP-core image and call the Dockerfile in the root of the wrapper accordingly. - -In terms of implementation, a wrapper developer needs to take care of the `Dockerfile`, -making sure to leave the first two lines as they are in the [wrapper development repo](https://github.com/simphony/wrapper-development/blob/master/Dockerfile). -`docker_install.sh` will only have to be modified with the proper tag for the wrapper image. - -## Continuous Integration - -GitLab provides Continuous Integration via the `.gitlab-ci.yml` file. -This should be used for checking both the style and the functionality of the code automatically after each commit. - -If the wrapper requires the installation of an engine, it would probably be best to install it in a Docker image -and push the image to Gitlab Container Registry so that the CI jobs use that image as the base system in which to run. - -The `Dockerfile` for the Container Registry image will be very similar to the one used for installing the engine. -However, here it might be useful to install other libraries like flake8 for style checks. - -## Utility functions for wrapper development - -We have developed some functions that will probably come in handy when developing a wrapper. -You can find them in [osp.core.utils.wrapper_development](https://github.com/simphony/osp-core/blob/master/osp/core/utils/wrapper_development.py). - -## Wrapper Examples - -Some wrappers we are developing are: - -- [SQLAlchemy](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/sqlalchemy-wrapper) -- [SQLite](https://github.com/simphony/osp-core/tree/master/osp/wrappers/sqlite) -- [SimLammps](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simlammps) -- [SimGromacs](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simgromacs) -- [SimOpenFoam](https://gitlab.cc-asp.fraunhofer.de/simphony/wrappers/simopenfoam) -- [Quantum Espresso](https://github.com/simphony/quantum-espresso-wrapper) From a81f63e27dc5ab49f41888abb274478816c8e8af Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Fri, 23 Sep 2022 17:58:57 +0200 Subject: [PATCH 2/8] Add pictures --- docs/_static/object_diagram.svg | 4 ++++ docs/_static/wrapper_lifecycle.svg | 4 ++++ 2 files changed, 8 insertions(+) create mode 100644 docs/_static/object_diagram.svg create mode 100644 docs/_static/wrapper_lifecycle.svg diff --git a/docs/_static/object_diagram.svg b/docs/_static/object_diagram.svg new file mode 100644 index 0000000..c82030e --- /dev/null +++ b/docs/_static/object_diagram.svg @@ -0,0 +1,4 @@ + + + +
session:
simphony_osp
.session::Session
session:...
Atom
Atom
Velocity
(1, 3, 5)
Velocity...
Position
(0, 7, 2)
Position...
User side
User...
Developer side
Devel...
Internal objects
Internal objects
graph: rdflib::Graph
graph: rdflib::Graph
driver: 
simphony_osp
.interfaces
.interface.py
::InterfaceDriver
driver:...
-graph
-graph
-store
-store
-interface
-interface
«interface»
wrapper:
simphony_osp
.development::Wrapper
«interface»...
-driver
-driver
old_graph: rdflib::Graph
old_graph: rdflib::Gr...
new_graph: rdflib::Graph
new_graph: rdflib::Gr...
buffer: rdflib::Graph
buffer: rdflib::Graph
session_base:
simphony_osp.session
::Session
session_base:...
session_old:
simphony_osp.session
::Session
session_old:...
session_new:
simphony_osp.session
::Session
session_new:...
added: set
added: set
updated: set
updated: set
deleted: set
deleted: set
-new_graph
-new_graph
-buffer
-buffer
-session_base
-session_base
-session_old
-session_old
-session_new
-session_new
-added
-added
-updated
-updated
-deleted
-deleted
base: rdflib::Graph
base: rdflib::Graph
-old_graph
-old_graph
-base
-...Text is not SVG - cannot display diff --git a/docs/_static/wrapper_lifecycle.svg b/docs/_static/wrapper_lifecycle.svg new file mode 100644 index 0000000..eb645f5 --- /dev/null +++ b/docs/_static/wrapper_lifecycle.svg @@ -0,0 +1,4 @@ + + + +
User side
User...
Developer side
Devel...
Open
session
Open...
Release the resources used by your software. For example:
- terminate the running process
- close the connection to a database
- close files that are not needed
Release the resources used b...
Populate the base session so that it represents the initial state of the software.
For example:
- given a path to a simulation settings file,
  populate the session with entities
  descibing the contents of the file
- populate the session with dataset
  individuals after connecting to a data
  repository
Populate the base session so that it r...
Secure access to the resources used by your software. For example:
- spawn simulation engine objects
- open a connection to a database
- open files that need to be accessed
Secure access to the resources us...
commit
commit
`commit`
method
`commit`...
Session
closed
Session...
`populate` method
`populate` me...
`open`
method
`open`...
`close`
method
`close`...
Perform a computation using the data currently present on the software's data structures and update the base session to reflect the changes afterwards. For example:
- run a simulation
Perform a computation using the d...
`compute`
method
(optional)
`compute`...
Reflect user's changes on the session in the software's data structure. For example:
- configure a simulation's settings
- update an SQL table
- modify a file

The difference with respect to the `compute` method is that this method cannot update the content's of the session itself.

Reflect user's changes on the...
compute
compute
No
No
Session ready
Session ready
information
is read
informatio...
RDF manipulation methods
(advanced, optional)
RDF manipulation methods...
File manipulation methods (advanced, optional)
File manipulation methods...
information
is added or removed
informatio...
removed
removed
added
added
`remove`
method
`remove`...
`add`
method
`add`...
Intercepts triple pattern queries. Can be used to generate information on the fly.
Intercepts triple pattern qu...
Files
Files
No
No
Yes
Yes
Yes
Yes
`save`
method
`save`...
Receive an identifier and a file handle and save the contents somewhere.
Receive an identifier and a f...
file is downloaded
file is dow...
Main methods
Main meth...
`load`
method
`load`...
Receive an identifer and retrieve the corresponding file.
Receive an identifer and retr...
`triples`
method
`triples`...
These methods operate at the RDF triple level. When a triple (or pattern) is added or removed, they can intercept the operation and decide whether the triple should go to the base graph or not when a commit operation is executed. The intercepted triples get stored in a buffer that is accessible during the commit operation.

They are useful when one wants to prevent storing the same information twice (in the base graph and in the software's data structure).
These methods operate at the RDF tri...
close
closeText is not SVG - cannot display From 825dadc91a901cf68269b43466474b5b01d067b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:50:49 +0200 Subject: [PATCH 3/8] Improve introduction --- docs/developers/wrappers.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md index 64fe826..9cfda05 100644 --- a/docs/developers/wrappers.md +++ b/docs/developers/wrappers.md @@ -1,19 +1,18 @@ # Wrapper development SimPhoNy Wrappers are software components that transform data from the -assertional-knowledge form that they take in SimPhoNy's -[sessions](../usage/sessions/introduction.ipynb) to the data structures of +assertional-knowledge form to the data structures of other software and back. Wrappers are abstracted to users as [sessions](../usage/sessions/introduction.ipynb), which may be viewed -as "boxes" where ontology individuals can be placed +as "boxes" where ontology individuals can be stored. Sessions work in a way similar to databases. To start using them, one first -has to “open” or “connect” to them. After that, changes can be performed on the -data they contain, but such changes are not made permanent until a “commit” is +has to “open” or “connect” to them. After that, changes can be made on the +data they contain, but such changes are not permanent until a “commit” is performed. When one finishes working with them, the connection should be “closed”. Unconfirmed changes are lost when the connection is “closed”. -The development of a wrapper involves implementing: +Therefore, developing a wrapper involves crafting: - An abstraction of the concepts handled by the software as terminological knowledge, that can then be used to represent the information as assertional @@ -21,10 +20,8 @@ The development of a wrapper involves implementing: - A database-like interface that is used by SimPhoNy to communicate with the software. -For the latter, SimPhoNy defines a _Wrapper API_ that can be implemented by the -developer. - -## Static perspective +For the latter, SimPhoNy defines the _Wrapper API_, that must be implemented by +the developer. The static view on the SimPhoNy's wrapper API highlights the software components that the user and wrapper developer interact with, and how they From 3fadc4da8c218a8aaf79541324dc06cece7f89a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:51:44 +0200 Subject: [PATCH 4/8] `Wrapper` abstract class section --- docs/_static/object_diagram.svg | 2 +- docs/developers/wrappers.md | 32 ++++++++++++++++---------------- 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/_static/object_diagram.svg b/docs/_static/object_diagram.svg index c82030e..ba2ae47 100644 --- a/docs/_static/object_diagram.svg +++ b/docs/_static/object_diagram.svg @@ -1,4 +1,4 @@ -
session:
simphony_osp
.session::Session
session:...
Atom
Atom
Velocity
(1, 3, 5)
Velocity...
Position
(0, 7, 2)
Position...
User side
User...
Developer side
Devel...
Internal objects
Internal objects
graph: rdflib::Graph
graph: rdflib::Graph
driver: 
simphony_osp
.interfaces
.interface.py
::InterfaceDriver
driver:...
-graph
-graph
-store
-store
-interface
-interface
«interface»
wrapper:
simphony_osp
.development::Wrapper
«interface»...
-driver
-driver
old_graph: rdflib::Graph
old_graph: rdflib::Gr...
new_graph: rdflib::Graph
new_graph: rdflib::Gr...
buffer: rdflib::Graph
buffer: rdflib::Graph
session_base:
simphony_osp.session
::Session
session_base:...
session_old:
simphony_osp.session
::Session
session_old:...
session_new:
simphony_osp.session
::Session
session_new:...
added: set
added: set
updated: set
updated: set
deleted: set
deleted: set
-new_graph
-new_graph
-buffer
-buffer
-session_base
-session_base
-session_old
-session_old
-session_new
-session_new
-added
-added
-updated
-updated
-deleted
-deleted
base: rdflib::Graph
base: rdflib::Graph
-old_graph
-old_graph
-base
-...Text is not SVG - cannot display +
session:
simphony_osp
.session::Session
session:...
Atom
Atom
Velocity
(1, 3, 5)
Velocity...
Position
(0, 7, 2)
Position...
User side
User...
Developer side
Deve...
graph:
rdflib::Graph
graph:...
driver: 
simphony_osp
.interfaces
.interface.py
::InterfaceDriver
driver:...
-graph
-graph
-store
-store
-interface
-interface
«interface»
wrapper:
simphony_osp
.development::Wrapper
«interface»...
-driver
-driver
old_graph:
rdflib::Graph
old_graph:...
new_graph:
rdflib::Graph
new_graph:...
buffer:
rdflib::Graph
buffer:...
session_base:
simphony_osp.session
::Session
session_base:...
session_old:
simphony_osp.session
::Session
session_old:...
session_new:
simphony_osp.session
::Session
session_new:...
added: set
added: set
updated: set
updated: set
deleted: set
deleted: set
-new_graph
-new_graph
-buffer
-buffer
-session_base
-session_base
-session_old
-session_old
-session_new
-session_new
-added
-added
-updated
-updated
-deleted
-deleted
base: rdflib::Graph
base: rdflib::Graph
-old_graph
-old_graph
-base
-...Text is not SVG - cannot display diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md index 9cfda05..148cb31 100644 --- a/docs/developers/wrappers.md +++ b/docs/developers/wrappers.md @@ -23,23 +23,17 @@ Therefore, developing a wrapper involves crafting: For the latter, SimPhoNy defines the _Wrapper API_, that must be implemented by the developer. -The static view on the SimPhoNy's wrapper API highlights the software -components that the user and wrapper developer interact with, and how they -are connected. +## `Wrapper` abstract class -As shown in the diagram below, the user sees a SimPhoNy session object, -which abstracts all the complexity, appearing just as a simple "box" -that can store ontology individuals and where relevant -operations and queries can be run. - -However, the session object is an interface to an -[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html). -As the user interacts with the session, triples from the underlying graph are -being queried, added or removed. +The database-like interface used by SimPhoNy to communicate with the software, +called _Wrapper API_, is defined by the `simphony_osp.development.Wrapper` +abstract class. Objects belonging to the `Wrapper` class are indirectly +controlled by the interactions between the user and session objects, as the +diagram below shows.
-![Static perspective](../_static/object_diagram.svg) +![Connection between sessions and wrappers](../_static/object_diagram.svg)
@@ -51,8 +45,13 @@ relevant from a developer's perspective._
-The [RDFLib](https://github.com/RDFLib/rdflib) library provides a further -abstraction, the +SimPhoNy makes use of the [RDFLib](https://github.com/RDFLib/rdflib) library to +handle [RDF](https://www.w3.org/TR/rdf-concepts/) data. Thus, the session is in +fact an interface to an +[RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html). +As the user interacts with the session, triples from the underlying graph are +queried, added or removed. +The library also provides a further abstraction, the [store](https://rdflib.readthedocs.io/en/stable/_modules/rdflib/store.html). Stores abstract [triplestores](https://en.wikipedia.org/wiki/Triplestore), a kind of database that can store collections of RDF graphs in the form of @@ -72,7 +71,8 @@ Perhaps the most important of all is the **base graph**. The base graph is an [RDFLib Graph](https://rdflib.readthedocs.io/en/stable/intro_to_graphs.html) that is accessible from the wrapper object, and must be kept in sync with the software's data structures at all times, as it constitutes their -**RDF representation**. +**RDF representation**. The goal of the SimPhoNy Wrapper API is to facilitate +this task to the developer. ```{note} If an From 762f0f313db3af2bc0599e86f79826319a09ab62 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:52:20 +0200 Subject: [PATCH 5/8] API Overview section --- docs/_static/wrapper_lifecycle.svg | 2 +- docs/developers/wrappers.md | 33 ++++++++++++++++++++---------- 2 files changed, 23 insertions(+), 12 deletions(-) diff --git a/docs/_static/wrapper_lifecycle.svg b/docs/_static/wrapper_lifecycle.svg index eb645f5..e6294c9 100644 --- a/docs/_static/wrapper_lifecycle.svg +++ b/docs/_static/wrapper_lifecycle.svg @@ -1,4 +1,4 @@ -
User side
User...
Developer side
Devel...
Open
session
Open...
Release the resources used by your software. For example:
- terminate the running process
- close the connection to a database
- close files that are not needed
Release the resources used b...
Populate the base session so that it represents the initial state of the software.
For example:
- given a path to a simulation settings file,
  populate the session with entities
  descibing the contents of the file
- populate the session with dataset
  individuals after connecting to a data
  repository
Populate the base session so that it r...
Secure access to the resources used by your software. For example:
- spawn simulation engine objects
- open a connection to a database
- open files that need to be accessed
Secure access to the resources us...
commit
commit
`commit`
method
`commit`...
Session
closed
Session...
`populate` method
`populate` me...
`open`
method
`open`...
`close`
method
`close`...
Perform a computation using the data currently present on the software's data structures and update the base session to reflect the changes afterwards. For example:
- run a simulation
Perform a computation using the d...
`compute`
method
(optional)
`compute`...
Reflect user's changes on the session in the software's data structure. For example:
- configure a simulation's settings
- update an SQL table
- modify a file

The difference with respect to the `compute` method is that this method cannot update the content's of the session itself.

Reflect user's changes on the...
compute
compute
No
No
Session ready
Session ready
information
is read
informatio...
RDF manipulation methods
(advanced, optional)
RDF manipulation methods...
File manipulation methods (advanced, optional)
File manipulation methods...
information
is added or removed
informatio...
removed
removed
added
added
`remove`
method
`remove`...
`add`
method
`add`...
Intercepts triple pattern queries. Can be used to generate information on the fly.
Intercepts triple pattern qu...
Files
Files
No
No
Yes
Yes
Yes
Yes
`save`
method
`save`...
Receive an identifier and a file handle and save the contents somewhere.
Receive an identifier and a f...
file is downloaded
file is dow...
Main methods
Main meth...
`load`
method
`load`...
Receive an identifer and retrieve the corresponding file.
Receive an identifer and retr...
`triples`
method
`triples`...
These methods operate at the RDF triple level. When a triple (or pattern) is added or removed, they can intercept the operation and decide whether the triple should go to the base graph or not when a commit operation is executed. The intercepted triples get stored in a buffer that is accessible during the commit operation.

They are useful when one wants to prevent storing the same information twice (in the base graph and in the software's data structure).
These methods operate at the RDF tri...
close
closeText is not SVG - cannot display +
User side
User...
Developer side
Deve...
open
open
Open
session
Open...
commit
commit
Session
closed
Session...
`populate` method
`populate` me...
`open`
method
`open`...
`close`
method
`close`...
`compute`
method
(optional)
`compute`...
compute
compute
No
No
information
is read
informat...
RDF manipulation
RDF manipulation
File manipulation
File manipulation
information
added
or removed
informat...
removed
removed
added
added
`remove`
method
(optional)
`remove`...
`add`
method
(optional)
`add`...
No
No
Yes
Y...
Yes
Yes
`load`
method
(optional)
`load`...
`triples`
method
(optional)
`triples`...
close
close
file is downloaded
file is...
`commit`
method
`commit`...
Session
ready
Session...
`delete`
method
(optional)
`delete`...
Yes
Y...
Yes
Yes
Files
Files
`save`
method
(optional)
`save`...Text is not SVG - cannot display diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md index 148cb31..b9e2099 100644 --- a/docs/developers/wrappers.md +++ b/docs/developers/wrappers.md @@ -82,16 +82,17 @@ implement a SimPhoNy wrapper. Just create a graph using the plug-in, and set it as the base graph for the SimPhoNy wrapper object. ``` -The goal of the SimPhoNy wrapper API is to facilitate this task to -the developer. +## API Overview -## Dynamic perspective +The [flowchart](https://en.wikipedia.org/wiki/Flowchart) below illustrates the +**lifecycle** of a session connected to a wrapper object: from its creation to +the moment it is closed. -The dynamic view on the SimPhoNy's wrapper API offers an insight on how the -interactions between the user and the session object translate into calls to -the methods of the wrapper object. The diagram below illustrates the -**lifecycle** of a session connected to a wrapper object: from its creation -to the moment it is closed. +A sequence of method calls is executed as a +consequence of each possible action the user can take. Each sequence is +represented using a different color, and the action that triggers it is written +next to its accompanying arrow, that points to the first method call in the +sequence.
@@ -99,14 +100,24 @@ to the moment it is closed.
-_Flowchart showing the actions a user can perform and how they translate -to calls to the methods of the wrapper object, that the wrapper developer must -implement._ +_Flowchart showing the catalogue of possible user actions and how they +translate to calls to the methods of the wrapper class, that the wrapper +developer must implement._
+The `Wrapper` object is spawned when the user opens the session. The `open` and +`populate` methods are then subsequently called in order to gain access to the +resources needed by the software and pre-populate the session with ontology +individuals if necessary. After that, the session is ready to be used. The user +may then access or modify the assertional knowledge (triggering the optional, +low-level RDF manipulation methods), access files associated to ontology +individuals belonging to the _File_ class (triggering the `load` method), +add or change files, commit the changes or request the software to compute new +results. When the user is done, the session is closed. + ## API Specification TODO From a5a07f2f0aea9dbe9c3632922e62e50e1ca760b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:52:53 +0200 Subject: [PATCH 6/8] API Specification and packaging template --- docs/developers/wrappers.md | 162 +++++++++++++++++++++++++++++++++--- 1 file changed, 152 insertions(+), 10 deletions(-) diff --git a/docs/developers/wrappers.md b/docs/developers/wrappers.md index b9e2099..cd34a17 100644 --- a/docs/developers/wrappers.md +++ b/docs/developers/wrappers.md @@ -120,22 +120,164 @@ results. When the user is done, the session is closed. ## API Specification -TODO +This section describes the expected behavior of all methods from the Wrapper +API. -## Writing your own wrapper +### Main methods -This page is meant to offer a mid-level view on what is a SimPhoNy wrapper -and how do they work. If you are interested in developing one, please check the -[wrapper development repository](https://github.com/simphony/wrapper-development), -where a template for building and packaging a wrapper can be found. +#### `open` + +Secure access to the resources used by your software. For example: + +- spawn simulation engine objects +- open a connection to a database +- open files that need to be accessed + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.open +``` + +#### `populate` + +Populate the base session so that it represents the initial state of the +software. For example: + +- given a path to a simulation settings file, + populate the session with entities + descibing the contents of the file +- populate the session with dataset + individuals after connecting to a data + repository + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.populate +``` + +#### `commit` + +Reflect user's changes on the session in the software's data structure. For +example: + +- configure a simulation's settings +- update an SQL table +- modify a file + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.commit +``` + +The difference with respect to the `compute` method is that this method should +not update the content's of the session itself. + +#### `close` + +Release the resources used by your software. For example: + +- terminate the running process +- close the connection to a database +- close files that are not needed + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.close +``` + +#### `compute` _(optional)_ + +Perform a computation using the data currently present on the software's data +structures and update the base session to reflect the changes afterwards. +For example: + +- run a simulation + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.compute +``` + +#### `__init__` _(optional)_ + +The `__init__` method allows the user to provide extra +parameters in the form of JSON-serializable keyword arguments. + +```{note} +This method does not appear on the flowchart for simplicity, but is executed +before the `open` method. +``` + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.__init__ +``` + +### File manipulation methods + +#### `load` _(optional)_ + +Receive an identifier of a file individual and retrieve the corresponding file. + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.load +``` + +#### `save` _(optional)_ + +Receive an identifier of a file individual and a file handle and save the +contents somewhere. + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.save +``` + +#### `delete` _(optional)_ + +Receive the identifier of a file individual and delete the stored file. + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.delete +``` + +### RDF manipulation methods + +These methods operate at the RDF triple level. When a triple (or pattern) is +added or removed, they can intercept the operation and decide whether the +triple should go to the base graph or not when a commit operation is executed. +The intercepted triples get stored in a buffer that is accessible during the +commit operation. + +They are useful when one wants to prevent storing the same information twice +(in the base graph and in the software's data structure). + +#### `add` _(optional)_ + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.add +``` + +#### `remove` _(optional)_ + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.remove +``` + +#### `triples` _(optional)_ + +```{eval-rst} +.. autofunction:: simphony_osp.development.Wrapper.triples +``` + +## Packaging template + +This page is meant to offer a mid-level view on what SimPhoNy Wrappers are +and how do they work. If you are interested in developing one, you may find a +template for building and packaging a wrapper in the +[wrapper development repository](https://github.com/simphony/wrapper-development). ```{warning} You are reading the documentation of a release candidate version of -SimPhoNy. This version has not yet been fully documented. +SimPhoNy. -In particular, the contents of the +Some details have not yet been fully documented. In particular, the +contents of the [wrapper development repository](https://github.com/simphony/wrapper-development) are still outdated. Consider using the -[SimLAMMPS wrapper code](https://github.com/simphony/simlammps) as -a template instead until the repository is updated. +[SimLAMMPS wrapper code](https://github.com/simphony/simlammps) +as a template instead until the repository is updated. ``` From d06f3dd99127f9a194ca773f5ac155929d0839d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:54:27 +0200 Subject: [PATCH 7/8] Update api_reference.md --- docs/api_reference.md | 21 +++++++++++++++++++++ docs/conf.py | 9 +++++++++ 2 files changed, 30 insertions(+) diff --git a/docs/api_reference.md b/docs/api_reference.md index b091b11..60718a1 100644 --- a/docs/api_reference.md +++ b/docs/api_reference.md @@ -163,3 +163,24 @@ to instantiate such class by yourself**, and thus doing so is unsupported. .. autofunction:: simphony_osp.tools.relationships_between ``` + +## Development + +```{eval-rst} +.. autoclass:: simphony_osp.development.Wrapper + :members: open, populate, commit, compute, close, load, save, delete, add, remove, triples + :special-members: __init__ + +.. autoclass:: simphony_osp.development.BufferType + :members: + :undoc-members: + :show-inheritance: + +.. autoclass:: simphony_osp.development.Operations + :members: + :undoc-members: + :show-inheritance: + :special-members: __init__ + +.. autofunction:: simphony_osp.development.find_operations +``` diff --git a/docs/conf.py b/docs/conf.py index 352c672..009799c 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -68,7 +68,9 @@ def setup(app): + import simphony_osp.development import simphony_osp.ontology + import simphony_osp.session # Override names and modules of members of "simphony_osp.ontology" in the # API reference. @@ -83,3 +85,10 @@ def setup(app): item = getattr(simphony_osp.session, obj) setattr(item, "__name__", obj) setattr(item, "__module__", simphony_osp.session.__name__) + + # Override names and modules of members of "simphony_osp.development" in + # the API reference. + for obj in simphony_osp.development.__all__: + item = getattr(simphony_osp.development, obj) + setattr(item, "__name__", obj) + setattr(item, "__module__", simphony_osp.development.__name__) From 4a350ed01e1d6b628b921695965f71dd7c06b8b4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Manuel=20Dom=C3=ADnguez?= Date: Mon, 26 Sep 2022 14:54:51 +0200 Subject: [PATCH 8/8] Edit release candidate warning. --- docs/index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/index.md b/docs/index.md index 660c6d2..78c5d94 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,8 +2,7 @@ ```{note} ⚠️ You are reading the documentation of a release candidate version of -SimPhoNy. This version has not yet been thoroughly tested, and its -functionality is not yet fully documented. The documentation for the latest +SimPhoNy. The documentation for the latest stable version of SimPhoNy can be found [here](https://simphony.readthedocs.io/en/latest/). ```