v0.0.9

This release contains an adapter that redirects logs from the internal logger of the plugin system, hclog, to the global logger of GatewayD, zerolog. This release marks the end of the milestone 0.0.x, which was focused on laying the foundation for further development. I tried to achieve correctness and integrity in this milestone, however, there are some stability issues that I'll deal with in milestone 0.1.x. Going forward, documentation becomes an integral part of the project and will be written progressively.

Looking back at the previous releases, a lot has changed. The project was bootstrapped with a few lines of code as a proof of concept to mimic a L4 gateway. Now the GatewayD has a plug-based architecture with many components to help facilitate the inner-workings of the database gateway:

• Server: for accepting connections from the database clients, like psycopg2 in Python.
• Client: for connecting to databases using a raw TCP connection.
• Generic pool: used in a few places throughout the code to manage connections and plugins. The generic pool is a wrapper around the sync.Map from Go stdlib.
• Proxy: an object that connect servers and clients and passes traffic through. There are two types of proxies:
  • Fixed: a pool of clients will be created and connected to the database. They will be waiting ready to be proxied. The pool won't grow, so extra clients will be rejected. Queuing will be added in the future.
  • Elastic: an empty pool is created and each new connection from a database client creates a new connection to the database. The connections can be reused if chosen.
• Plugin system: can tap into the hooks and try to influence the behavior of the gateway by observe and manipulating data as it passes through between client and server. Plugins are managed by a plugin registry. The plugins can be written in any language that one can produce protobuf stubs and can interface through gRPC. The protocol file is available in plugin/v1/plugin.proto and the generated stubs for Go and the plugin interface are available in the same directory. There is a test plugin that shows how to work with hooks. Upon loading of each plugin, its dynamic configuration is read by calling its GetPluginConfig function. This effectively registers the list of hooks that plugin exposes with the hook system. All plugin functions (RPC) are unary for now.
• Hooks: callback function that plugins can register to. The server will run the hooks based on events, like onConfigLoaded and it causes the configuration options to be sent to plugins registered for that specific event. Each hook can trigger many plugins and they will be called based on their priority. The returned result of the hook from the plugin will be used to update the configuration dynamically (in-memory). Other hooks might have different side-effects. Some hooks are only notification hooks, so that the plugin can tell when an event happen, plus some pieces of information. The list of hooks are available here. The returned result of each hook from each registered plugin will be passed to the next plugin in the list.
  There is a verification function to check the signature of the parameters passed to the hook against the signature of the returned results and acts based on the policy chosen in the global config. There are two modes: strict and non-strict.
  • In strict mode, the parameter and return value signatures should match 1:1. In strict mode, either of the following three actions can be taken:
    1. ignore: ignore the result of the non-matching hook and continue with the next call to another plugin.
    2. abort: abort execution of the hook and return the last returned result from the previous hook.
    3. remove: ignores the result of the current registered hook and removes it from the registry, so the next call won't hit it again.
  • In non-strict, aka. permissive, mode, the parameters and the returned result should not match and the results of the hooks will be merged. This verification policy is called passdown.
• Logging: zerolog is used throughout the code to log events as they happen with various levels of importance. Loggers of other parts of the system has adapters to send logs to zerolog, as mentioned in What's changed? below.
• CLI: a run command with two options for reading: a) the global config file and b) the plugins config file.
• Config parser: there are two config file parsers that accept configuration in YAML format:
  • Global config (gatewayd.yaml): for setting up logger, clients, pool, proxy, server and plugins objects.
  • Plugin config (gatewayd_plugins.yaml): for adding the list of plugins to be loaded. Plugins priority is determined by the appearance of the plugin config in the file.
  • Dynamic config: there is a dynamic configuration hook, onConfigLoaded, to let plugins changes the global configuration before the server boots and after the global config is read, as demonstrated in the test plugin.
• Buf for managing protobuf files and generated stubs for the plugin system.
• Unit- and integration tests.
• CI workflows for a) testing unit-tests and b) testing integration with an example plugin.

What's Changed

• Add logging adapter for the plugin system by @mostafa in #45

Full Changelog: v0.0.8...v0.0.9

v0.0.8

This release includes several changes including the plugin system, major refactoring and adding tests. A new test plugin, gatewayd-plugin-test, is developed to help developers understand how to write a plugin for GatewayD in Go. Plugins can be written in practically any language that can produce and consume protobuf stubs and expose a gRPC interface. For various reasons, the Hashicorp's go-plugin is used to develop the plugin system.

What's Changed

• Refactorings before implementation of the plugin system by @mostafa in #39
• Plugin system by @mostafa in #40
• Add more tests by @mostafa in #42

Full Changelog: v0.0.7...v0.0.8

v0.0.7

I have heavily refactored the hook system and resolved a lot of issues and bugs I found along the way and added tests wherever needed. These are the major changes:

• buf for linting, building and generate stubs from protocol buffer files.
• Hashicorp's go-plugin is used to build the plugin system, as explained in #8.
• Introduce chaining of hook results (for use in chaining results of function in the plugins registered to each hook with their priority)
• Merge config from disk with the one retrieved from the plugins registered to the OnConfigLoaded hook.

What's Changed

• Refactor hook system by @mostafa in #35
• Non-strict (permissive) verification policy by @mostafa in #38

Full Changelog: v0.0.6...v0.0.7

v0.0.6

This release contains all the groundwork needed by the plugin system, including introduction of hooks, generic pools and other minor changes. The plugin system issue #8 has a lot of branches, which were dealt with in this release. I also added other issues based on TODOs in the code, which will be added or fixed in v0.0.x milestone.

To visualize how GatewayD works, I've created a flow diagram to help understand the inner workings and interconnections.

What's Changed

• Add hooks functionality, so plugins can attach to them by @mostafa in #21
• Use time.Duration instead of int for durations by @mostafa in #22
• Create a generic pool to contain client and server connections with customizable keys and values by @mostafa in #24

Full Changelog: v0.0.5...v0.0.6

v0.0.5

I added a config parser to accept a YAML file, and possibly other formats in the future, and configure and run a GatewayD instance. I also provided an example config file, gatewayd.yaml, to show how it works.

What's Changed

• Add config parser by @mostafa in #17

Full Changelog: v0.0.4...v0.0.5

v0.0.4

I added a basic CLI application skeleton with a run command to start a new instance of GatewayD.

What's Changed

• Add CLI application by @mostafa in #14

Full Changelog: v0.0.3...v0.0.4

v0.0.3

In this release I decoupled the objects from each other, so they can be configured separately. This lays the foundation for a CLI, configuration handling and the logger. I also replaced logrus with zerolog, and refactored the code.

What's Changed

• Decouple objects by @mostafa in #12
• Better logging with zerolog by @mostafa in #13

Full Changelog: v0.0.2...v0.0.3

v0.0.2

A lot has been changed since the previous release. In this release, I added as many tests as I can to lay the foundation for future work on the project. I also added CI to lint, test and soon report coverage of the code.

What's Changed

• Add unit tests by @mostafa in #10 (part of #5)
• Add CI with lint, test and (commented out) coverage reporting by @mostafa in #11 (part of #5)

Full Changelog: v0.0.1...v0.0.2

v0.0.1

The GatewayD project is just started and it was in the research/investigation phase for the last few months. The code is a WIP, in active development and unstable. So, I don't recommend using this anywhere for now. This is for me to keep track of releases according to the project board. The current release fulfills the requirements for the v0.0.x milestone.

What's Changed

• Connection pool and proxy by @mostafa in #1

Full Changelog: https://github.com/gatewayd-io/gatewayd/commits/v0.0.1