Plugin system

[mostafa]

Proposal:
Implement the plugin system and wire up parts of the system to it, like pools, query parser, and others.

Implementation:
There are three ways to implement the plugin or extension system, and each has its own pros and cons. The go-plugin is currently my preferred implementation, unless the complexity outweighs its benefits. A good reference is the Eli Bendersky's post about plugins in Go.

1. The Go plugin package, which requires binary shared objects to be shipped with the program.
2. The Hashicorp's go-plugin package that works with plugins over gRPC or NetRPC.
3. The xcaddy approach, which is also implemented for k6 as xk6, which compiles extensions with the program binary as a single package.

Tasks:

• Map the hook functions and the workflow to be exposed to plugins #19
• Create extra hook functions for start, in-between and end-states #20
• Create generic pool for client and gnet connections #18
• Create the plugin system and expose hooks to the plugins #25
• Plugin output chaining #23
• Use buf to manage protobuf files for plugins #34
• Write tests for the plugin system #41

Resources:

1. hashicorp/go-plugin
2. Hashicorp Plugin System Design and Implementation
3. 2016 Kickoff - Hashicorp & Go Plugin Architecture
4. GopherCon 2017 - Lightning Talk: Bob Argenbright - Simple Plugin Architectures in Go
5. Plugins and Go! - Kenneth Shaw
6. Go Plugins in a Nutshell / Nitzan Zada
7. An example doc article on CrowdSec plugin - Writing Plugin in Go
8. Another example on ConsenSys plugin - Develop plugins
9. A practical guide to Go-Plugin and other HashiCorp libraries
10. Backend plugins in Grafana uses go-plugin
11. Plugins in Go
12. Go Plugin System over WebAssembly (possible future plan)

Nice-to-have:

• The plugins can be distributed across a network (or the internet) via gRPC or other protocols, and doesn't necessarily need to reside in the same place as the gateway. Hashicorp's go-plugin doesn't support communication over the network yet, which seems to me to be because of security concerns.
• A plugin SDK, like https://github.com/hashicorp/packer-plugin-sdk.

[mostafa]

Released in v0.0.8.