This module provides a plugin system that can be used in applications that require the ability to distribute functionality across external shared libraries. It provides the following functionality:

Build plugin source file into shared library
Monitor source files for changes and rebuild (hot code reloading)
Load/unload/reload shared library
Provide standard and custom callback framework
Allow shipping in binary-only mode (no source or hot code reloading) if required

The library requires --threads:on since the monitoring function runs in a separate thread. It also needs --gc:boehm to ensure that memory is handled correctly across multiple threads and plugins. This is true both for the main application as well as the individual plugins. To build in binary mode, the -d:binary flag should be used.

The boehm garbage collector or libgc can be installed using the package manager on most Linux distros and on OSX. For Windows, prebuilt 32-bit and 64-bit binaries are available here.

This module should be imported in the main application. Plugins should import the plugins/api module which provides typical functionality required by the shared library plugins.

The main procs of interest are:

initPlugins() which initializes the system and loads all plugins
syncPlugins() which should be called in the main application loop and performs all load/unload/reload functionality plus execution of any callbacks invoked
stopPlugins() which stops the system and unloads cleanly

Many of the procs and callbacks accessible from plugins can also be called from the main application if required. Refer to the documentation for the plugins/api module for more details.

Types

CmdData = ref object params*: seq[string] ## Sequence of string params including callback ## name to send callback pparams*: seq[pointer] ## Sequence of pointer params to send objects to ## the callback failed*: bool ## Whether callback succeeded or failed returned*: seq[string] ## Sequence of strings returned by callback preturned*: seq[pointer] ## Sequence of pointers returned by callback: Object to send params and receive returned values to callbacks
Plugin = ref object manager*: PluginManager ## Access plugin manager from plugin name*: string ## Name of the plugin path: string handle: LibHandle depends: seq[string] ## Plugins this plugin depends on dependents: HashSet[string] ## Plugins that depend on this plugin pluginData: pointer ## Pointer to store any type T within plugin to make ## data accessible across all callbacks within plugin ## Used by `getPluginData()` and `freePluginData()` onDepends: proc (plugin: Plugin; cmd: CmdData) onLoad: proc (plugin: Plugin; cmd: CmdData) onUnload: proc (plugin: Plugin; cmd: CmdData) onTick: proc (plugin: Plugin; cmd: CmdData) onNotify: proc (plugin: Plugin; cmd: CmdData) onReady: proc (plugin: Plugin; cmd: CmdData) cindex: HashSet[string] callbacks: Table[string, proc (plugin: Plugin; cmd: CmdData)]: Plugin state information is stored in this object - each plugin has an instance of this
Run = enum executing, stopped, paused: States of the plugin system - can be changed using the ppause, presume and pstop global callbacks
PluginManager = ref object run*: Run ## State of system ready*: bool ## True when all plugins are loaded cli*: seq[string] ## Commands to run when system is ready tick: int pmonitor: ptr PluginMonitor plugins: OrderedTable[string, Plugin] pluginData: Table[string, pointer] callbacks: Table[string, pointer]: Manager of all loaded plugins and callbacks

Procs

proc newShared[T](): ptr T

Allocate memory of type T in shared memory

proc freeShared[T](s: var ptr T)

Free shared memory of type T

proc getManagerData[T](plugin: Plugin): T

Use this proc to store any type T in the plugin manager. Data will persist across plugin unload/reload and can be used to store information that requires such persistence.

Only first call allocates memory. Subsequent calls returns the object already allocated before.

Ensure freeManagerData() is called to free this memory.

import plugins/api

type
  PlgData = object
    intField: int

pluginLoad:
  var pData = getManagerData[PlgData](plugin)
  pData.intField = 5

pluginTick:
  var pData = getManagerData[PlgData](plugin)
  pData.intField += 1

proc freeManagerData[T](plugin: Plugin)

Use this proc to free memory allocated in the plugin manager with getManagerData()

import plugins/api

type
  PlgData = object
    intField: int

proc reloadAll(plugin: Plugin, cmd: CmdData) {.pluginCallback.} =
  freeManagerData[PlgData](plugin)
  var plgData = getManagerData[PlugData](plugin)

proc getPluginData[T](plugin: Plugin): T

Use this proc to store any type T within the plugin. Data will be accessible across plugin callbacks but will be invalid after plugin unload.

Only first call allocates memory. Subsequent calls returns the object already allocated before.

Ensure freePluginData() is called to free this memory before plugin unload.

import plugins/api

type
  PlgData = object
    intField: int

pluginLoad:
  var pData = getPluginData[PlgData](plugin)
  pData.intField = 5

pluginTick:
  var pData = getManagerData[PlgData](plugin)
  pData.intField += 1

proc freePluginData[T](plugin: Plugin)

Use this proc to free memory allocated within the plugin with getPluginData()

import plugins/api

type
  PlgData = object
    intField: int

pluginUnload:
  freePluginData[PlgData](plugin)

proc splitCmd(command: string): tuple[name, val: string] {...}{.raises: [], tags: [].}

Split "xxx yyy zzz" into "xxx" and "yyy zzz"

proc newCmdData(command: string): CmdData {...}{.raises: [], tags: [].}

Create new CmdData with command split using os.parseCmdLine() and stored in CmdData.params for processing by receiving callback

proc getVersion(): string {...}{.raises: [], tags: [].}

Get the Git version hash - repo state at compile time

proc getVersionBanner(): string {...}{.raises: [ValueError], tags: [].}

Get the version banner which includes Git hash if any and compiler version used to build main application

proc quit(manager: PluginManager) {...}{.raises: [], tags: [].}

Stop the plugin manager

proc notify(manager: PluginManager; msg: string) {...}{.raises: [Exception, KeyError], tags: [RootEffect].}

Invoke pluginNotify() across all plugins with msg as argument

proc plist(manager: PluginManager): seq[string] {...}{.raises: [], tags: [].}

Return a list of all loaded plugins

proc pload(manager: PluginManager; cmd: CmdData) {...}{.raises: [], tags: [].}

Reload all plugins if CmdData.params is empty, else (re)load the plugin(s) specified

proc punload(manager: PluginManager; cmd: CmdData) {...}{. raises: [KeyError, Exception, ValueError], tags: [RootEffect].}

Unload all plugins if CmdData.params is empty, else unload the plugin(s) specified

proc presume(manager: PluginManager) {...}{.raises: [Exception, KeyError], tags: [RootEffect].}

Resume plugin monitor - monitor plugin files and recompile and reload if changed

proc ppause(manager: PluginManager) {...}{.raises: [Exception, KeyError], tags: [RootEffect].}

Pause the plugin monitor - plugin files are not monitored for changes

This helps during development of plugins if source files need to be edited for an extended period of time and saving incomplete/broken code to disk should not lead to recompile and reload.

proc pstop(manager: PluginManager) {...}{.raises: [Exception, KeyError], tags: [RootEffect].}

Stop the plugin monitor thread - loaded plugins will stay loaded but the monitor thread will exit and no longer monitor plugins for changes

proc getPlugin(manager: PluginManager; name: string): Plugin {...}{.raises: [KeyError], tags: [].}

Get plugin by name - if no such plugin, result.isNil will be true

proc getCallback(manager: PluginManager; pname, callback: string): proc ( plugin: Plugin; cmd: CmdData) {...}{.raises: [KeyError], tags: [].}

Get custom callback by plugin name and callback name

import plugins

var
  cmd = newCmdData("callbackparam")
  plugin = getPlugin("plugin1")
  callback = getCallback(manager, "plugin1", "callbackname")
if not callback.isNil:
  callback(plugin, cmd)

proc call(manager: PluginManager; callback: string; cmd: CmdData) {...}{. raises: [KeyError, Exception, ValueError], tags: [RootEffect].}

Invoke custom callback across all plugins by callback name

CmdData.params should only include the parameters to pass to the callback

import plugins

var
  cmd = newCmdData("callbackparam")
call(manager, "callbackname", cmd)

proc callPlugin(manager: PluginManager; pname, callback: string; cmd: CmdData) {...}{. raises: [KeyError, Exception], tags: [RootEffect].}

Invoke custom callbacks by plugin name and callback name

CmdData.params should only include the parameters to pass to the callback

import plugins

var
  cmd = newCmdData("callbackparam")
callPlugin(manager, "plugin1", "callbackname", cmd)

proc callCommand(manager: PluginManager; cmd: CmdData) {...}{. raises: [Exception, KeyError, ValueError], tags: [RootEffect].}

Invoke custom callbacks in a command line format

CmdData.params should include the callback name and will be searched across all loaded plugins.

import plugins

var
  cmd = newCmdData("callbackname callbackparam")
callCommand(manager, cmd)

proc getCommandResult(manager: PluginManager; command: string): seq[string] {...}{. raises: [Exception, KeyError, ValueError], tags: [RootEffect].}

Shortcut for running a callback defined in another plugin and getting all string values returned

CmdData.params should include the callback name and will be searched across all loaded plugins.

import plugins/api

proc somecallback(plugin: Plugin, cmd: CmdData) {.pluginCallback.} =
  var
    ret = getCommandResult(plugin, "othercallback param1 param2")

# Assume this callback is in another plugin
proc othercallback(plugin: Plugin, cmd: CmdData) {.pluginCallback.} =
  for param in cmd.params:
    cmd.returned.add param & "return"

proc getCommandIntResult(manager: PluginManager; command: string; default = 0): seq[int] {...}{. raises: [Exception, KeyError, ValueError], tags: [RootEffect].}

Shortcut for running a callback defined in another plugin and getting all integer values returned

If no value is returned, return the default value specified.

CmdData.params should include the callback name and will be searched across all loaded plugins.

import plugins/api

proc somecallback(plugin: Plugin, cmd: CmdData) {.pluginCallback.} =
  var
    ret = getCommandResult(plugin, "othercallback 1 2")

# Assume this callback is in another plugin
proc othercallback(plugin: Plugin, cmd: CmdData) {.pluginCallback.} =
  for param in cmd.params:
    try:
      cmd.returned.add $(parseInt(param) * 2)
    except:
      cmd.returned.add ""

proc initPlugins(paths: seq[string]; cmds: seq[string] = @[]): PluginManager {...}{.raises: [ Exception, ResourceExhaustedError, IOError, ValueError, OSError, Defect], tags: [RootEffect, TimeEffect, ReadDirEffect, ReadIOEffect, ExecIOEffect].}

Loads all plugins in specified paths

cmds is a list of commands to execute after all plugins are successfully loaded and system is ready

Returns plugin manager that tracks all loaded plugins and associated data

proc stopPlugins(manager: PluginManager) {...}{.raises: [KeyError, Exception, ValueError], tags: [RootEffect].}

Stops all plugins in the specified manager and frees all associated data

proc syncPlugins(manager: PluginManager) {...}{.raises: [KeyError, Exception, ValueError], tags: [ ReadDirEffect, RootEffect, WriteDirEffect, TimeEffect, ReadIOEffect, WriteIOEffect].}

Give plugin system time to process all events

This should be called in the main application loop