Hydrafying your Application
While Hydra makes it very easy to expand your existing Delphi/Win32 projects with new code written in either .NET or Delphi, there is one major requirement that must be met before you can start writing your .NET plugins: your application architecture must be open for extensibility, so that the plugin-based approach can be used for future development.
Unless you have been using previous versions of Hydra when writing your application, chances are that your application architecture is more or less monolithic, with all functionality and features linked into a single project, and possibly deeply intertwined.
This article provides a guideline as to how to change that, to revise your application's architecture to be extensible and plugin-based. Please be aware that this task (and the complexity of it) will largely depend on your current implementation. As such, this article does not provide a detailed step-by-step process that can be followed along blindly - instead, it tries to explain the basic concepts and address the issues and design decisions that are commonly encountered during this task. You might find some of the concepts discussed in here to be non-issues for your specific case, or it might be possible that your project might need a slightly different approach to certain changes.
What are Plugins?
Hydra uses the term "plugins" to describe extensibility modules that integrate with and extend your application. It is important to realize that these plugins can, but do not necessarily have to, resemble the "traditional" sense as end-user-managed extensions to your application.
If you consider Adobe Photoshop filter plugins, or third party experts and add-ins in Delphi or Visual Studio, these are plugins that are added by the end-user to extend the application. Usually, the program will provide some sort of user interface where the user can load and unload these extensions, and manage them. The application will then provide access to these extensions through a common front end, for example the "Filters" menu in Photoshop, or display the plugins on separate tabs.
Hydra does provide support for these types of plugins-enabled applications, but it also allows you to use plugins in your internal application design to partition your project logic into modules. The end-user of your application might never know that the program is modular.
Plugins might be used to implement different aspects of your application's user interface or calculations. An accounting application, for example, might use a plugin to implement the current year's tax rules, making it very easy to update the logic next year without updating the main application.
Benefits of using a modular, plugin-based approach to development include:
- in general a better abstracted and more flexible design.
- the ability to implement parts of your application in Delphi and others in .NET.
- the ability to easily switch and replace parts of your functionality without touching the main application; possibly even while the main application is running.
- optionally, opening your application to extensibility by end-users or third parties.
- allowing these extension developers to pick their platform of choice to implement the plugins.
Now that we have discussed the benefits of building plugin-based applications, let's have a look at what's involved in opening your existing project up to extensibility with Hydra.
Planning Your Architecture
The first thing to consider is how you want to modularize your application. There are various options available here, and the complexity of the solution will largely depend on the complexity of your original project, and the range of features you want to provide.
A simple case would be to move some existing application-logic into a non-visual plugin. This can be achieved by literally writing a handful of lines of code, enabling you to move the logic out to a plugin (and possibly migrate or rewrite in .NET). An example of this might be the accounting application mentioned above, where only the tax calculation logic is extracted into a plugin.
A more complex situation might be to repartition your entire application to be plugin-based, eventually ending up with a hollow shell of a host application that obtains all functionality through plugins. This is very often the best solution when starting a new project from scratch because it provides the most flexibility, as it gives you a host that can do pretty much everything, given the right plugin. An example for such a system is Visual Studio itself, where the basic IDE simply provides an application framework of menus, window management and little else, and all the core functionality the end-user sees is actually added by so-called packages that extend the product.
As a first step, it is important to review your application, both its current state and your future intentions with it, to determine what kind of solution you need and what will be the best way to partition it. Candidates for extraction into plugins could include:
- functionality that performs "parallel" tasks or is similar by interface, for example:
- a list of Photoshop-like filters,
- different panes or tab-sheets of your application's UI,
- functionality that should be replaceable (whether by the end-user or by you), such as:
- One report view vs. another,
- One calculation module vs. another,
- One language spell-checker vs. another language.
- functionality that you simply want to move out of your main project for other reasons, for example to:
- update and maintain it separately from the main system,
- deploy or license it separately from the main application,
- translate it into .NET,
- update to newer versions at runtime without restarting the application.
These can be visual parts of your user interface (typically individual tab-sheets or MDI windows, but also simple parts of your general application's UI) or non-visual code.
Code that will be moved into plugins can be homogenous across plugins, but it needn't be.
Examples of homogenous interfaces would be plugins that run in parallel or can replace one another. For example, one can easily imagine that all Photoshop filters implement a common interface, allowing the application to access any given filter in a consistent manner. On the other hand, when extracting three independent calculation modules into plugins, each plugin might get its own, specific interface, tailored to the exact need of the application logic it encompasses.
In other words, you might find it sensible to move a certain feature out into a plugin, even if there'll never be two implementations of the same, and there will only ever be this one plugin loaded. An example of this is the Hydra IDE integration itself: the interface import functionality (which we'll discuss later in this article) is implemented as a Hydra plugin in .NET code. The IDE integration uses the Hydra library itself to load this plugin into the IDE (be it Delphi or Visual Studio), to access its functionality.
Defining Your Interfaces
Once you have decided how to partition your application across plugins, you will need to consider how your core application will communicate with the plugins, and vice versa.
Hydra uses interfaces to allow the host application and plugins to communicate with each other. Using interfaces, it is easy to define the methods and properties made available by a plugin (the plugins "interface", so to speak) in an abstract way, decoupled from any concrete plugin implementation.
The host application can then use all plugins that provide a certain interface in a consistent manner.
Most typically, plugins will implement interfaces and passively react to calls made from the host application. However, it is entirely possible for the host to implement interfaces of its own, allowing plugins to call back into the host (for example to trigger actions or notify the host of status or changes to the plugins internal state).
The Hydra library provides a number of standard interfaces to perform tasks common to many plugins, such as hosting a visual plugin inside the host's UI, handling menu- and toolbar-merging, etc. When creating a visual or non-visual plugin from the templates or when using the provided base classes, these kinds of functionality are provided for you out of the box.
However, for most non-trivial solutions, you will want to define your own custom interfaces (usually in addition to the predefined ones).
Existing plugin interfaces to consider include:
- IHYPlugin interface: provides basic plugin functionality such as accessing provided access to the Host object, etc.
- IHYVisualPlugin interface: contains basic functionality to show/hide/host visual plugins in the host application.
- IHYNonVisualPlugin interface: provides a Windows-service like Start/Stop interface for controlling plugins (ideal for plugins that do work on their own).
- Anything else: define your own plugin interface(s) to meet your needs.
You can decide which interfaces to implement on plugins, as dictated by your application's needs. You don't need to implement the same set (or even sub-set) of interfaces on all plugins, and you can have some plugins share certain interfaces, while others provide their own.
Consider the following options:
- You can have plugins share the same interface to use them in common scenarios, use them in parallel or switch between them transparently.
- Certain plugins can implement their own unique interface, depending on how they will be used.
You can use an interface to "detect" the type of a certain plugin. For example, if your host application has loaded 20 plugin modules, you can check for available interfaces to determine which plugins are serving specific areas of your application. Those plugins that implement interface A might end up showing in one area of your program, while those implementing interface B will be used for a different purpose.
When planning to go cross-platform, make sure to limit interfaces to the restrictions needed for cross-platform compatibility.
For starters, all interfaces used across .NET/Delphi boundaries must descend from IHYCrossPlatformInterface (defined in uHYCrossPlatformInterfaces.pas in Delphi, and in the RemObjects.Hydra.CrossPlatform namespace on .NET). Cross-platform interfaces must also specify a GUID, using Delphi's square-bracket GUID syntax or the .NET "System.Runtime.InteropServices.Guid" attribute.
The types used inside the interface (as method parameters and return values, and as property types) must be limited to simple types (Strings, Integers, Doubles, etc.) and other interfaces. Additional more advanced types can be passed, if you are familiar with .NET/COM Interop and use the appropriate MarshalAs attributes.
For obvious reasons, object references (whether basic TObject/System.Object or more concrete types) cannot be directly passed between Delphi and .NET. If you need to pass references to other objects, the best option is to define additional interfaces on these objects, and pass around the interfaces.
Obviously, the interfaces that you define for your plugins will need to cover all the functionality that your plugin wants to surface to the host. Because Host and plugin live in different compilation modules (or might even be implemented in different platforms), the host cannot directly access the methods defined on a specific plugin, or vice versa as the concrete type of object is not known at compile-time.
If there is an overlap in exposed functionality - for example if you find that all plugins must provide a certain method or number of method - it is usually a good idea to put the shared methods into common interface, rather than duplicating it. This way, the host can call the method(s) in question in a consistent manner, regardless of the other interfaces available on the plugin.
You can use either Delphi or .NET code to define your interfaces, and later import them into the other platform using Hydra's Interface Import IDE feature. Importing interfaces from .NET into Delphi is done based on the compiled assembly containing the interfaces, rather than the source code; this way, interfaces can be reliably imported, no matter what .NET language was used to define them. Importing interfaces from Delphi to .NET happens by parsing the Delphi source code.
Sharing Code between Plugins and Host
Another consideration is the sharing of code between plugins and host.
This is more of an issue for Delphi than it is for .NET, as the core Delphi RTL and VCL make certain assumptions. In particular, it is important to use packages when using a Delphi host with one or more plugin modules written in Delphi, so that the base RTL is shared between the two. Neglecting to do so will cause problems such as "Cannot assign a TStringList to a TStringList" errors, as two competing versions of the RTL and VCL exist in the same process.
Any code shared between Delphi plugins should probably be placed in packages. This includes:
- Any custom code you want to use in several of your plugins, or share between plugin and host. This can, for example, include base classes or function libraries.
- Any classes or types you want to pass from plugin to host, or back.
- Any third party components or visual controls you want to use in both visual plugins and the host (this is not an issue for code only in the host).
- The Hydra library and the core Delphi RTL/VCL libraries.
You are not limited to using the existing sets of packages provided by Delphi, Hydra and your third party component providers. You are completely free to structure your packages to fit your project's needs. As a matter of fact, it is often convenient to create one big base package containing the shared code, as it frees you from having to worry about package versioning as third party packages are updated (Please refer to When and why Hydra requires packages for more details on this topic).
Packaging options include:
- Use the standard runtime packages provided by Delphi, Hydra and third party vendors (and possibly one custom package for your own code).
- Use the base RTL/VCL packages, but put all the rest into one big custom package.
- Put everything, including the VCL, into one big package.
Packages are not an issue for a Delphi host that will load only managed/.NET plugins, or for Delphi plugins loaded into a managed/.NET host. In both of these scenarios, you can (but don't have to) build without packages.
Loading and Managing Plugins
The host application will use the Module Manager component to perform the core task of loading, managing and possibly unloading plugin modules.
How modules are loaded is entirely up to the host application; you can load modules given an exact, hard-coded filename (for example to load a specific module that provides core functionality needed by the application; this is the method the Hydra IDE integration uses, for instance), dynamically look for modules with a specific naming pattern (such as *Plugin.dll) in your application folder or a sub-folder, or use your own type of configuration database (for example Registry entries or an .ini file) to determine what to load.
Once modules are loaded, you can use the module manager to query for available plugins. Again, it is up to your application how to discover plugins for specific scenarios. You can use the CheckPluginInterface method to see whether a plugin type supports a specific interface (before actually creating an instance) and decide whether it's suitable for a particular scenario. You could also use plugin names (or namespaces) or check for certain attributes attached to (managed) plugins, using the CheckPluginAttribute method.
Once discovered, you can instantiate your plugins and host them in your application, or you can provide user interface placeholders that will create the plugin on demand, when the user accesses the application feature.
Some Hands-On Guidelines on Getting Started
Once all the planning is done and you have decided how to structure your project, it's time to get your hands dirty and start the actual conversion work.
If you identified multiple points of extensibility where you want to move features out into plugins, it might be sensible to start by concentrating on one of the areas first, and extract those parts into plugins. Once that has been completed and you have familiarized yourself with the process, the remaining areas can be processed in the same way.
First, start by extracting your app into plugins. Follow a procedure similar to this:
- Create a project group, and add one or more new plugin modules to it, for the functionality you want to extract. Remember that you can put multiple plugins into the same module, or have separate modules for each. What's best really depends on how tightly the plugins are coupled (for example, two plugins that belong together, say one to Import and one to Export a specific file format, might be best to go into the same module).
- Add the existing application or a new host application project.
- If needed, add a package project to contain your shared code and interfaces.
- Create new plugins to fill with the code you want to move.
- Move the code (and/or the UI) from your main application into the plugins. Depending on how your main app is structured, this might mean simply moving a unit from the main project into the plugin module, or cutting/pasting and restructuring the code around.
- Update your plugin classes to implement the custom interface(s) you defined (if any), and hook those methods up to the plugin's logic.
- Make your host plugin-aware by dropping a Module Manager centrally in the main application, typically in your main data module (which you should have!) or the main form.
- Add code to load plugin modules. There are several options:
- Load plugins on startup (say in OnCreate).
- Load plugins on user request (user could browse to the dll, or to a folder).
- Anything else you can think of.
- Integrate the loaded plugins. Discover plugin types by looking at implemented interfaces or metadata.
- Hook up plugins with the host application. This can happen in several ways, and largely depends on the types of plugins:
- Dynamically create menu items for the user to invoke plugins (e.g. like Photoshop filters).
- Add plugins to some list in the application (e.g. export filters in a File|Export dialog).
- Create tab-sheets or MDI windows to show visual plugins.
- Store plugin references internally for use directly from your host's code.
- Host plugins can be directly integrated into your main UI or store references to non-visual plugins so your application can call them when needed.
Once the above steps are completed, you have an extensible plugin based project. Depending on the approach you took, your application might look and work the same to the end-user as it did before, or you might have exposed new UI that allows the user to manage plugins and extensions himself.
In either case, the hard part is now over, and your application is ready to benefit from the features provided by Hydra. For example, you can now think about expanding it with .NET code.
Again, there are many options on how to proceed, including:
- Port one (or more) of your existing plugins to .NET (just to see it work, or because you're planning to expand them with new .NET technologies).
- Create a new plugin to replace or work in parallel with the existing ones.
The basic steps for adding new .NET based plugins to your project are:
- Create a new .NET "plugin module" project in Visual Studio or BDS.
- Import the custom interfaces, if any, using the Hydra IDE integration. If you plan on sharing the interfaces between multiple plugin modules, it might make sense to create a separate class library project to contain them (and any other code you might want to share later).
- Add new plugin classes to the module.
- Implement the necessary interfaces, if any, and hook them up.
That's it. If you deploy your plugin dlls in the right folders and with the right names, your Delphi host will automatically be able to load and use them.
This article has explored the task of converting a monolithic Delphi/Win32 project into an extensible application that uses Hydra to manage plugins written in either Delphi or .NET. As stated before, this is a task that very largely depends on your original project type and structure, and this guide is not intended to provide complete coverage over all possible scenarios.
However, if there are any areas that you feel are worthy discussing in more detail, or have any further questions in general, please do not hesitate to contact us and let us know.