PluginRefactor

<<TableOfContents: execution failed [Argument "maxdepth" must be an integer value, not "[2]"] (see also the log)>>

Proposal

Define the mechanism through which plugins interact with Cytoscape and can communicate with other plugins. This is closely related to the Layering RFC, as this describes the functionality we would want to expose. The mechanism need to be powerful enough to permit general operations applied to networks, maximizes reuse and allows for incorporation of new views, and addition of new import/export file types.

Background

The rich functionality offered through the plethora of plugins is Cytoscape’s greatest strength. As the software has evolved, and more plugins have been written, the lack of a defined plugin operation specification has become a problem. Currently plugins access core data structures directly so that any change in the core will break any number of plugins. Due to the tight coupling of the plugins to core code it is possible for a plugin to conflict with core processes and data structures, conflict with other plugins and even crash Cytoscape entirely. In particular, two problems are immediately apparent with the current architecture: the lack of resource management means that Cytoscape presents runtime robustness, performance and scalability problems; and the lack of backwards compatibility (through an established API or similar) means that plugins must be reimplemented between versions. By creating a clean and functional set access mechanisms for each layer (see Layering RFC) plugins should be easily able to continue extending the functionality of Cytoscape while isolating them from changes made to core structures.

Use Cases

• Write a plugin once, continues to work in subsequent versions
• Resource management to prevent plugins from crashing each other or Cytoscape
• Standardize public interfaces

Overview

There are three central questions that need answering:

• What functionality should be exposed to a plugin?
• How should the functionality be exposed to a plugin?
• How do we manage the life cycle of a plugin?

Other topics for discussion:

• Can we provide a unit test framework that plugins an use to test against cytoscape?
• How will plugins communicate with one another?
• How will events be propagated to/from the core?
• Can we manage plugin resources or tasks centrally?

What functionality should be exposed to a plugin?

Based on the packages suggested in the layering RFC several external API’s may be suggested. Only those that implement the interface that depends on the Model would be accessible to any implementation (such as scripting or headless mode) of Cytoscape that doesn’t use any other layers.

1. Model - Basic commands available on any model.network object (networks, nodes, edges, attributes, groups):
   • Select
   • De-select
   • Modify
   • Delete
   • Create
2. View - Utilize any current views available such as layouts and create entirely novel views. Event/thread resources would need to be handled within the view package and parceled out to external code (plugins) separately.
   • Create a pre-defined view on a model object
   • Delete a view on a model object
   • Modify a view on a model object
   • Add/Create a new type of view (ex. molecular pathway view)
3. IO
   • Add import file type
   • Add export file type
   • Add security information
4. Application - It may be useful to limit what/where things may be added to prevent confusion on the part of the user.
   • Add a menu

   • Add a panel (ex. CytoPanels)

   • Access resources (e.g. threads, connections)

How should the functionality be exposed to a plugin?

Opinions differ on how plugins should be managed, there are basically three different views:

1. Interfaces: a traditional approach would be to expose a set of interfaces which would remain stable between releases. The implementations of these interfaces could change, resulting in a simple (and understandable) “programming by contract” mechanism. While this is simple and elegant, problems with ensuring component compatibility will arise as requirements to change interface method signatures occur.
2. Component: allow for components to discover each other using a high level management system (e.g. OSGi). This does help to ensure component compatibility, but there are questions as to how the “plugin framework” (e.g. cytoscape core) can be componentized.
3. Aspects: annotating methods presents a modern mechanism for allowing support for plugins, and has been used successfully used in a number of containers. It can be used to overcome limitations associated with interface based programming (e.g. by allowing for dependency injection). While it will allow for the evolution of interfaces, there are problem as POJOs are not suitable for all plugins.

The correct solution can, and probably should, draw in all three views to allow for the development of a plugin system which is useful and maintainable.

Interfaces This is the simplest mechanism is to expose interfaces and finders for the associated instances. The diagram below illustrates how to find a specific model and listen for changes to the model.

Although interfaces provide for a degree of backwards compatibility, they are limited to a freeze on method signatures. Small incremental changes to interfaces can possibly be absorbed through the use of dual class loaders, with method call delegation being used.

The interfaces that we can expose are described in the Layered RFC.

Components A large number of component based frameworks and architectures exist, and range from simple patterns (e.g. JavaBeans) to dynamic discovery systems (e.g. OSGi). We can encourage plugin developers to follow/adopt a component based architecture enabling better support for both the discovery of plugins (runtime or otherwise) and provide a system for “auto updating”. The core components within Cytoscape could also be made available through OSGi, and inter-plugin communication could also be supported. This would require:

• Using a taxonomy to describe plugins (extension of what occurs currently for the plugin manager)
• Creation of a service registry to define where to register/find a plugin

Components do not remove the problems with backwards compatibility, but do provide for:

• Better organization of code
• Runtime discovery of plugins and inter-plugin communications
• better reliability, as the different versions of plugins and core code can be made available.

The core components that we can expose as services are:

• NetworkArrayModel component (in the end, there can be only one), which can be used to find the list of currently loaded models of graphs. The user can then listen for events for modifications.

• Application component to allow for menubar merging and similar operations
• Each of the views, these would each be a separate component with a unique name. A hierarchical taxonomy based on tasks (e.g. macro view) and domain (“single network view”) could also be used to ensure the system is both transportable to other platforms and is extensible.

Aspects AOP offers a means to provide for both runtime and compile time injection of code. It can be used to dramatically simply frameworks integration (e.g. EJB 2.x versus EJB 3), and allows for code to be easily transportable (e.g. POJOs). Unfortunately it is relatively easy to overuse aspects. Such overuse results in too much functionality being made available through a somewhat arbitrary list of annotations. By carefully considering if/when we wish to use this powerful, albeit confusing, programming construct we can alleviate some of the problems associated with plugin maintenance.

We would suggest encouraging the use of aspects to support:

1. The registration and naming of plugins e.g. http://static.springframework.org/maven2/spring-osgi

2. Replace any factory methods that are required with attributes (e.g. inversion of control)

3. Support multiple delivery mechanisms for plugin functionality (e.g. [WebService] tags)

4. To support plugin developers “nice” (thread management) use of resources (e.g. “link during configuration not compliation”)

Aspects can be made available to developers, and can be backed up by interfaces. In this way we can encourage, but not enforce, their usage.

How do we manage the life cycle of a plugin?

• This question will be answered in a separate RFC. The main questions are:
  • What is the life cycle of a plugin?
  • What resources do we need to manage?
  • Do we want plugins to run as web services or similar?

Can we provide a unit test framework that plugins an use to test against cytoscape?

Using clear interfaces, mock objects, and dependency injection, it should be fairly easy for plugins to test themselves.

How will plugins communicate with one another?

If plugins become OSGi bundles, then plugins should be able to communicate through whatever services that other plugins publish through the Service Registry should become available to those plugins.

How will events be propagated to/from the core?

This will be handled through the event handling framework described in RFC 52.

Can we manage plugin resources or tasks centrally?

We're exploring this issue in RFC 50.

Possible API

The rough outlines of a new Cytoscape API can be found here:

http://chianti.ucsd.edu/svn/cytoscape/branches/cytoscape-osgi/new-src/

Project Managment

Project Plan

PluginPlan.xls

Tasks and Milestones: All time estimates based on 1 FTE.

1. Milestone 1: Develop Interfaces

   1. Develop interface to the model layer (estimated at 2 weeks) to handle the basic commands outlined above. Should be compilable with only the model as a dependency.
   2. Unit test/documentation (estimated at 2 weeks) for model interface
   3. Develop interface to the view layer (estimated at 8 weeks). This includes resource handling for parceling out events to plugins as well as the create/delete/modify/add commands above. Should be compilable with the model and view as dependencies.
   4. Unit test/documentation (estimated at 8 weeks) for view interface
   5. Develop interface to the IO layer (estimated at 2 weeks) to handle adding import/export file types.
   6. Unit test/documentation (estimated at 2 weeks) for io interface
   7. Develop interface to the application later (estimated at 1 week) to handle adding menus and actions to the menu bar.
   8. Unit test/documentation (estimated at 1 week) for application interface

2. Milestone 2: Componentize Layers/Plugin

   1. Refactor manager to handle plugins via OSGi. This includes creating/managing the registry (estimated at 8 weeks)

3. Milestone 3: Add Aspects

   1. Provide aspects for factory methods (estimated at 8 weeks)
   2. Provide aspects for OSGi registration (estimated at 2 weeks)
   3. Provide aspects for resources (estimated at 4 weeks)
   4. Provide aspects for application and view layer (estimated at 4 weeks)

Issues

1. Do we want plugins to communicate with each other?

   • MikeSmoot 8/29/07

     • Yes, I don't think we actually have much choice on this. Plugins need to communicate with one another. Users have requested this over and over.
2. Do we want the core system available as separate components. If so how should we divide the system up (e.g. by layer, by domain or by both)
3. To what level do we wish to support web delivery – do we want cytoscape plugins to readily be available a tomcat webapp, do we want alternative (e.g. portlet) views?
4. An audit of all the major plugins needs to be undertaken to ensure that the required functionality is available.

   • MikeSmoot 8/29/07

     • I'm not sure we can do a comprehensive audit since we don't have the source for so many plugins. Life would be much, much, much easier if we had the source to all of the plugins out there. We definitely shouldn't rely on this as something to base our design decisions on.

     SarahKillcoyne 9/12/07

     • We certainly can't do all of them, but we could get a reasonable idea from going through what the various plugins we've put in csplugins and perhaps ask for input from plugin developers of the most popular plugins about what they do with their plugins.
5. Need to establish what aid we wish to give to plugin developers (e.g. are all we going to provide is a cytoscape_plugin.jar)?

   • MikeSmoot 8/29/07

     • If plugins are one of our biggests strengths, then we need to support developers. At the minimum we need developer documentation describing how to interoperate with whatever plugin mechanism we come up with.

Comments

MikeSmoot 8/29/07

• In general, I prefer thinking about the core of cytoscape as a group of indepependent modules that each expose a reasonable interface. Instead of trying to define a specific plugin interface to this core, I think it would be better to focus on modularizing the core such each module only exposes a limited interface. Plugins would then have access only to those interfaces. It accomplishes the same goal, just approached from a different perspective.

SarahKillcoyne 9/12/07

• I agree it would accomplish the same goal, but I think it would help to have a concrete idea of which perspective we develop the core code from. It seems that we can look at Cytoscape as a framework for plugins or as a set of independent modules. This would probably be a good discussion to be had around layering as well.