Proposal

In Cytoscape 3.2.1, app management is handled by two distinct components - those being the App Manager (located in app-impl), and Felix FileInstall (a component of the default Karaf distribution). The App Manager tracks what apps are installed and what state they are in - be that installed, disabled, uninstalled, or failed to load. App Manager also loads and starts simple (non-OSGi) apps. Bundle (OSGi) apps, on the other hand, are loaded and started by Felix FileInstall. This has proved to be problematic, as FileInstall allows few avenues for customizing the process and does not allow errors to be captured. Given this, we propose extending the existing App Manager to load and start bundle apps directly using the OSGi API.

Background

Felix FileInstall, like Cytoscape’s core components, is an OSGi bundle that starts when Cytoscape is executed. Upon startup, Felix FileInstall begins executing in a loop that repeats at an interval specified by the felix.fileinstall.poll property. At the beginning of the loop, it checks if the framework start level is greater than the value set in feilx.fileinstall.active.level - if not, the file scanning/processing steps are skipped until the next iteration. If the active level threshold has been reached, FileInstall will scan for any files that have been added, modified, or removed - by definition, the first scan will include all bundles currently in the installed directory. Based on the changes to this directory, it will load, update, or unload bundles as appropriate, catching all errors and logging them. Finally, if any bundle’s state has changed since the last scan and the framework start level is greater than that set in felix.fileinstall.start.level, FileInstall will then attempt to start all bundles that have not been manually stopped. This will repeat per the specified poll interval as long as Cytoscape is running.

Though it has generally worked for us in current versions of Cytoscape 3.x, using this framework imposes significant limitations on app management. Fundamentally, FileInstall provides virtually no public API, and does not allow any customization of the bundle load/start process beyond what can be set in its properties file. This presents a real problem for Cytoscape, as FileInstall’s default behavior has major flaws. For one, it doesn’t provide any way of capturing bundle errors on the App Manager level, as it consumes them without providing a listener interface. Additionally, by using FileInstall we have two separate threads simultaneously scanning the installed directory for updates - FileInstall and the App Manager. Finally, using FileInstall gives us an additional point of failure that we don’t control. In fact, the most recent update to FileInstall - introduced as part of Karaf 3 - uses a new file scanner that doesn’t properly handle some bundle updates or network drives. While those issues have a workaround (setting the felix.fileinstall.disableNio2 option to true), we are still left with a suboptimal solution that provides few avenues for error handling, multiple threads performing redundant file scans, and an extra point of failure.

Use Cases

Provide examples of how the products of this project will be used.

Implementation Plan

As an alternative to FileInstall, we propose loading and starting bundles using the OSGi framework directly. To monitor the installed directory, we would simply rely on the existing App Manager. The main change would be that the App Manager would invoke the bundle loading and startup directly - as it already does for simple apps - rather than relying on FileInstall to do so.  We would then have full control of the app installation process using the OSGi APIs, and would be able to capture and respond to errors in the App Manager as desired. Only one component (the App Manager) would be monitoring the installed bundles directory at any given time, and we would have one less point of failure. 

Currently, the App Manager scans all the Cytoscape app directories (installed, install-on-restart, uninstalled, and disabled), moves the apps in “install-on-restart” to “installed”, and parses the JARs in each directory into an App object. It then registers each App object with the App Manager and calls the appropriate method (install, uninstall, disable) based on the status. Calling these methods sets the status value inside the App object. For simple apps, the install method will also load and start the app. We propose that bundle apps should also be installed by this same method (which will use OSGi APIs for bundle apps), though in order to ensure bundle dependencies are fulfilled before starting them we will separate install into two separate methods, load and start. Furthermore, we will rename the App object's other methods to distinguish them from the actions in App Manager with the same name, and set the status values in the AppManager to decouple this from the method calls. Bundles will be started immediately after loading, and provide status updates (including any errors) in the CyUserLog (displayed in the task history window and status bar). If a bundle fails to load or start, we will also set its status in the App Manager to “Failed to Load” or “Failed to Start”.

After startup, the App Manager monitors the app directories for changes - as it does now. If an app is added to the installed directory, the App Manager will call the install method of the App object. For the uninstalled and disabled directories, it will call the uninstall or disable method. For simple apps, calling “install” will install the app - otherwise it will just set the status and rely on FileInstall to do the rest. We propose that for both simple and bundle apps, adding apps to the directories will load/start/unload them using the appropriate methods. As on startup, status updates (including errors) would be displayed in the task monitor. If a bundle fails to install or start, we will also set its status in the App Manager to “Failed to Install” or “Failed to Start”.

Apps can also be installed from the App Store. In this case, the install is invoked via a task which downloads the app to a temporary location, reads the list of dependencies and queues them for install, and upon completion of the download moves it to the installed directory - triggering the same process as if the user manually copied the file. Currently the dependencies are installed after the selected app - they will now install before the app to ensure they are resolved before the app attempts to start.

Project Management

Project Timeline

Provide a timeline for implementation. Insert a graphic if you can. Try this free online tool for making project timelines -> Help-u-Plan (create a new chart; modify; right-click to save gif; then attach to this page)

Tasks and Milestones

Outline the major milestones and tasks involved in implementation.

1. Milestone 1: …

   1. Task 1: ...
   2. Task 2: ...

2. Milestone 2: …

Project Dependencies

Outline and projects that depend on this project, link to relevant RFC's and note at what point dependent projects could be started.

Related RFCs

Link to other related RFCs

Issues

* If someone manually copies two apps A and B, and B depends on A, an error will occur if Cytoscape is open and B is written before A. This should be a rare case, as few apps depend on other apps and manually updating two such apps is even more unusual. This is not an issue for apps installed via the App Store as dependencies will be automatically fulfilled in the proper order. However, if this happens, the user can restart Cytoscape or manually restart the failed bundle.

* When using CyUserLog, error messages will only be displayed in the status bar (not a pop-up window), and will clear automatically after a few seconds. Furthermore, only a short "Failed to install/start" will be shown there - more details will only appear in the log file. Ultimately, it would be preferable to display a pop-up dialog for errors with more details when installing apps using the UI - however, this would require more refactoring to do. To help alleviate this, we do plan on changing the status bar icon to the error/warning sign when an error/warning occurs until the task history dialog is viewed.

* Detailed stack trace information from app load/start failures that is logged is only viewable by manually opening framework-cytoscape.log. It would be ideal to view these messages in the UI somewhere, though difficult to do without a performance penalty. One alternative to display this information would be to display it in a task exception dialog, or output it to the console.

Test Cases

Test each of the following. In all cases, make sure the behavior is as desired/expected. Be sure to test both simple and bundle apps - note that dependencies are only allowed for bundle/OSGi apps. We may need to build some dummy apps in order to test some of these cases.

* Start up Cytoscape with only “good” JARs installed. “Good” means any JAR that can reasonably be expected to load and start without an error.

* Start up Cytoscape with only “bad” JARs installed. “Bad” means any JAR that will fail to load or start. Include non-JARs with the JAR extension, JARs that are not valid Cytoscape apps, JARs that fail to load, bundle JARs that import unavailable packages, and JARs that crash in CyActivator/CyApp.

* Start up Cytoscape with a mix of good/bad JARs. Try disabling a few, and then re-enable some of them. Restart Cytoscape and try to re-enable the remaining disabled apps.

* Start up Cytoscape and copy some “good” JARs to the installed directory. Then, copy some “bad” JARs over. Uninstall all the bundles using the App Manager and restart to confirm they have been removed.

* Start up Cytoscape and open the App Store using App Manager. Install an app with dependencies that need to be fulfilled.

* Start up Cytoscape and open the App Store using a Web browser. Install an app with dependencies that need to be fulfilled.

* Open the framework-cytoscape.log file in a text file. Verify that all logging messages look correct.

Comments

• Add comment here…

How to Comment

Edit the page and add your comments under the provided header. By adding your ideas to the Wiki directly, we can more easily organize everyone's ideas, and keep clear records. Be sure to include today's date and your name for each comment. Try to keep your comments as concrete and constructive as possible. For example, if you find a part of the RFC makes no sense, please say so, but don't stop there. Take the extra step and propose alternatives.

MoinMoin Appliance - Powered by TurnKey Linux