Size: 6481
Comment:
|
Size: 13311
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 14: | Line 14: |
These days, there are several public biological database web wervices such as Biomart, NCBI, or KEGG. It is very useful for many users if Cytoscape have an access to those huge data resources through simple user interface. By using this function, users can access virtually all kinds of annotations for the nodes, including variety of ID's, sequences, descriptions, orthologs, pathway names which contains the nodes, etc. Also, this is a general solution for all types of node ID mapping problems Cytoscape currently have. In addition, users can access network database web services thirough the same mechanism. The goal of this project is the following: * Implement a mechanism to manage web service clients in the Cytoscape core. |
These days, there are many public biological database web wervices such as [http://www.biomart.org/ Biomart], [http://www.ncbi.nlm.nih.gov/entrez/query/static/esoap_help.html NCBI], or [http://www.genome.jp/kegg/soap/ KEGG]. It is very useful for many users if Cytoscape have an unified access to those huge data resources through simple user interface. By this function, users can access virtually all kinds of annotations for the nodes and edges, including variety of ID's, sequences, descriptions, orthologs, pathway names which contains the nodes, etc. Also, this is a general solution for all types of node ID mapping problems Cytoscape currently have. In addition, users can access network (interaction) database web services through the same mechanism. From plugin writers' point of view, simplified API for accessing web services enables them to use a lot of remote functions just like other Cytoscape API. This means plugin writers can use web services as building blocks to build their workflow. The goal of this project is the following: * Implement a mechanism to manage web service clients in the Cytoscape core (''Web Service Client Manager''). |
Line 18: | Line 23: |
* Design mechanism how web service clients communicate with the core (Event handling). * Implement an simple UI for importing annotations and networks from the web services. |
* Design mechanism how web service clients communicate with the core (custom event handling). * Implement an simple GUI for importing annotations and networks from the web services. |
Line 31: | Line 36: |
Each web service is a building block to make higher-level functions. This means plugin writers should be able to use registered web services as if they are local Java APIs. To achive this, we should be careful to separate UI from the core web service clients. | Each web service is a building block to make higher-level functions. This means plugin writers should be able to use registered web services as if they are local Java APIs. To achive this goal, we should be careful to separate UI from the web service clients. |
Line 34: | Line 39: |
This component consists of 5 parts: | This project consists of five parts: |
Line 36: | Line 41: |
- Maintaining list of web service clients. - Each web service client implements an interface !WebServiceClient 1. Attribute mapping module - Mapping fetched annotations onto !CyAttributes. 1. Event Handlers - Implementing custom event and its listeners only for !WebServiceClients 1. Web Service Clients - these are jar files which contains properties and classes created from WSDL. They will be loaded as normal cytoscape plugins. |
* Maintaining list of web service clients. * GUI to view/edit properties of the clients. 1. Attribute mapping module * Mapping fetched annotations onto !CyAttributes. 1. Event Handlers * Implementing custom event and its listener only for !WebServiceClients * Higher-level functions, such as '''''importNetwork(Object query)''''' or '''''importAttribute(Object query)''''' will be called through this event handler. 1. Sample Web Service Clients * Implement several web service clients for popular biological databases. * Each client is a jar file which contains properties and classes created from WSDL. They will be implemented as a regular Cytoscape plugins and loaded through Plugin Manager. * If WSDL is not available (like Biomart), we need to implement a simple stub to access the service. |
Line 42: | Line 53: |
* GUI to use web services will be implemented as separate plugins. Simple GUI for importing attributes and networks will be provided as a part of core. | |
Line 47: | Line 59: |
* Browsing UDDI Registory | |
Line 49: | Line 62: |
* Pathway Commons / cPath integration to the new Network Import UI | * Pathway Commons / cPath integration to the new simplifed Network Import GUI |
Line 52: | Line 65: |
* Connection / integration to [http://taverna.sourceforge.net/ Taverna] or other workflow engins (like [http://kepler-project.org/ Kepler]). * More clients for other functions (analysis). * Use web services in batch (headless/daemon) mode analysis (3.0 or later). |
|
Line 55: | Line 70: |
=== Biological Web Services === | |
Line 58: | Line 73: |
==== Exmaple Web Services which will be implemented as a core plugins ==== * Biomart * NCBI * IntAct * UniProt * KEGG |
* [http://www.biomart.org/martservice.html Biomart] * [http://www.ncbi.nlm.nih.gov/entrez/query/static/esoap_help.html NCBI Entrez E-Utilities SOAP Service] * [http://www.ebi.ac.uk/~intact/devsite/modules/ws.html IntAct Web Services] * [http://www.ebi.ac.uk/uniprot/remotingAPI/ UniProtJAPI] * [http://www.genome.jp/kegg/soap/ KEGG API] * [http://www.reactome.org/cgi-bin/mart Reactome Mart Service] (Implemented by Biomart backend) === Technologies Around Web Service === * [http://ws.apache.org/axis/ Apache Axis] * [https://metro.dev.java.net/ GlassFish Metro Project] * [https://jax-ws.dev.java.net/ JAX-WS] (part of Metro project) |
Line 70: | Line 90: |
Web Service Client Manager is a mechanism to manage list of clients loaded in Cytoscape. Usually, a Web Service Client (WSC) is created from a WSDL and packed as a jar file. The WSC jar file '''must''' contain classes implementing the following interfaces: | '''Web Service Client Manager''' is a mechanism to manage list of clients loaded in Cytoscape. Usually, a Web Service Client (WSC) is created from a WSDL and packed as a jar file. The WSC jar file '''must''' contain classes implementing the following interfaces: |
Line 78: | Line 98: |
WebServiceClientManager.getClient(client_name); }}} |
WebService Client client = WebServiceClientManager.getClient(client_name); }}} In general, most of the methods required for clients are common to all. To avoid duplicate code, each web service client extends the superclass called ''WebServiceClientImpl''. Then, the actual client code will be similar to the following (this sample is for [http://www.ebi.ac.uk/intact/site/index.jsf IntAct] network database): {{{#!java public class IntactClient extends WebServiceClientImpl { // Define name and display name for this client private static final String DISPLAY_NAME = "IntAct PPI Database Client"; private static final String SERVICE_NAME = "intact"; // Client should be a singlton. private static WebServiceClient client; static { client = new IntactClient(); } // Private constructor private IntactClient() { super(SERVICE_NAME, DISPLAY_NAME, ClientType.NETWORK); stub = new BinarySearchServiceClient(); } // This will be called from IntactClientPlugin. public static WebServiceClient getClient() { return client; } // Tentative method to catch the events from the core. public void webServiceEvent(CyWebServiceEvent e) { if(e.getEventType.equals(IMPORT_NETWORK)) { importNetwork(); } } private void importNetwork(Object query) { // Actual code to import network data through the stub will be here... } } }}} And the class to register this as a plugin will be the following: {{{#!java public class IntactClientPlugin extends CytoscapePlugin { /** * Creates a new IntActClientPlugin object. */ public IntactClientPlugin() { // Register this client to the manager. WebServiceClientManager.registerClient(IntactClient.getClient()); } } }}} Plugin writers have two access method for these services: * Through the '''''Client Stub''''' * By calling '''''getStub()''''' method, users can access all methods available from the service. Object type of the stub depends on the web service. For example, NCBI E-Utilities client returns '''''EUtilsServiceSoap''''' as the stub. If the client is registered properly, plugins can use it in the following way: {{{#!java // Get client from the manager. WebServiceClient ncbiClient = WebServiceClientManager.getClient("ncbi"); // Then get the stub from the client. EUtilsServiceSoap ncbiStub = (EUtilsServiceSoap)ncbiClient.getStub(); // Use the stub EInfoResult res = ncbiStub.run_eInfo(new EInfoRequest()); }}} * Through CyWebServiceEvent * All clients implement !'''''CyWebServiceEventListener''''' and have mechanism to invoke method like ''importNetwork'' or ''importAttribute''. Plugin writers can access these methods by firing events. === Custom Event Model === (Under construction) Instead of using standard PropertyChange events, core and clients sending custom events. ==== CyWebServiceEvent ==== This event should contain the following information: * Type of event (IMPORT_NETWORK, IMPORT_ATTRIBUTE, EXECUTE_ANALYSIS, etc.) * Arguments (query command) ==== CyWebServiceEventListener ==== |
Line 83: | Line 186: |
Based on the architecture above, we need to design and implement several web service clients for popular biological databases. |
|
Line 84: | Line 189: |
Web Service Clients use the following components in the core: * Client Property and Tunables * Network Import Panel * Attribute Import Panel |
Web Service Clients (WSC) use the following components in the core: 1. Client Property and Tunables * WSC should have a '''''WSCProperty''''' object which includes basic information of the client and property values which controls the behaviour of WSC. * Basic information * End Point (URI of the service) * Description * Example property values: * ''Search Mode'' - keyword or ID list. * ''Maximum Number of Result'' - Limit the number of search result (interactions / gene IDs) returned by the service. * These values will be managed through '''''Tunable''''' class. Currently it is used only by the Layout Manager, but will be generalized to edit all property values for WSC. * All of these properties are accessible from '''''WebServiceClientManagerDialog'''''. attachment:wscm_dialog.png Left panel shows available (loaded) WSC and its category in the core. Right panel displays editable properties generated by ''Tunable''. 1. Network Import Panel attachment:network_import_panel.png 1. Attribute Import Panel attachment:attribute_import_panel.png In many cases, attribute import client needs the following information to map attributes onto Cytoscape's data structure (CyAttributes): 1. Data Source - source database name 1. Attribute name used for mapping 1. Data type of the attribute - Official Symbol, EntrezGene ID, Ensembl Ac. Number, etc. 1. List of available attributes in the selected datasource. The GUI above will be in the core as the utility for attribute import clients. It takes icon and labels as its parameters. Checkbox list should be created by the client. (Example above is for [http://beta.uniprot.org/ UniProt KB]) |
Line 96: | Line 226: |
Biomart client mainly used as ID mapping. | Biomart client mainly used as ID mapping. For simple UI, ''Filter'' is always created from list of ID / Attribute. |
RFC Name : BioWebServiceConnectivity |
Editor(s): KeiichiroOno |
Status: Open for Comment |
Proposal
(Still under construction!)
These days, there are many public biological database web wervices such as [http://www.biomart.org/ Biomart], [http://www.ncbi.nlm.nih.gov/entrez/query/static/esoap_help.html NCBI], or [http://www.genome.jp/kegg/soap/ KEGG]. It is very useful for many users if Cytoscape have an unified access to those huge data resources through simple user interface. By this function, users can access virtually all kinds of annotations for the nodes and edges, including variety of ID's, sequences, descriptions, orthologs, pathway names which contains the nodes, etc. Also, this is a general solution for all types of node ID mapping problems Cytoscape currently have. In addition, users can access network (interaction) database web services through the same mechanism.
From plugin writers' point of view, simplified API for accessing web services enables them to use a lot of remote functions just like other Cytoscape API. This means plugin writers can use web services as building blocks to build their workflow.
The goal of this project is the following:
Implement a mechanism to manage web service clients in the Cytoscape core (Web Service Client Manager).
- Design an interface which wraps each web service client.
- Design mechanism how web service clients communicate with the core (custom event handling).
- Implement an simple GUI for importing annotations and networks from the web services.
Biological Questions / Use Cases
User loaded network with EntrezGene ID as node identifier into Cytoscape. He/she needs to import annotation files with Gene Symbol as the key.
I need to check the orthologs for the genes in my network.
I want to expand existing network with known interaction data in [http://www.ebi.ac.uk/intact/site/index.jsf IntAct]
- Mapping known pathways onto network data in Cytoscape.
- Executing an analysis program running on remote server which is implemented as a web service.
General Notes
Each web service is a building block to make higher-level functions. This means plugin writers should be able to use registered web services as if they are local Java APIs. To achive this goal, we should be careful to separate UI from the web service clients.
Requirements
This project consists of five parts:
- Web Service Client Manager in the core
- Maintaining list of web service clients.
- GUI to view/edit properties of the clients.
- Attribute mapping module
Mapping fetched annotations onto CyAttributes.
- Event Handlers
Implementing custom event and its listener only for WebServiceClients
Higher-level functions, such as importNetwork(Object query) or importAttribute(Object query) will be called through this event handler.
- Sample Web Service Clients
- Implement several web service clients for popular biological databases.
- Each client is a jar file which contains properties and classes created from WSDL. They will be implemented as a regular Cytoscape plugins and loaded through Plugin Manager.
- If WSDL is not available (like Biomart), we need to implement a simple stub to access the service.
- GUI for each web services
- GUI to use web services will be implemented as separate plugins. Simple GUI for importing attributes and networks will be provided as a part of core.
Deferred Items
Open Issues
- Taxonomy of the services
- Browsing UDDI Registory
Backward Compatibility
- Pathway Commons / cPath integration to the new simplifed Network Import GUI
Expected growth and plan for growth
Connection / integration to [http://taverna.sourceforge.net/ Taverna] or other workflow engins (like [http://kepler-project.org/ Kepler]).
- More clients for other functions (analysis).
- Use web services in batch (headless/daemon) mode analysis (3.0 or later).
References
Biological Web Services
[http://taverna.sourceforge.net/index.php?doc=services.html Biological Web Services] by myGrid/Taverna Project
[http://www.biomart.org/martservice.html Biomart]
[http://www.ncbi.nlm.nih.gov/entrez/query/static/esoap_help.html NCBI Entrez E-Utilities SOAP Service]
[http://www.ebi.ac.uk/~intact/devsite/modules/ws.html IntAct Web Services]
[http://www.ebi.ac.uk/uniprot/remotingAPI/ UniProtJAPI]
[http://www.genome.jp/kegg/soap/ KEGG API]
[http://www.reactome.org/cgi-bin/mart Reactome Mart Service] (Implemented by Biomart backend)
Technologies Around Web Service
[http://ws.apache.org/axis/ Apache Axis]
[https://metro.dev.java.net/ GlassFish Metro Project]
[https://jax-ws.dev.java.net/ JAX-WS] (part of Metro project)
Implementation Plan
Web Service Client Manager
- attachment:wsc_design.png
Web Service Client Manager is a mechanism to manage list of clients loaded in Cytoscape. Usually, a Web Service Client (WSC) is created from a WSDL and packed as a jar file. The WSC jar file must contain classes implementing the following interfaces:
CytoscapePlugin
WebServiceClient
This means each of WSC is a Cytoscape plugin. Therefore, all WSC will be loaded through Plugin Manager which is already in the core. Once WSC is loaded through Plugin Manager, WSC registers itself to Web Service Client Manager. Plugin writers can access all web services by using the following method:
WebService Client client = WebServiceClientManager.getClient(client_name);
In general, most of the methods required for clients are common to all. To avoid duplicate code, each web service client extends the superclass called WebServiceClientImpl. Then, the actual client code will be similar to the following (this sample is for [http://www.ebi.ac.uk/intact/site/index.jsf IntAct] network database):
1 public class IntactClient extends WebServiceClientImpl {
2
3 // Define name and display name for this client
4 private static final String DISPLAY_NAME = "IntAct PPI Database Client";
5 private static final String SERVICE_NAME = "intact";
6
7 // Client should be a singlton.
8 private static WebServiceClient client;
9
10 static {
11 client = new IntactClient();
12 }
13
14 // Private constructor
15 private IntactClient() {
16 super(SERVICE_NAME, DISPLAY_NAME, ClientType.NETWORK);
17 stub = new BinarySearchServiceClient();
18 }
19
20 // This will be called from IntactClientPlugin.
21 public static WebServiceClient getClient() {
22 return client;
23 }
24
25 // Tentative method to catch the events from the core.
26 public void webServiceEvent(CyWebServiceEvent e) {
27 if(e.getEventType.equals(IMPORT_NETWORK)) {
28 importNetwork();
29 }
30 }
31
32 private void importNetwork(Object query) {
33 // Actual code to import network data through the stub will be here...
34 }
35
36 }
37
And the class to register this as a plugin will be the following:
1 public class IntactClientPlugin extends CytoscapePlugin {
2 /**
3 * Creates a new IntActClientPlugin object.
4 */
5 public IntactClientPlugin() {
6 // Register this client to the manager.
7 WebServiceClientManager.registerClient(IntactClient.getClient());
8 }
9 }
10
Plugin writers have two access method for these services:
Through the Client Stub
By calling getStub() method, users can access all methods available from the service. Object type of the stub depends on the web service. For example, NCBI E-Utilities client returns EUtilsServiceSoap as the stub. If the client is registered properly, plugins can use it in the following way:
1 // Get client from the manager.
2 WebServiceClient ncbiClient = WebServiceClientManager.getClient("ncbi");
3 // Then get the stub from the client.
4 EUtilsServiceSoap ncbiStub = (EUtilsServiceSoap)ncbiClient.getStub();
5
6 // Use the stub
7 EInfoResult res = ncbiStub.run_eInfo(new EInfoRequest());
8
Through CyWebServiceEvent
All clients implement !CyWebServiceEventListener and have mechanism to invoke method like importNetwork or importAttribute. Plugin writers can access these methods by firing events.
Custom Event Model
(Under construction)
Instead of using standard PropertyChange events, core and clients sending custom events.
CyWebServiceEvent
This event should contain the following information:
- Type of event (IMPORT_NETWORK, IMPORT_ATTRIBUTE, EXECUTE_ANALYSIS, etc.)
- Arguments (query command)
CyWebServiceEventListener
Individual Client Design
Based on the architecture above, we need to design and implement several web service clients for popular biological databases.
Shared Components
Web Service Clients (WSC) use the following components in the core:
- Client Property and Tunables
WSC should have a WSCProperty object which includes basic information of the client and property values which controls the behaviour of WSC.
- Basic information
- End Point (URI of the service)
- Description
- Example property values:
Search Mode - keyword or ID list.
Maximum Number of Result - Limit the number of search result (interactions / gene IDs) returned by the service.
- Basic information
These values will be managed through Tunable class. Currently it is used only by the Layout Manager, but will be generalized to edit all property values for WSC.
All of these properties are accessible from WebServiceClientManagerDialog.
- attachment:wscm_dialog.png
Left panel shows available (loaded) WSC and its category in the core. Right panel displays editable properties generated by Tunable.
- attachment:wscm_dialog.png
- Network Import Panel
- attachment:network_import_panel.png
- Attribute Import Panel
- attachment:attribute_import_panel.png
In many cases, attribute import client needs the following information to map attributes onto Cytoscape's data structure (CyAttributes):
- Data Source - source database name
- Attribute name used for mapping
Data type of the attribute - Official Symbol, EntrezGene ID, Ensembl Ac. Number, etc.
- List of available attributes in the selected datasource.
The GUI above will be in the core as the utility for attribute import clients. It takes icon and labels as its parameters. Checkbox list should be created by the client. (Example above is for [http://beta.uniprot.org/ UniProt KB])
ID Mapping and Attribute Import
Biomart
attachment:biomartidmapping1.png
Biomart client mainly used as ID mapping. For simple UI, Filter is always created from list of ID / Attribute.
UniProt
Recently, UniProt released UniProt Java Remoting API (UniProtJAPI). We will wrap this API with the WebServiceClient interface in Cytoscape.
KEGG
Network Import
IntAct
NCBI
- attachment:ncbi_network1.png
NCBI Entrez database has a unified web service called [http://www.ncbi.nlm.nih.gov/entrez/query/static/esoap_java_help.html NCBI E-Utilities]. The [http://www.ncbi.nlm.nih.gov/sites/entrez?db=gene Entrez Gene] database will be the main resource for Cytoscape. This DB can be accessed through the E-Utilities web service. Users can use both keywords and list of Entrez Gene ID as the search term. Actual network data will be extracted form Interaction section of the returned data. Since we can extract interactions of one gene at a time, we need use Java SE 5's new Concurrency Framework to run multiple threads for fetching data efficiently.
Comments
How to Comment
Edit the page and add your comments under the provided header. By adding your ideas to the Wiki directly, we can more easily organize everyone's ideas, and keep clear records. Be sure to include today's date and your name for each comment. Try to keep your comments as concrete and constructive as possible. For example, if you find a part of the RFC makes no sense, please say so, but don't stop there. Take the extra step and propose alternatives.
SarahKillcoyne August 16, 2007
This RFC would seem to be very related to the [:DataIntegration: Data Integration] one. Might be useful to discuss this as part of a larger web services integration if we decide a larger, more general solution would be useful.