RFC_Custom_Graphics_VizMapper

Status

3/9/2010 - KeiichiroOno: start working on actual design and implementation

4/9/2010 - KeiichiroOno: start designing dynamic graphics function.

4/13/2010 - KeiichiroOno: Static image custom graphics feature implemented in branch code.

Proposal

We would like to provide some mechanism for allowing users to specify custom graphic images that can be applied to a network using the VizMapper. This includes new mapping from attributes to custom graphics.

Background

The goal of this RFC is to explore ideas for bringing custom graphics out of the programmer-only domain and allow normal users to specify custom graphics for nodes using normal Cytoscape tools like the VizMapper.

Use Cases

• List/Map attribute values mapped to node graphics - Currently, Cytoscape can map only one attribute value to a node. This means even if user wants to map multiple colors to node based on GO terms, it is impossible.

  • 

    

• A user wants to draw expression values as histogram on nodes (see the pictures above)
• A user wants a node shape that we don't supply by default.
• A user wants some non-standard graphic to be applied on or around a node (Like yEd. See the screenshot below).

  • 

• Draw complex graphics using multiple custom graphic objects.

  • 

Implementation Plan

• The general idea is to create a new NODE_CUSTOM_GRAPHIC visual property that would allow users to map specific images to nodes.

• This visual property would behave the same as other discrete properties like node shape or line type. This implies that most of the existing VizMapper GUI widgets could be used with minimal modification.

• We would provide a new menu option that would allow users to load image files and make them accessible to the VizMapper.

• We would need to consider I/O issues related to Session saving and loading.

  • One thought is to make a Custom Graphic a new type of CyAttribute. The advantage is that we could use the existing I/O mechanisms and we also allow passthrough mappings to point to the image. However, that might not buy us much as we'd need to add support in MANY places. It also doesn't make much sense to have graphics attached to specific nodes from the outset.

Specific Ideas

CyCustomGraphics Interface

To implement this function, I defined a new interface called CyCustomGraphics

   1 public interface CyCustomGraphics <T> {
   2         /**
   3          * Display name is a simple description of this image object.
   4          * 
   5          * May not be unique.
   6          * 
   7          * @return display name as String.
   8          */
   9         public String getDisplayName();
  10         public void setDisplayName(final String displayName);
  11         
  12         public Collection<T> getCustomGraphics();
  13         
  14         public Image getImage();
  15         public Image resizeImage(int width, int height);
  16         
  17         /**
  18          * List of properties, i.e., details of this object.
  19          * 
  20          * @return
  21          */
  22         public Collection<CustomGraphicsProperty<?>> getProps();
  23 }
  24 

and then we can define a new Visual Property NODE_CUSTOM_GRAPHICS:

NODE_CUSTOM_GRAPHICS("Node Custom Graphics", "nodeCustomGraphics", 
                                         "node.customGraphics", "defaultNodeCustomGraphics", 
                                         CyCustomGraphics.class, new NodeCustomGraphicsProp(),
                                         new GraphicsParser(), true, true)

By implementing CyCustomGraphics interface, plugin writers can use this framework to create their own version of Custom Graphics.

Custom Graphics Types

There are two types of custom graphics.

• Static image - usually a bitmap image.

  • 

• Dynamic image - appearance of image will be dynamically generated from attributes. For example, line chart generated from list attribute is a dynamic image.

  • 

Static image is just an mapping from an image object to custom graphics. However, a dynamic image is controlled by an associated node attribute. This requiers an additional layer to create CyCustomGraphics object on-the-fly (will be discussed later).

Mapping Problem

• Discrete / Continuous Static Image Mapping - This is straightforward. Just like other mapping, user can select images from pool of images (described below).

  • 

Simple use case of Custom Graphics is a mapping from nodes to static images.

 Attr value 1 = Image URL 1
 Attr value 2 = Image URL 2
 Attr value 3 = Image URL 1
 Attr value 4 = Image URL 3
 .
 .
 .

This type of mapping will be done by an extended PassThroughMapping. Currently, Pass Through Mapping only supports String object to label. However, it can be extended to accept other objects including custom graphics. If a list of URL strings is mapped to NODE_CUSTOM_GRAPHICS, the Pass Through Mapping generates Image objects from the URL and wrap it with CyCustomGraphics.

• URL Pass Through Example:

  • 

Static Image Custom Graphics Pool

All Custom Graphics generated from static (bitmap) images will be stored in a manager object called CustomGraphicsPool. This collection of images will be saved to session or .cytoscape/images directory.

Generate Dynamic Custom Graphics using Custom Mapping

Static images can be handled in the existing UI/framework in VizMapper. However, for graphics generated dynamically from attributes (Dynamic Graphics), we need an additional layer. This problem can be handled by adding new kind of Visual Mapping. Currently, we have three types of mappings: continuous, discrete, and passthrough. All of these implements the following interface:

• Original Object Mapping

   1 public interface ObjectMapping<V> extends Cloneable {
   2 
   3         /**
   4          * Class of mapped object.  For example, if this is an Node Color mapping,
   5          * this value is Color.class.
   6          * 
   7          * @return
   8          */
   9     public Class<V> getRangeClass();
  10 
  11     /**
  12      * Return the classes that the ObjectMapping can map from, eg. the contents
  13      * of the data of the controlling attribute.
  14      * <p>
  15      * For example, DiscreteMapping {@link DiscreteMapping} can only accept
  16      * String types in the mapped attribute data. Likewise, ContinuousMapping
  17      * {@link ContinuousMapping} can only accept numeric types in the mapped
  18      * attribute data since it must interpolate.
  19      * <p>
  20      * Return null if this mapping has no restrictions on the domain type.
  21      *
  22      * @return Array of accepted attribute data class types
  23      */
  24     public Class<?>[] getAcceptedDataClasses();
  25 
  26     
  27     /**
  28      * Set controlling attribute of this mapping.
  29      * 
  30      * @param controllingAttrName - name of the attribute associated with this mapping.
  31      * 
  32      */
  33     public void setControllingAttributeName(final String controllingAttrName);
  34     
  35     
  36     /**
  37      * Get the controlling attribute name
  38      */
  39     public String getControllingAttributeName();
  40 
  41     /**
  42      * Add a ChangeListener to the mapping. When the state underlying the
  43      * mapping changes, all ChangeListeners will be notified.
  44      *
  45      * This is used in the UI classes to ensure that the UI panes stay consistent
  46      * with the data held in the mappings.
  47      *
  48      * @param    l    ChangeListener to add
  49      */
  50     public void addChangeListener(ChangeListener l);
  51 
  52     /**
  53      * Remove a ChangeListener from the mapping. When the state underlying the
  54      * mapping changes, all ChangeListeners will be notified.
  55      *
  56      * This is used in the UI classes to ensure that the UI panes stay consistent
  57      * with the data held in the mappings.
  58      *
  59      * @param    l    ChangeListener to add
  60      */
  61     public void removeChangeListener(ChangeListener l);
  62 
  63     /**
  64      * Create a mapped visual representation from the given attribute value.
  65      * 
  66      * @param attrBundle
  67      * @return
  68      */
  69     public V calculateRangeValue(final Map<String, Object> attrBundle);
  70 
  71     public JPanel getLegend(VisualPropertyType type);
  72 
  73     public Object clone();
  74 
  75     public void applyProperties(Properties props, String baseKey, ValueParser<V> parser);
  76 
  77     public Properties getProperties(String baseKey);
  78 }
  79 

• De

   1 public interface CustomGraphicsProperty<T> {
   2         
   3         public String getDisplayName();
   4         public T getDefaultValue();
   5         
   6         public T getValue();
   7         public void setValue(T value);
   8 }
   9 

To generate variety of custom graphics, we need a mechanism to map multiple attribute values to a graphics. For example, if user wants to generate histograms for nodes, it requires List attributes one for x-axis, and the other one for y-axis.

List Attr 1 = {t1, t2, ...., tn}
List Attr 2 = {0.21, 0.53, ..., m}

To solve this problem, we can define a new mapping type called CustomMapping:

   1 public interface CustomMapping<V> {
   2         
   3         public Component getEditor();
   4 }
   5 

Custom Mapping Editor

Each custom mapping should have a GUI editor component.

Project Management

Project Dependencies

• This depends on the CustomGraphic API present in !DNodeView, something we might want to expose in GINY instead.

Issues

• What sort of image types will we support? PNG and JPG should be easy, but what about vector graphics like PDF or SVG?
• Would we want to provide facilities for scaling or other affine transforms for manipulating the images once they're loaded?

• We might want a separate visual property for setting custom graphic position. If not then we might need to extend current discrete VizMapper widgets to allow use to choose North/South/East/West positioning. Either way, this might be messy given the current API.

• What about background images?
  • Can we provide a minimal API that allows users to set background images?

  • How would this integrate with the VizMapper?

Comments