RFC_Custom_Graphics_VizMapper

Status

• New: Sample plugin using this feature is now available (Google Chart Plugin)

3/9/2010 - KeiichiroOno: start working on actual design and implementation 4/9/2010 - KeiichiroOno: start designing dynamic graphics function. 4/13/2010 - KeiichiroOno: Static image custom graphics feature implemented in branch code. 6/15/2010 - KeiichiroOno: Code had been merged to trunk. Will be available as 2.8.0 feature.

Proposal

We would like to provide some mechanism for allowing users to specify custom graphic images that can be applied to a network using the VizMapper. This includes new mapping from attributes to custom graphics.

Background

The goal of this RFC is to explore ideas for bringing custom graphics out of the programmer-only domain and allow normal users to specify custom graphics for nodes using normal Cytoscape tools like the VizMapper.

Use Cases

• List/Map attribute values mapped to node graphics - Currently, Cytoscape can map only one attribute value to a node. This means even if user wants to map multiple colors to node based on GO terms, it is impossible.

  • 

• A user wants to draw expression values as histogram on nodes (see the pictures above)
• A user wants a node shape that we don't supply by default.
• A user wants some non-standard graphic to be applied on or around a node (Like yEd. See the screenshot below).

  • 

• Draw complex graphics using multiple custom graphic objects.

  • 

Implementation Plan

• The general idea is to create a new NODE_CUSTOM_GRAPHIC visual property that would allow users to map specific images to nodes.

• This visual property would behave the same as other discrete properties like node shape or line type. This implies that most of the existing VizMapper GUI widgets could be used with minimal modification.

• We would provide a new menu option that would allow users to load image files and make them accessible to the VizMapper.

• We would need to consider I/O issues related to Session saving and loading.

  • One thought is to make a Custom Graphic a new type of CyAttribute. The advantage is that we could use the existing I/O mechanisms and we also allow passthrough mappings to point to the image. However, that might not buy us much as we'd need to add support in MANY places. It also doesn't make much sense to have graphics attached to specific nodes from the outset.

Basic Mechanism

CyCustomGraphics Interface

To implement this function, I defined a new interface called CyCustomGraphics

   1 public interface CyCustomGraphics <T> {
   2         /**
   3          * Display name is a simple description of this image object.
   4          *
   5          * May not be unique.
   6          *
   7          * @return display name as String.
   8          */
   9         public String getDisplayName();
  10         public void setDisplayName(final String displayName);
  11 
  12         public Collection<T> getCustomGraphics();
  13 
  14         public Image getImage();
  15         public Image resizeImage(int width, int height);
  16 
  17                 /**
  18          * Map of properties, i.e., details of this object.
  19          * Key is the property name, and Value is prop object.
  20          *
  21          * @return
  22          */
  23         public Map<String, CustomGraphicsProperty<?>> getProps();
  24 
  25         public void update();
  26 }
  27 

and then we can define a new Visual Property NODE_CUSTOM_GRAPHICS:

NODE_CUSTOM_GRAPHICS("Node Custom Graphics", "nodeCustomGraphics",
                                         "node.customGraphics", "defaultNodeCustomGraphics",
                                         CyCustomGraphics.class, new NodeCustomGraphicsProp(),
                                         new GraphicsParser(), true, true)

By implementing CyCustomGraphics interface, plugin writers can use this framework to create their own version of Custom Graphics. In short, this is just a new Visual Property. Users can use current VizMap UI to map attributes to graphics.

Custom Graphics Types

There are two types of custom graphics.

• Static image - usually a bitmap image.

  • 

• Dynamic image - appearance of image will be dynamically generated from attributes. For example, line chart generated from list attribute is a dynamic image.

  • 

Static image is just an mapping from an image object to custom graphics. However, a dynamic image is controlled by an associated node attribute. This requiers an additional layer to create CyCustomGraphics object on-the-fly (will be discussed later).

Static Custom Graphics Mapping

• Discrete / Continuous Static Image Mapping - This is straightforward. Just like other mapping, user can select images from pool of images (described below).
  • Discrete Mapping example1:

    • 

  • Degree 1 nodes are mapped to a leaf image:

    • 

• Simple use case of Custom Graphics is a mapping from nodes to static images:

  •  Attr value 1 = Image URL 1
     Attr value 2 = Image URL 2
     Attr value 3 = Image URL 1
     Attr value 4 = Image URL 3
     .
     .
     .

We can create discrete mapping from attribute value to URL image, but this type of mapping can be handled by an extended PassThroughMapping. Currently, Pass Through Mapping only supports String object to label. However, it can be extended to accept other objects including custom graphics. If a list of URL strings is mapped to NODE_CUSTOM_GRAPHICS, the Pass Through Mapping generates Image objects from the URL and wrap it with CyCustomGraphics.

• URL Pass Through Mapping Example - The attribute URL String is a simple string attribute. New Passthrough mapping parses the string and creates Custom Graphics object from the URL:

  • 

Static Image Custom Graphics Pool

All Custom Graphics generated from static (bitmap) images will be stored in a manager object called CustomGraphicsPool. This collection of images will be saved to session or .cytoscape/images directory.

Dynamic Custom Graphics and Custom Mapping

Static images can be handled in the existing UI/framework of VizMapper. However, for graphics generated dynamically from attributes (Dynamic Graphics), we need an additional layer. This problem can be handled by adding new kind of Visual Mapping. Currently, we have three types of mappings: continuous, discrete, and passthrough. All of these implements the following interface:

• Original Object Mapping

  •    1 public interface ObjectMapping<V> extends Cloneable {
       2 
       3         /**
       4          * Class of mapped object.  For example, if this is an Node Color mapping,
       5          * this value is Color.class.
       6          *
       7          * @return
       8          */
       9     public Class<V> getRangeClass();
      10 
      11     /**
      12      * Return the classes that the ObjectMapping can map from, eg. the contents
      13      * of the data of the controlling attribute.
      14      * <p>
      15      * For example, DiscreteMapping {@link DiscreteMapping} can only accept
      16      * String types in the mapped attribute data. Likewise, ContinuousMapping
      17      * {@link ContinuousMapping} can only accept numeric types in the mapped
      18      * attribute data since it must interpolate.
      19      * <p>
      20      * Return null if this mapping has no restrictions on the domain type.
      21      *
      22      * @return Array of accepted attribute data class types
      23      */
      24     public Class<?>[] getAcceptedDataClasses();
      25 
      26 
      27     /**
      28      * Set controlling attribute of this mapping.
      29      *
      30      * @param controllingAttrName - name of the attribute associated with this mapping.
      31      *
      32      */
      33     public void setControllingAttributeName(final String controllingAttrName);
      34 
      35 
      36     /**
      37      * Get the controlling attribute name
      38      */
      39     public String getControllingAttributeName();
      40 
      41     /**
      42      * Add a ChangeListener to the mapping. When the state underlying the
      43      * mapping changes, all ChangeListeners will be notified.
      44      *
      45      * This is used in the UI classes to ensure that the UI panes stay consistent
      46      * with the data held in the mappings.
      47      *
      48      * @param    l    ChangeListener to add
      49      */
      50     public void addChangeListener(ChangeListener l);
      51 
      52     /**
      53      * Remove a ChangeListener from the mapping. When the state underlying the
      54      * mapping changes, all ChangeListeners will be notified.
      55      *
      56      * This is used in the UI classes to ensure that the UI panes stay consistent
      57      * with the data held in the mappings.
      58      *
      59      * @param    l    ChangeListener to add
      60      */
      61     public void removeChangeListener(ChangeListener l);
      62 
      63     /**
      64      * Create a mapped visual representation from the given attribute value.
      65      *
      66      * @param attrBundle
      67      * @return
      68      */
      69     public V calculateRangeValue(final Map<String, Object> attrBundle);
      70 
      71     public JPanel getLegend(VisualPropertyType type);
      72 
      73     public Object clone();
      74 
      75     public void applyProperties(Properties props, String baseKey, ValueParser<V> parser);
      76 
      77     public Properties getProperties(String baseKey);
      78 }
      79 
    

If we need to add a new type of mapping, new editor component is necessary. To solve this problem, we can define a new mapping type called CustomMapping:

   1 public interface CustomMapping<V> {
   2         public Component getEditor();
   3 }
   4 

If the new mappings implement this interface AND ObjectMapping, then users can edit arbitrary many properties of the Custom Graphics objects.

Define How to Create Dynamic Custom Graphics (Custom Graphics Property)

To generate Custom Graphics dynamically, we need details about how to generate graphics from data. Suppose a user wants to create a histogram Custom Graphics from node attributes. In that case, Cytoscape needs the following information:

1. Which node attribute is associated with the length of the bar?
2. Appearance of the chart (width of the bars, bar color, etc.)
3. Size of the histogram graphics (px)

These relationships will be encoded as CustomGraphicsProperty:

•    1 public interface CustomGraphicsProperty<T> {
     2     public T getDefaultValue();
     3 
     4     public T getValue();
     5     public void setValue(Object value);
     6 }
     7 
  

Basically, this is a sort of sub Visual Properties only for NODE_CUSTOM_GRAPHICS.

Sample Dynamic Custom Graphics

• Gradient Bar Node

Custom Mapping Editor

Each custom mapping should have a GUI editor component.

Each editable field represents a CustomGraphicsProperty.

Custom Mapping Example

   1 public class SingleBarChartGeneratorMapping extends
   2                 AbstractMapping<CyCustomGraphics<?>> implements
   3                 CustomMapping {
   4 
   5         private final CustomGraphicsBuilder builder;
   6 
   7         public SingleBarChartGeneratorMapping(Class<CyCustomGraphics<?>> rangeClass,
   8                         final String controllingAttrName, final String targetPropertyName) {
   9                 super(rangeClass, controllingAttrName);
  10                 this.builder = new SimpleVectorBarBuilder(targetPropertyName);
  11                 // Create editor component here.
  12         }
  13 
  14 
  15         @Override
  16         public CyCustomGraphics<?> calculateRangeValue(Map<String, Object> attrBundle) {
  17                 if(attrBundle == null || attrBundle.get(controllingAttrName) == null)
  18                         return null;
  19                 final Object data = attrBundle.get(controllingAttrName);
  20                 return builder.getGraphics(data);
  21         }
  22  .
  23  .
  24  .
  25 
  26 }
  27 

   1 public class SimpleVectorBarBuilder implements CustomGraphicsBuilder {
   2 
   3         private final String targetPropName;
   4 
   5         public SimpleVectorBarBuilder(String targetPropName) {
   6                 this.targetPropName = targetPropName;
   7         }
   8 
   9         @Override
  10         public CyCustomGraphics<?> getGraphics(Object data) {
  11                 final CyCustomGraphics<?> graphics = new GradientRectangleCustomGraphics();
  12                 graphics.getProps().get(targetPropName).setValue(data);
  13                 graphics.update();
  14 
  15                 return graphics;
  16         }
  17 }
  18 

• TODO: should I use Annotation like Tunable in Cytoscape 3...?

Project Management

Project Dependencies

• This depends on the CustomGraphic API present in !DNodeView, something we might want to expose in GINY instead.

Issues

• What sort of image types will we support? PNG and JPG should be easy, but what about vector graphics like PDF or SVG?
• Would we want to provide facilities for scaling or other affine transforms for manipulating the images once they're loaded?

• We might want a separate visual property for setting custom graphic position. If not then we might need to extend current discrete VizMapper widgets to allow use to choose North/South/East/West positioning. Either way, this might be messy given the current API.

• What about background images?
  • Can we provide a minimal API that allows users to set background images?

  • How would this integrate with the VizMapper?

Comments

• will custom images be composible with other visual properties? For example, can you color them according to NodeFillColor?

• Can a dynamic custom graphics builder take more than one attribute as an input? For example, in building a chart, you may want to use multple attribute values. The alternative is to make one List attribute for this, but then you can't easily show the individual attribute values as easily under columns in the data panel or in a heatmap.
  • I think that supporting vector graphics like SVG would be extremely useful and would solve the scaling challenge.
  • I would not worry about background images -- I think they should be handled by our existing background layer and utilize a separate API. If you implement scaling using SVG/PDF, thought, it would be really nice to take advantage of that mechanism.