Diff for "AutomaticBugReporter"

TableOfContents([2])

Proposal

Automatic error detecting and reporting has become common practice in desktop and web-based applications. The bugs found in the field may be types that wouldn't ever be caught during testing. With an automatic bug report system, filing a bug report is just a click on a button in a dialog. Many more bug-reports from various kinds of users become possible, including non-technical users and those who can't be bothered to login to a bug tracker and file a detailed report.

Cytoscape should consider a similar automatic bug reporting system, which could potentially increase feedback from Cytoscape users and provide developers useful information needed to fix the bugs. This RFC describes how we might build an automatic bug reporter for Cytoscape.

Use Cases

1. GUI mode

Scenario: A user is using Cytoscape and suddenly Cytoscape runs into a problem. An exception is caught and the exception is classified as a bug. A dialog box pops up and asks user if he/she wants to report the error to Cytoscape. If user says “Yes”, information about the bug will be sent to Cytoscape bug tracker.

• attachment:CyAutoBugReportDialog.JPG attachment:CyAutoBugContents.JPG

2. Headless mode

(1) Interactive mode If Cytoscape is running in headless mode and the mode is “interactive”, then when an exception is caught and the exception is classified as a bug, a message on the console will tell the user what happened and ask the user if he/she wants to report the error to Cytoscape. If the user says “Yes”, information about the bug will be sent to Cytoscape bug tracker.

(2) Server mode Cytoscape is running in headless mode and the mode is “server”. Cytoscape will a property “AutomaticBugReporter_server”. Report the error only if the property value is true.

3. Do not report any bug

In the case that a user decides not to report any error, the user has the option to set the property value “automaticErrorReport” (true, by default) to false and Cytoscape will remember the decision.

Collecting data from Cytoscape user

When Cytoscape crashes or exception thrown, the following information from user should be collected and submitted directly to Cytoscape bug tracker, Mantis

• Cytoscape version
• OS version and other system info
• The class and line number in the code where the crash occurred
• The error message (full stack trace)
• User’s Description (Optional)
• User email address (Optional)
• Command history – everything the user did since startup – recorded by Cystocape logger
• Stack dump from OSGi that tells us what bundles are loaded, versions, etc

The category of bug report in Mantis may be infered from the specific exception type. We encourage users to give a description of what they did before Cytoscape ran into the problem. This will be very helpful for developers to reproduce the bug. User email address is optional. Many users may not give us their email address at all. However, in cases where the same bug is reported multiple times, and developer has questions about it, it could be useful to contact user if he/she did provide an email address

Format of bug report and identification of duplicated bugs

The subject (summary) of bug report should have the format something like this:

ExceptionType:ClassName:Method:LineNumber:OS:OSVersion This string is used to determine if a bug has already been reported.

If a bug has already been reported (with the same subject), the report should be appended to the existing one, rather than opened as a new one.

OS Set (WIN, LINUX, MAC, OTHER)

OS version XP/???

Product build Cytoscape 3.0

Description StackTrace

Additional Information User’s comment User’s e-mail contact, if any

Steps to reproduce Command history

Attached file If session file is attached, it will be added as an attached file.

Notes If duplicated bug, add a note only. OS OS_Version User’s e-mail User’s comment Time stamp

Automatic bug reporter could be part of CyLogger

Since Cytoscape 2.6.1, Cytoscape has its own Logger, CyLogger, which catches all error messages not already handled by the task framework and presents them to the users. CyLogger could be extended to include an automatic error reporter. CyLogger can catch different log messages, including warning, info, and debug. It seems reasonable that only Error or Fatal should be reported as bugs. Any exception thrown in Cytoscape should be caught and passed through the automatic reporting system where an appropriate ErrorCode would be assigned based on the nature of the error.

Implementation Plan

The implementation of the automatic bug reporter includes two parts, client side (Cytosape) and server side (Mantis). The client would be responsible for data collecting and submission only and the server for data processing. If a task can be done on either the client or the server side, then we'd prefer to put it on server side as it makes maintenance easier in the future.

Cytoscape (client side)

Two Java classes in cytoscape.logger package would be needed for automatic bug reporter.

1. ErrorCode.java Definition of various Error codes

2. SubmitErrorReport.java If GUI mode, generate dialogs and submit the data to Cytoscape bug tracker if user says “YES”. If headless mode, prompt for user input and submit the data to Cytoscape bug tracker if user says “YES”.

The automatic error reporter will be triggered by exceptions caught at running time. The following is a snippet of code to show how the automatic bug reporter is triggered.

try {
        //some code
}
catch exception (Exception e){
        logger.SubmitErrorReport(ErrorCode.XXXX, e);
}

Mantis (Server side)

1. Virtual person: Automatic bug reporter should have its own user name “Automatic bug reporter” in Mantis -- a virtual person, instead of a real developer.

2. New PHP script (bug_report_auto.php): the new Mantis PHP script will be triggered by “SubmitErrorReport” of Cylogger. This PHP script will (1) check the summary (subject line of the bug report) of the Bug report (2) if this is a new bug, create a new Bug in Mantis, if this is a duplicated bug, append the new report to existing one.

Web page about fixed-bugs

We may need a web page to list bugs-fixed (CyBugsFixed.php) at Cytoscape web site. On the page, fixed bugs for each Cytoscape versions would be listed, so that users could search for the bugs they care about and get advice/suggestions on possible work-arounds. Currently users can search for particular bugs in Cytoscape bug tracker, Mantis. But this is mainly for developers, not for ordinary users.

Integrating Mantis and Subversion

Mantis can be integrated with Subversion, so that if a developer fixes a bug and checks in the change into Subversion, the change can be immediately reflected in Mantis. There is a step-by-step instruction on how to setup the post-commit script in Subversion to make this happen, [http://alt-tag.com/blog/archives/2006/11/integrating-mantis-and-subversion/ here].

Related RFCs

Link to other related RFCs

Issues

List any issues, conflict, or dependencies raised by this proposal

Comments

MikeSmoot

• How would ErrorCodes get assigned? Would we need to manually assigned a code to every exception that gets thrown? Would it be easier to assign codes based on the Exception or the package that contains the Exception?

• This seems to dovetail with merging the error messages produced by CyLogger and those produced by the Task framework.

• I would expect the bug report to include the version of cytoscape and possibly a status dump from OSGi that tells us what bundles are loaded, versions, etc..
• It would be useful if the dialog provided an option for users to submit their session file along with the report.

• It would be really useful if we had a command provenance or history system that could tell us everything the user did since startup and we were able to include that in the report.

PietMolenaar

• I assume plugins can use the same mechanism, but how should this be implemented at the server side? Should that be something configurable to your own needs? Or direct bugs to specific parts of Mantis?

(PengLiangWang) Yes, Plugin can use the same mechanism to report bug to plugin's own bug tracker. In the case of Mantis, at the server side, we can use either the webservice API (MantisConnect) or Mantis core API to create a customized PHP script.

GaryBader

• Great idea! This may overload Mantis, since there could be thousands of bugs reported about the same issue - would these all be reported under the same bug ID (i.e. avoid duplication)? Additional information could be added as a note.
• There should be an option for automatic reporting of bugs without asking the user. This could be set as a property and would be useful for headless mode running as a server.
• There should be a way to turn this off, with a property, for when people are debugging code.
• Would this report plugin bugs too?
• Would people be able to change the Mantis server that bugs are reported to? That could be useful when you are debugging in a local user environment.

Peng-Liang Wang

• If a bug is determined “duplicated” of existing one based on its summary (title), new issue will not be created in Mantis, instead it will edit the existing one, add a note for new information.

• We can add a property, say “AutomaticBugReport_always = true/false”, to support the headless “server” mode.

• User has the option to turn it off by setting the property “AutomaticBugReport= true/false”

• Core plugin should report bugs to Cytoascape, other plugins should report to their own bug tracker if they wish.

• I think the URL to report Cytoscape bug should be something like this: http://cytoscape.org/automaticBugReport.php Any request to this URL is then redirected to the Mantis server. The reason is because we may change the location of Mantis server, or even replace Mantis with other bug tracker.

BrianTurner

• Bug reporting should really be linked to an overall exception handling strategy. In this [http://www.ibm.com/developerworks/library/j-ejbexcept.html article] there is an interesting idea about how to avoid logging exceptions multiple times and uniquely identifying an exception--the article's focus is on EJB, but I think some of the ideas still apply in particular:

public class LoggableEJBException extends EJBException {
    protected boolean isLogged;
    protected String uniqueID;
    public LoggableEJBException(Exception exc) {
        super(exc);
        isLogged = false;
        uniqueID = ExceptionIDGenerator.getExceptionID();
    }
        ..
        ..
}

try {
    OrderHome orderHome = EJBHomeFactory.getInstance().getOrderHome();
    Order order = orderHome.findByPrimaryKey(Integer id);
} catch (RemoteException re) {
    LoggableEJBException le =
       ExceptionLogUtil.createLoggableEJBException(re);
    String traceStr = StackTraceUtil.getStackTrace(le);
    Category.getInstance(getClass().getName()).error(le.getUniqueID() +
":" + traceStr);
    le.setLogged(true);
    throw le;
}