Diff for "AutomaticBugReporter"

TableOfContents([2])

Proposal

Automatic error detecting and reporting has become common practice in desktop and web-based applications. The bugs found in the field may be types that wouldn't ever be caught during testing. With an automatic bug report system, filing a bug report is just a click on a button in a dialog. Many more bug-reports from various kinds of users become possible, including non-technical users and those who can't be bothered to login to a bug tracker and file a detailed report.

Cytoscape should consider a similar automatic bug reporting system, which could potentially increase feedback from Cytoscape users and provide developers useful information needed to fix the bugs. This RFC describes how we might build an automatic bug reporter for Cytoscape.

Use Cases

1. GUI mode

Scenario: A user is using Cytoscape and suddenly Cytoscape runs into a problem. An exception is caught and the exception is classified as a bug. A dialog box pops up and asks user if he/she wants to report the error to Cytoscape. If user says “Yes”, information about the bug will be sent to Cytoscape bug tracker.

• attachment:CyAutoBugReportDialog.JPG

  attachment:CyAutoBugContents.JPG

2. Headless mode

If Cytoscape is running in headless mode and the mode is “interactive”, not “server”, then when an exception is caught and the exception is classified as a bug, a message on the console will tell the user what happened and ask the user if he/she wants to report the error to Cytoscape. If the user says “Yes”, information about the bug will be sent to Cytoscape bug tracker.

3. Do not report any bug

In the case that a user decides not to report any error, the user has the option to set the property value “automaticErrorReport” (true, by default) to false and Cytoscape will remember the decision.

Collecting data from Cytoscape user

When Cytoscape crashes or exception thrown, the following information from user should be collected and submitted directly to Cytoscape bug tracker, Mantis

• Cytoscape version
• OS version and other system info

• ErrorCode (or Error Number) – depend on the bug category, API/CORE/GUI/PLUGIN/VISUALSTYLE/OTHER

• The class and line number in the code where the crash occurred
• The error message (full stack trace)
• User’s Description
• User email address (Optional)

ErrorCode would be used for bug classification purposes and to make bug management in Mantis easier. We encourage users to give a description of what they did before Cytoscape ran into the problem. This will be very helpful for developers to reproduce the bug. User email address is optional. Many users may not give us their email address at all. However, in cases where the same bug is reported multiple times, and developer has questions about it, it could be useful to contact user if he/she did provide an email address

Format of bug report and identification of duplicated bugs

The subject (summary) of bug report should have the format something like this:

• ErrorCode:ClassName:Method:LineNumber:OS:OSVersion

This string is used to determine if a bug has already been reported.

If a bug has already been reported (with the same subject), the report should be appended to the existing one, rather than opened as a new one.

Automatic bug reporter could be part of CyLogger

Since Cytoscape 2.6.1, Cytoscape has its own Logger, CyLogger, which catches all error messages not already handled by the task framework and presents them to the users. CyLogger could be extended to include an automatic error reporter. CyLogger can catch different log messages, including warning, info, and debug. It seems reasonable that only Error or Fatal should be reported as bugs. Any exception thrown in Cytoscape should be caught and passed through the automatic reporting system where an appropriate ErrorCode would be assigned based on the nature of the error.

Implementation Plan

The implementation of the automatic bug reporter includes two parts, client side (Cytosape) and server side (Mantis). The client would be responsible for data collecting and submission only and the server for data processing. If a task can be done on either the client or the server side, then we'd prefer to put it on server side as it makes maintenance easier in the future.

Cytoscape (client side)

Two Java classes in cytoscape.logger package would be needed for automatic bug reporter.

1. ErrorCode.java Definition of various Error codes

2. SubmitErrorReport.java If GUI mode, generate dialogs and submit the data to Cytoscape bug tracker if user says “YES”. If headless mode, prompt for user input and submit the data to Cytoscape bug tracker if user says “YES”.

The automatic error reporter will be triggered by exceptions caught at running time. The following is a snippet of code to show how the automatic bug reporter is triggered.

try {
        //some code
}
catch exception (Exception e){
        logger.SubmitErrorReport(ErrorCode.XXXX, e);
}

Mantis (Server side)

1. Virtual person: Automatic bug reporter should have its own user name “Automatic bug reporter” in Mantis -- a virtual person, instead of a real developer.

2. New PHP script (bug_report_auto.php): the new Mantis PHP script will be triggered by “SubmitErrorReport” of Cylogger. This PHP script will (1) check the summary (subject line of the bug report) of the Bug report (2) if this is a new bug, create a new Bug in Mantis, if this is a duplicated bug, append the new report to existing one.

Web page about fixed-bugs

We may need a web page to list bugs-fixed (CyBugsFixed.php) at Cytoscape web site. On the page, fixed bugs for each Cytoscape versions would be listed, so that users could search for the bugs they care about and get advice/suggestions on possible work-arounds. Currently users can search for particular bugs in Cytoscape bug tracker, Mantis. But this is mainly for developers, not for ordinary users.

Related RFCs

Link to other related RFCs

Issues

List any issues, conflict, or dependencies raised by this proposal

Comments

MikeSmoot

• How would ErrorCodes get assigned? Would we need to manually assigned a code to every exception that gets thrown? Would it be easier to assign codes based on the Exception or the package that contains the Exception?

• This seems to dovetail with merging the error messages produced by CyLogger and those produced by the Task framework.

• I would expect the bug report to include the version of cytoscape and possibly a status dump from OSGi that tells us what bundles are loaded, versions, etc..
• It would be useful if the dialog provided an option for users to submit their session file along with the report.

• It would be really useful if we had a command provenance or history system that could tell us everything the user did since startup and we were able to include that in the report.