Difference between revisions of "How to report bugs?"

Latest revision as of 19:26, 8 October 2016

Introduction[edit]

One of the most frustrating experiences one can have as a software user is to submit a problem report only to have it summarily closed with a terse and unhelpful explanation like “not a bug” or “not useful PR”. Similarly, one of the most frustrating experiences as a software developer is to be flooded with problem reports that are not really problem reports but requests for support, or that contain little or no information about what the problem is and how to reproduce it.

This document attempts to describe how to write good problem reports. What, you ask, is a good problem report? Well, to go straight to the bottom line, a good problem report is one that can be analyzed and dealt with swiftly, to the mutual satisfaction of both user and developer.

Although the primary focus of this Document is on GhostBSD problem reports, most of it should apply quite well to other software projects.

Note that this document is organized thematically, not chronologically, so you should read through the entire document before submitting a problem report, rather than treat it as a step-by-step tutorial.

When to submit a problem report[edit]

There are many types of problems, and not all of them should engender a problem report. Of course, nobody is perfect, and there will be times when you are convinced you have found a bug in a program when in fact you have misunderstood the syntax for a command or made a typographical error in a configuration file (though that in itself may sometimes be indicative of poor documentation or poor error handling in the application). There are still many cases where submitting a problem report is clearly not the right course of action, and will only serve to frustrate you and the developers. Conversely, there are cases where it might be appropriate to submit a problem report about something else than a bug—an enhancement or a new feature, for instance.

So how do you determine what is a bug and what is not? As a simple rule of thumb your problem is not a bug if it can be expressed as a question (usually of the form “How do I do X?” or “Where can I find Y?”). It is not always quite so black and white, but the question rule covers a large majority of cases. If you are looking for an answer, consider posing your question to the FreeBSD general questions mailing list.

A bug that can not be reproduced can rarely be fixed. If the bug only occurred once and you can not reproduce it, and it does not seem to happen to anybody else, chances are none of the developers will be able to reproduce it or figure out what is wrong. That does not mean it did not happen, but it does mean that the chances of your problem report ever leading to a bug fix are very slim. To make matters worse, often these kinds of bugs are actually caused by failing hard drives or overheating processors — you should always try to rule out these causes, whenever possible, before submitting a problem report.

Next, to decide to whom you should file your problem report, you need to understand that the software that makes up GhostBSD is composed of several different elements:

• Code in the base system that is written and maintained by FreeBSD contributors, such as the kernel, the C library, and the device drivers (categorized as kern); the binary utilities (bin); the manual pages and documentation (docs); and the web pages (www). All bugs in these areas should be reported to FreeBSD.
• Code in the GhostBSD base system that is written and maintained by others, and imported into FreeBSD or GhostBSD and adapted. Examples include Pidgin, GBI(GTK BSD Installer) and Firefox. Most bugs in these areas should be reported to the GhostBSD developers; but in some cases they may need to be reported to the original authors instead if the problems are not GhostBSD-specific. Usually these bugs will fall under either the bin or gnu categories.
• Individual applications that are not in the base system but are instead part of the FreeBSD Ports Collection. Most of these applications are not written by FreeBSD and GhostBSD developers; what GhostBSD provides is FreeBSD desktop system already to use. Therefore, you should only report a problem to the GhostBSD developers when you believe the problem is GhostBSD-specific; otherwise, Report it to FreeBSD if it FreeBSD-specific. If it is not GhostBSD-specific or FreeBSD-specific you should report it to the authors of the software.

Then you should ascertain whether or not the problem is timely. There are few things that will annoy a developer more than receiving a problem report about a bug she has already fixed.

If the problem is in the base system, you should first read the FAQ section on GhostBSD Wiki, if you are not already familiar with the topic. It is not possible for GhostBSD to fix problems in anything other than certain recent branches of the base system, so filing a bug report about an older version will probably only result in a developer advising you to upgrade to the latest version to see if the problem still recurs.

Preparations[edit]

A good rule to follow is to always do a background search before submitting a problem report. Maybe your problem has already been reported; maybe it is being discussed on the mailing lists, or recently was; it may even already be fixed in a newer version than what you are running. You should therefore check all the obvious places before submitting your problem report. For GhostBSD, this means:

• The GhostBSD Frequently Asked Questions (FAQ) list. The FAQ attempts to provide answers for a wide range of questions, such as those concerning hardware compatibility, user applications and more.
• The mailing lists. If your problem has not been discussed on the lists, you might try posting a message about it and waiting a few days to see if someone can spot something you have overlooked.
• Optionally, the entire web. use your favorite search engine to locate any references to your problem. You may even get hits from archived mailing lists or newsgroups you did not know of or had not thought to search through.
• Next, the search GhostBSD Track system. Unless your problem is recent or obscure, there is a fair chance it has already been reported.

However, if the problem is in something that was installed as a part of the FreeBSD Ports Collection, you should refer to /usr/ports/UPDATING (for individual ports) or /usr/ports/CHANGES (for changes that affect the entire Ports Collection). http://svnweb.freebsd.org/ports/head/UPDATING?view=log and http://svnweb.freebsd.org/ports/head/CHANGES?view=log are also available via svnweb.

Writing the problem report[edit]

Now that you have decided that your issue merits a problem report, and that it is a GhostBSD problem, it is time to write the actual problem report. Before we get into the mechanics of the program used to generate and submit problem reports, here are some tips and tricks to help make sure that your problem report will be most effective.

Tips and tricks for writing a good problem report[edit]

Avoid using a weak “Synopsis” line. You should not assume that anyone reading your problem report has any context for your submission, so the more you provide, the better. For instance, what part of the system does the problem apply to? Do you only see the problem while installing, or while running? To illustrate, instead of Synopsis: portupgrade is broken, see how much more informative this seems: Synopsis: port ports-mgmt/portupgrade coredumps on -current.

If you have logs, say so. A problem report with logs included is much more likely to be looked at than one without. If you are including one, put the string [log] (including the brackets) at the beginning of the “Synopsis”. (Although it is not mandatory to use that exact string.)

Be specific. The more information you supply about what problem you are having, the better your chance of getting a response.

• Include the version of GhostBSD you are running (there is a place to put that) and on which architecture.
• If the problem can be reproduced easily, include information that will help a developer to reproduce it themselves. If a problem can be demonstrated with specific input then include an example of that input if possible, and include both the actual and the expected output.

Avoid vague requests for features. Problem reports of the form “someone should really implement something that does so-and-so” are less likely to get results than very specific requests. Remember, the source is available to everyone, so if you want a feature, the best way to ensure it being included is to get to work! Also consider the fact that many things like this would make a better topic for discussion on GhostBSD development mailing list than an problem report, as discussed above.

Make sure no one else has already submitted a Problem report similar. Although this has already been mentioned above, it bears repeating here. It only take a minute or two to use the web-based search engine at https://sourceforge.net/p/ghostbsdproject/tickets/

Report only one issue per Problem Report. Avoid including two or more problems within the same report unless they are related. When submitting patches, avoid adding multiple features or fixing multiple bugs in the same problem report unless they are closely related, such problem reports often take longer to resolve.

Avoid controversial requests. If your problem report addresses an area that has been controversial in the past, you should probably be prepared to not only offer patches, but also justification for why the patches are “The Right Thing To Do”. As noted above, a careful search of the mailing lists is always good preparation.

Be polite. Almost anyone who would potentially work on your problem report is a volunteer. No one likes to be told that they have to do something when they are already doing it for some motivation other than monetary gain. This is a good thing to keep in mind at all times on Open Source projects.

Before you begin[edit]

If your submission will be lengthy, you should to prepare your work offline so that nothing will be lost in case there is a problem submitting it. This can especially be a problem with the web form.

Filling out the template[edit]

Originator: Please specify your real name, followed by your email address in the email interface.

Synopsis: Fill this out with a short and accurate description of the problem. The synopsis is used as the subject of the problem report email, and is used in problem report listings and summaries; problem reports with obscure synopses tend to get ignored. As noted above, if your problem report includes a log, please have the synopsis start with [log] (including the brackets). If your problem report includes a patch, please have the synopsis start with [patch] (including the brackets)

Category: Choose an appropriate category.

The first thing you need to do is to decide what part of the system your problem lies in. Remember, GhostBSD is a complete operating system, which installs both a kernel, the standard libraries, many peripheral drivers, and a large number of utilities. However, there are thousands of additional applications in the Ports Collection.

Here is the current list of categories:

• System Bug: Any errors encountered in the the default system.
• Software Bug: Errors on software from a default install of GhostBSD .
• Website Bug: errors on GhostBSD website.
• Syntax-Issue: Syntax error in the GhostBSD website, documentation or software.

Release: The version of FreeBSD that you are running. This is filled out automatically if you are using send-pr(1) and need only be changed if you are sending a problem report from a different system than the one that exhibits the problem.

Finally, there is a series of multi-line fields:

• Environment: This should describe, as accurately as possible, the environment in which the problem has been observed. This includes the operating system version, the version of the specific program or file that contains the problem, and any other relevant items such as system configuration, other installed software that influences the problem, etc.—quite simply everything a developer needs to know to reconstruct the environment in which the problem occurs.
• Description: A complete and accurate description of the problem you are experiencing. Try to avoid speculating about the causes of the problem unless you are certain that you are on the right track, as it may mislead a developer into making incorrect assumptions about the problem.
• How-To-Repeat: A summary of the actions you need to take to reproduce the problem.
• Fix: Preferably a patch, or at least a workaround (which not only helps other people with the same problem work around it, but may also help a developer understand the cause for the problem), but if you do not have any firm ideas for either, it is better to leave this field blank than to speculate.

Sending off the problem report[edit]

If you are using the web form:

Before you hit submit, you will need to fill in a field containing text that is represented in image form on the page. This unfortunate measure has had to be adopted due to misuse by automated systems and a few misguided individuals. It is a necessary evil that no one likes; please do not ask us to remove it.

Note that you are strongly advised to save your work somewhere before hitting submit. A common problem for users is to have their web browser displaying a stale image from its cache. If this happens to you, your submission will be rejected and you may lose your work.

Follow-up[edit]

Once your problem report has been filed, you will receive a confirmation by email which will include the tracking number that was assigned to your problem report and a URL you can use to check its status. With a little luck, someone will take an interest in your problem and try to address it, or, as the case may be, explain why it is not a problem. You will be automatically notified of any change of status, and you will receive copies of any comments or patches someone may attach to your problem report's audit trail.

If someone requests additional information from you, or you remember or discover something you did not mention in the initial report, please email your followup to ghostbsdproject-bugbusters@lists.sourceforge.net, making sure that the tracking number is included in the subject.

Wrong way:

Subject: that PR I sent

Right way:

Subject: Re: ports/12345: compilation problem with foo/bar

If the problem report remains open after the problem has gone away, just send a follow-up (in the manner prescribed above) saying that the problem report can be closed, and, if possible, explaining how or when the problem was fixed.

If you are having problems[edit]

Most problem reports go through the system and are accepted quickly; however, at times we see the email warning, you may not get your email confirmation for a day or even longer. Please try to be patient.