Removing Form Data in FireFox and FireBird
Information stored while browsing in FireFox is stored in encrypted files within the install directory. When a user asks FireFox to delete this information, the files actually grow in size, meaning that the information might not actually be erased.
Mozilla's FireFox is an Internet browser available for Linux, MacOS, and Windows. At the time of writing this presentation, FireFox was still pre-1.0 (meaning that it was still a preview release for testing purposes).
This example demonstrates the use of Regression testing. We use Regression testing to ensure that the bugs we have already encountered do not reappear in later versions of the program after they have been fixed. Regression testing is not a specified type of testing, rather it is more the awareness and implementation of past testing techniques to monitor applications for the reappearance of issues or the breaking of a working function by new code.
In this presentation, we will be looking for a bug that used to affect the first generation code of FireFox, named FireBird. The bug in FireBird was that when a user asked FireBird to delete the form history (a copy of the information that users have typed into forms), the information was not actually deleted. The encrypted file in which it was stored, formhistory.dat, actually grew in size. The risk is that the information is not actually removed, but instead the application is just told to ignore it.
A recent version of FireFox on Linux operating systems exhibited the same bug as the one in FireBird. If a user had a formhistory.dat file that was 5 kilobytes, asking FireFox to delete saved form information resulted in formhistory.dat growing to 5.3 kilobytes. Likewise, a 10 kb file would grow to 10.3 kb. This implies that the saved form information is still within the file, and therefore it may still be accessible, whether by accident or by a malicious user. That saved information can range from search queries and addresses to creditcard numbers and passwords.
The testing technique used originally to find the bug was Exploratory testing. When this bug was first found, we could have taken notes on it, reviewed the notes, and streamlined our exploration efforts into an efficient and duplicable test. As we repeat the test, we are now performing Regression testing.
Regression testing is a very necessary technique for maintaining a stable product release (one that is at least as stable as the releases before in terms of the same features). We observe the results of a regression test by directly examining the program's reaction to the tests that have previously been run before, and expect to receive the same result as we did upon a previous successful run of the test (the result we recieved once the bug was corrected).
Performing the Test
As we can see, our formhistory.dat file is extremely small upon first using FireFox. It is just under 1 kilobyte in size.
The formhistory.dat file has grown roughly 9 kilobytes since we started the test, now being 10 kilobytes. Therefore, we can assume that FireFox has now stored about 9 kilobytes worth of form data that we entered.
Not only has our formhistory.dat file retained its size, but it has actually grown in size. The information is encrypted within the file, but the fact that it still remains is of some concern. We are trusting the application to remove our protected, private information. For it not to do so is a critical security concern.
So, in this case, which is more important: that we found that private information may still be stored on the computer or that we keep testing to find if it is accessible? Determining if it actually is accessible is not necessarily important for the tester (unless they know how to do it without wasting important time that could be dedicated to finding other bugs). Why? It is not too important because we have already found the flaw and we know of the risk. Whether or not we verify that there is a way to retrieve the information and judge the severity of the bug, the issue is still that information that was trusted to be removed was not. Whether other users can easily access it or not (though an important question) may not even be the focus: a customer has an important expectation, and if that expectation is not fulfilled, they may cease using the program, especially if that expectation involves a large amount of trust.
The programmers at Mozilla are (at the time of writing this presentation) currently working on the bug and hope to have it fixed by the next minor release.
Similar Tests/Additional Notes
For more information on the bug in this presentation, see the Mozilla Bugzilla database.
Alternatively, search through the Mozilla Bugzilla database and search for newly discovered bugs. Then try to find those same bugs on older versions of FireFox to better recognize how Regression testing aides in the development of software applications.
Testing Mozilla's FireFox v0.9.3 on: