Checking Code for Correctness

It is very difficult to be perfectly certain that a program of any complexity is error-free. But, one can peform certain somewhat rigorous and very useful tests.

Testing calculations

The following is tedious and painful and we curse the entire time while we are performing it, but we have found that it is very helpful in checking code that performs any sort of calculation.

1. Write your code either at the beginning or the end. It doesn’t matter. In some cases, this procedure can be used to help design your code as well as to check it.
2. Write into your code, at various points within the calculation, the ability to have the current values of important variables written to a file. Write these values to at least 15 significant figures.

• Only 12 or so are likely to actually be good on a typical system. If your system keeps up with more, then you can print more. But, there’s little point in printing many more than your spreadsheet uses (you’re on your own to figure out how many that is).
• Writing so many decimal places makes it easier to find subtle errors when considering only a few steps of a calculation.
• Of course, most errors are not so subtle as to require a check to 12 significant figures, but if you’re going to bother to test, you might as well do this, just in case.
• It is best to have a flag defined in your program that turns these writes on and off rather than commenting the writes out or deleting them.
• Have the code write these values out in a format that will be easy to use for the following steps.

3. Design a spreadsheet (using your favorite application) that performs a do-able number of calculations that are identical to those performed by your program.

• Vary the number of calculations based on their complexity. If you are performing a series of the same very complex calculation, it is best to go through at least 3 and preferably 4 iterations. If the calculation is fairly simple, do 100 or 1000.
• If you’re doing a correlation or something that only gives meaningful answers when carried to the end, don’t feel you have to go so far as being meaningful. Just make sure that you’ve done at least a few of every type of calculation you have to do. Don’t forget to write your program so that it can produce the exact (even meaningless) calculation your spreadsheet performs.
• Make your spreadsheet print at least 15 significant figures, too.
• Do not try to do an entire complex calculation in just a few cells. Split out sines, square roots, etc. Remember that the point is to be able to compare the calculations at whatever detail is required to troubleshoot and debug.
• Make space in your spreadsheet for copying in the results from the program and comparing them to the spreadsheet results.

4. Run your program and then import the results to the spreadsheet and compare them.

• Typically, you will get at most 10-12 significant figures of exact agreement, depending on a number of factors. For example, your system might only keep up with 12 significant figures in the first place. You might also see more drift with certain operations (square root, for example).
• If the final calculation doesn’t agree to 10-12 significant figures, you should search for the source(s) of the discrepancy between the spreadsheet and the program.
• In most cases, discrepancies indicate a computational error, either in the spreadsheet or your program, however…
• Sometimes, the discrepancies are legitimate. But you should know exactly where they occur and why they occur.

Obviously, this procedure will only ensure that your program and the spreadsheet are performing the same calculation. If you have not designed the calculation properly, it can still be wrong. But, at least with this you can be fairly certain that the program is doing what you had intended for it to do, whether those intentions were well founded or not.

To check whether your intentions are good, we suggest finding some external measure. For the external measure, you cannot typically expect 10-12 significant figures of agreement. But, you can at least know if your calculation is in the ballpark. For example, if you are calculating the dipole of a molecule, have some other program perform an equivalent calculation and compare them.

If you want to contribute to GLYLIB, we request that you submit your spreadsheet and your external comparison. Include explanations for any places where there are disagreements. Please submit the spreadsheet either in Open Office or Microsoft Excel format. We realize this is tedious, but we aren’t asking you for anything we wouldn’t do ourselves.

Testing Text Manipulations (file reads and such)

We will happily receive any suggestions you have!