What You Should Know About Computer Models
By Jason Makansi
Printed in Practical Homeschooling #15, 1997.
Jason Makansi tells us how to tell if a computer projection is likely to be accurate or wildly wrong.
Wise homeschoolers know the power of the computer. You use it to obtain valuable resources off the Internet, participate in discussion groups, and run educational software tailored to their own needs.
But power is easily abused. The use of "computer modeling" to understand complex phenomena, like what happens in our environment, is a great example. Kids absolutely must know what a computer model is, what it can do that's right, and what it can do that's terribly wrong.
Let me offer an example from my college days. I earned a B.S. degree in Chemical Engineering, but I took "liberal arts" electives, because they interested me. In an anthropology class, I learned all about early hominids, the creatures that are thought to link apes to humans. What surprised me was that the textbooks showed detailed depictions of these creatures, especially skeletal structures, based on only a few bone or skull fragments. In at least one case, a complete hominid ancestor was, according to my instructor, constructed from only one tooth. One tooth! Scientists developed theories and constructed a complex hominid starting with one tooth. I was, at the time, impressed and amazed. Well, that's similar to what happens with computer modeling.
Many environmental scientists believe that the earth is warming up dangerously - so-called global warming - because computer simulations predict higher global temperatures centuries, perhaps only decades, into the future. The model, or simulation, is based on equations that describe the complex chemical and physical interactions that take place between the atmosphere, the land, and our water. With some knowledge of starting conditions, the model (with perhaps dozens of interdependent equations) is then run to calculate and predict what temperatures may be in the future.
The trouble is, one scientist or group of scientists may believe in one set of equations that describe these interactions. Another group may support an entirely different set.
Remember the tooth? Anthropologists developed what they consider to be plausible models of hominids. These models are not just pulled out of thin air. They have teeth from other creatures, in many cases with lots of other bone parts, that they've used to construct "models." So they have at least some ability to relate a tooth to a full skeleton. But you have to realize how subjective the process really is. Anthropologists debate passionately over how the known input data - in this case a single tooth - is used to "predict" the look of a hominid ancestor. It's the same with computer models!
Models Must Be Validated
How do you know when a computer model is a good one? The model must be validated. Let's go back to the computer model for global warming. One way to test a model is to see if it can predict something that's known with some degree of certainty. In other words, a computer model that is supposed to predict the average temperature of the earth in fifty years should be able to confirm what the average temperature was fifty years ago.
Can you begin to see how difficult it is to validate a complex computer model? Would you be convinced that we even have enough information and data to know what the average temperature of the earth was fifty years ago? And how accurately do you think we know that information? Remember, we're not talking about the average temperature in St. Louis for the past fifty years, but the average temperature around the globe! Do you think they had good records of temperatures in the Amazon jungle in 1936? The quality of the data input to the model is just as important as the assumptions used to construct the model.
Here's another good question about computer models. How many parameters are included? Let's go back to the tooth. Perhaps the easiest way to understand how a skeleton could be constructed from a tooth is to apply dimensional analysis. It's a big phrase but all it means is that the dimensions of the tooth - length, density, crown size, etc. - are compared to the dimensions of the skeleton. That's only one aspect, an obvious physical aspect. What if the model were able to relate chemical composition of the tooth to chemical composition of the body?
Some of the most recent models of global warming did not account for clouds. Surely most meteorologists would agree that cloud cover has much to do with how hot or cold the surface of the earth will be. Other models may not have properly accounted for sporadic events, like volcanoes, that are known to dramatically affect the weather. What kind of records do you think exist for volcanoes that erupted fifty years ago? One hundred years ago? Still other models assumed that oceans did not exist or used far too simplistic assumptions about the potential role the oceans or clouds have in affecting global temperatures.
We know that computers have become really powerful over the last few years. But it doesn't take too many parameters in a complex phenomenon like global warming to wear out all but the most powerful mainframes.
What Does it Mean?
Suppose you accept that the computer model can accurately predict an average temperature of the earth. What does that really tell you? And if the model shows that average temperature has indeed risen, it certainly does not tell you why. The premise in global warming seems to be that human activity is responsible for rising temperatures. An entirely different analysis is necessary to determine that the rise was not a purely natural or repetitive occurrence over a long period of time.
While my discussion here stresses the use of computer models in environmental issues, you can apply this insight anytime a computer model is involved. And you must recognize that these types of models are being used for everything. In fact, many scientists and engineers are worried because computer models are often substituted for real-world testing and analysis. Computer computational time is less expensive than paying people to take measurements over long periods of time, so we may be relying on it for the wrong reasons - to save money, not to improve the accuracy of discovery and analysis.
Environmental authorities often will accept a computer model of the impact on air quality of emissions from a new industrial facility rather than actually take measurements of the air in the surrounding community. All of the things I mentioned about computer models apply here. The one difference is that the smaller the space and the shorter the time span, the easier it is to model something. Thus, a local model for environmental impact has more validity than a regional, national, or global model.
Another example would be an auto manufacturer which simulates crashes of a new model car on the computer to determine safety features, rather than sacrifice the expense of crashing a few prototype cars in the real world. How the company constructs the computer model has everything to do with how safe that car will be.
So anytime you hear something like, "because the computer model predicted it . . ." or "based on our computer simulations . . ." you know some questions to ask. You can't change the fact that computers are an integral part of your kids' lives. But you can make sure your kids apply the same inquisitiveness and healthy skepticism to what's behind the computer. Remember, the existence of an entire hominid from millions of years ago was extrapolated from one tooth, before computers were even used for these purposes! Imagine what that creature might have looked like if anthropologists supplemented their informed imaginations with computers then!