The Test of Excellence

This week we were required to read an interesting article on math pedagogy entitled “Relational Understanding and Instrumental Understanding” by Richard Skemp. In the article, Skemp makes his case that too much math pedagogy is based on teaching calculation techniques and procedures (what he calls “Instrumental” understanding) as opposed to instilling a deeper understanding of mathematics, from which it should be apparent to the student why those techniques (“Relational” understanding). His thesis, if I may so condense it, is that students are rarely being taught actual mathematics but instead of simply memorizing computational rules and the situations in which they apply. Students then can correctly do specific tasks very well, but if presented with something they haven’t seen before, will be at a loss on how to proceed.  In principle, I very much agree with his concerns and have seen the equivalent problem in the first-year physics students I used to TA, who often viewed physics as little more than applying a succession of mysterious formulas of dubious origin. However, I am less enthusiastic when I imagine what would be required in order to implement Skemp’s ideas effectively, and what the outcome would be if the attempt failed.

There is something dangerously optimistic about viewing prospective public policy through the lenses of the best-case scenario. I am sure Skemp himself would be very effective at teaching his own methods, not least because he is an actual mathematician. However, a teacher cannot teach what he does not know, and I wonder what would happen if a teacher who had little to no background mathematics was made to teach in this manner, as though he were an expert. Speaking for myself, I believe I could teach math and physics in this way (and I intend to), but I am not sure that I could do it with chemistry. This is not because I don’t know anything about chemistry, but because I don’t have nearly the same level of expertise in it that I have in physics. I have observed this pedagogical approach go awry when I tutored high school students in physics. I had one student who was struggling quite a bit with relative motion. The student showed me diagrams the teacher was having them draw in order to describe different kinds of reference frames. It was a very “relational understanding” approach, and perhaps could have been effective. However, from the student’s course notes, copied from the board, I could see that proper use of the diagram had been explained incorrectly and that by making these diagrams, the student had been reinforcing a serious misunderstanding of the material. This would take a lot of work to fix because the error ran so deep in the student’s understanding. Thinking of experiences like this, the question that comes to my mind when I read this kind of pedagogical theory is: used well, this method promises certain obvious benefits, but what is the damage if used badly?

I am naturally resistant to a wide and enforced implementation of ideas, even ideas I agree with. This is because ideas have unintended consequences, and if some of the consequences are bad, a broad implementation will make them widespread instead of local and more easily addressed. Additionally, it is often the weakest and poorest members of society who are least equipped to cope with the consequences of bad policy. I recently read a very sobering article by the British psychiatrist Theodore Dalrymple, who worked for most of his career as a doctor in an English prison and in a neighboring slum hospital. He has produced a huge body of very depressing essays and articles describing the disastrous social consequences of ideas that begin at the top, trickle down, take root, and then run amok. In his article “We Don’t Want No Education”, he described some of his countless observations on the current educated state of his teenaged patients, part of which he blames on experimental educational practices:

… I now test the basic literacy of nearly every such youth I meet, in case illiteracy should prove to be one of the causes of his misery. (I had a patient recently whose brother committed suicide rather than face the humiliation of public exposure in the social security office of his inability to read the forms he was required to fill in.) One can tell merely by the way these youths handle a pen or a book that they are unfamiliar with these instruments … I cannot recall meeting a sixteen-year old from the public housing estates near my hospital who could multiply nine by seven (I do not exaggerate). Even three by seven often defeats them. One boy of seventeen told me, “We didn’t get that far.” This after twelve years of compulsory education (or should I say, attendance at school).

 http://www.city-journal.org/html/we-don’t-want-no-education-12348.html

I don’t think situations like this could flow immediately, or even naturally from a single idea or theory, but I do think that a good idea for a particular situation can become a very bad idea when pulled and stretched into a broad catch-all policy. Skemp’s theory, and theories like it I feel are abused when they used to downplay the importance of calculation. The question is often raised, why bother focusing on hand written calculation when we have calculators? Why not just focus on “understanding”. What strikes me most about this question (as well as the related, “why bother with spelling when we have spell-check”), is the way in which these skills are so carelessly disposed of, as though the mental development that comes from mastering these basic skills isn’t reason enough to teach them, as well as the enrichment of our lives that can only come from filling our minds with meaningful things. Why learn facts about history if you can just look it up when you need it? Why memorize any great sayings from poetry or philosophy when we have Wikiquote? This philosophy, though easy and convenient, seems to me to lead to a cold and stifling materialism. It is in this respect that I fear that we as a society are becoming a bit like Lady Bracknell, who disapproved of anything that interferes with natural ignorance. Setting aside the question of the intrinsic value of knowledge, at the heart of the pro-calculator stance there sits an assumption I’d like to address with respect to the purpose of calculation.

I occasionally observe an attitude towards the importance of calculation that I think misses the point, namely, that it is an end in and of itself. In any level of physics or mathematics, do we give students problem sets so that they can practice calculations, or rather so that by doing the calculations, they will further cement the foundations they are being taught? To give a simple example, a common beginner physics problem is to describe the motion of a box sliding down a ramp. Typically some variables will be given and the student has to use Newton’s Laws to calculate the speed of the box, or the coefficient of friction, or the angle the ramp makes with the ground etc. A student in high school or in the first year of university will see many problems like this, all alike but slightly different. At the same time it can be shown algebraically that these problems are mathematically identical and that, having solved it once, you’ve actually solved every permutation of it as well. What then is the point of doing the same thing over and over again? It is to force the student to re-derive the algebraic result, but each time within slightly different context, or from a different direction, hopefully deepening their understanding in the process. The students can be shown (and I do try to show them) that they are in fact doing the same thing over and over again, but it takes a lot of thinking and a lot of practice in order to see it. Once you reach that point, I would say that you truly understand it. In this case, the arithmetic involved is secondary, and it would not defeat the purpose of the exercise to use a calculator. (As an aside, I’ve directly observed many students whose ability to do these problems is impeded by their reliance on calculators, because instead of working through the problem algebraically and then putting in the numbers, they just number crunch the entire time. If they get the wrong answer, it’s impossible to see where they went wrong because their work up to that point is just a meaningless list of numbers.)

This kind of physics problem builds on top of skills that should already have been learned. In particular, arithmetic, solving sets of linear equations, and rearranging equations. To learn each of these skills, you would have to go through the same process described above, but at a lower level. You would learn the theory (hopefully taught meaningfully, as Skemp would advocate), and then practice that skill over and over, from different angles, cementing your learning and fixing your misconceptions. For learning arithmetic, using a calculator would completely defeat the purpose. If the point was to simply do the calculation, it could make sense to use a calculator since it may be faster and less error-prone (though for really simple problems, I’m not so sure). But we do not learn arithmetic to do simple calculations, we do calculations to assess and develop our arithmetic, and to ensure that the foundation is in place in order that we may proceed to more advanced topics.

I’ve seen this kind of thinking before, albeit in a very different context. One of the most important pedagogical works in the history of music is Johann Fux’s Gradus ad Parnassum from 1725. This book was used to instruct many of histories greatest composers including Mozart, Haydn, and Beethoven. J.S Bach, perhaps the greatest musician of all time, strongly approved of it. I would like to write about it in more depth in a later post but for now, I will note that the structure is written as an actual dialogue between a teacher and a student. The teacher explains the principle and the student does a series of short exercises. After each exercise, the student explains what he did and why, and the teacher approves or corrects them while explaining the reason for his mistakes. Each chapter builds on the one that preceded it, until by the end, you are writing in full four-part harmony. It’s a remarkable approach, but what strikes me about it is the balance Fux achieves in combining theory and conceptual understanding with practice. Sometimes I think we feel that if the understanding is strong enough, correct usage will follow, but in my own experience as a student, this was rarely the case. They feed and correct each other. Fux states this explicitly in his introduction:

You will notice dear reader, that I have given very little space in this book to theory and much more to practice, since action being the test of excellence, this was the greater need.

Similarly, to take a more contemporary example, J. J. Sakurai in the foreword to his graduate level textbook on quantum mechanics states, “the reader who has read the book but cannot do the problems has learned nothing”.  I think most people would agree with this sentiment when applied to advanced science, but why is it a point of contention when applied to elementary math? I would maintain that the student who has been taught arithmetic but needs a calculator has learned nothing since, as Fux said, action is the test of excellence.

Verdicts that Demand Some Evidence

You had that action and counteraction which, in the natural and in the political world, from the reciprocal struggle of discordant powers draws out the harmony of the universe. These opposed and conflicting interests which you considered as so great a blemish in your old, and in our present constitution interpose a salutary check to all precipitate resolutions. They render deliberation a matter, not of choice, but of necessity; they make all change a subject of compromise, which naturally begets moderation; they produce temperaments preventing the sore evil of harsh, crude, unqualified reformations, and rendering all the headlong exertions of arbitrary power, in the few or in the many, forever impracticable.

 – Edmund Burke (1729-1797), Reflections on the Revolution in France

Socially and technologically, the world is perhaps more impermanent and unsettled than it has ever been. The 20^th century was best characterized by the decade. For the 21^st century that division may prove to be even smaller. Despite such uncertainty, innovation and change continue to be popularly embraced as good in and of themselves. In a strange reversal, the tried and true are more often viewed with feelings of mistrust and quite often, outright hostility. To take an important contemporary example, we are currently facing the very real possibility of losing the “first past the post” electoral system. Leaving aside the claimed merits of the current and proposed systems, in the surrounding debate how often is the fact that this system has functioned satisfactorily for the past 800 years used as an argument against its favour? I remember an email advertisement from the NPD that made point explicitly, noting in particular and with disdain that the “first past the post” system predates penicillin and the Internet (what that has to do with anything, I do not know). MP Elizabeth May made a similar argument in the House of Commons claiming, incorrectly I might add, that the Medieval Britain believed that the Earth was flat. It is not this political issue that I want to discuss, but rather what it means about our society that major political parties and high-profile public figures can get away with such weak reasoning on such a major question. What has happened to the modern world that we are so deficient of caution that we happily trade nearly a millennium of continuity for vague and untried abstractions? What is the source of this remarkable confidence in innovation?

This question of innovation and the confidence we place in it came to my mind last week. In one of my classes, we were discussing the current practices used in schools to integrate children with special needs more fully into the classroom. We were shown a quote from a government document that immediately struck most of the class as going against the current wisdom. A wave of comprehending “ahhs” swept through the room when it was revealed that the document dated way back to 2006. I wondered then what it means when an educational policy hardly 10 years old would be so recognizably out of date. I suppose one could argue that it is all part of that inexorable march forwards, but I am not so sure. If this or that policy or curriculum has a shelf life of barely a decade, what confidence can we reasonably place in the supposed merit of the current version? Will we always wake up a few years later, recognize it’s indefensible shortcomings and go back to the drawing board? Grounded to nothing, is education to be stuck in an endless present, constantly in a state of overhaul and reinvention? Do children really change that much? Does math?

I suspect that the penchant of politicians to implement frequent changes in policy based on innovation and the latest ideas, as well as the willingness of the public at large to accept them, is that these ideas lay claim to a foundation in science and “the latest research”. It is nothing new (nothing ever really is), but I seem to be noticing more than ever that the public perceptions of science and ‘expertise’ appear to be more akin to a banner of authority than as a method of investigation that welcomes scrutiny and debate. I believe that the practical effect of treating science and expertise as an ipso facto authority, combined with a dismal and socially endemic level of numeracy, is a general loss of useful skepticism. An interesting and thorough investigation of this idea can be found in Dr. Thomas Sowell’s book Intellectuals and Society.

I believe that skepticism coupled with logic is perhaps the most important regular use of arithmetic. Yet I think it is one of the least inculcated. We are flooded with facts and figures everyday, through the news, political advertisements, commercial advertisements etc. Are we as a society equipped to properly decipher this information, and to separate the solid from the untrustworthy? I would like now to cover a few of the ways in which we can do this. Do we catch the notable omissions, such as numerators without denominators? I saw a simple example of this in a political ad from the last federal election. Then Liberal Party candidate Justin Trudeau presented a graph comparing of the growth of the size of the economy over the past few decades, compared to the seemingly small growth in household income. I can’t exactly remember the message and I don’t want to misrepresent it but I think it was along the lines of “the rich are getting richer and poor are getting poorer” (paradoxically, people have been saying that unceasingly for hundreds of years). The issue is that household income is a numerator without a denominator. It is not a useful number without corresponding trends in household size and the number of earners per household. The graph then, though factually correct, did not present information from which conclusions could be reliably drawn.

Do we ever track down the original data or do we except shocking statistics at face value? Do we naturally ask follow-up questions? I’ve heard that the best one to ask, after “what is your evidence?” is “compared to what?”. To take an example from the economist Thomas Sowell, there was a media scare in the United States about 20-30 years ago when statistics appeared painting a very dire picture about hunger in the United States. Different news outlets everywhere started reporting on the national crisis. Dr. Sowell found the numbers suspiciously high and tracked down the original data. It turned out that the methodology used in the original report was unreliable. In order to measure hunger, the authors defined as “hungry” anyone who is eligible for food stamps but not receiving food stamps. This meant that someone who worked a job that was low income but included food and board was by definition hungry, which greatly inflated the numbers. Incidentally, the ‘hungriest’ county in the United States turned out to be a farming community where they grew and ate their own food. Is this to say then that there wasn’t a hunger problem? Not at all! But a countrywide crisis would require a very different response than localized malnutrition in, for example, poor urban centres. The information as it was presented generated an inappropriate response, and could have caused some damage. This kind of fact checking is hard in practice, but a good sense of numbers should provide even a non-expert with an instinct for reasonable statistics.

Are we in the habit or checking whether bold claims clash with our own experience? Although people can make abstract arguments against the rote teaching of arithmetic, I cannot bring myself to reject the rote method (n.b. arithmetic, not mathematics). This is not because I have no faith in science or reason, but because I believe that the claim is too bold and the evidence I have seen too meager against my personal experience, both as a student and as a tutor. I consider myself to have a good sense of numbers, and I attribute it to the fact that my homework when I was young often consisted of pages of arithmetic problems (without a calculator), and that my mom and my grandmother would randomly ask me arithmetic questions at home. I’ve never come across a student who had no capacity or even slight affinity for abstract mathematics, but I have met many, even in university, who were predestined to failure by a very poor sense of numbers, rendering even the simplest equation rearrangement, or expression simplification an ordeal.

The other danger of misplaced confidence is that it can allow nonsense with the appearance of rigor to go unchallenged. A funny example of a falsehood presented with an air of presumed authority and under the banner of SCIENCE is found in Friedrich Engels' polemic Anti-Dühring. The Russian mathematician and colleague of Alexandr Solzhenitsyn, Igor Shafarevich wrote a tract entitled The Socialist Phenomenon that explored the common themes of Socialism throughout the world and throughout history. In the chapter on 19^th century socialism, he is particularly critical of the apparent regard the writers had for the scientific method, which he says in practice amounted to little more than lip service:

The concept of the ‘scientific method’ was of extraordinary importance for the development of 19^th century socialism. Hence it was steadily and persistently elaborated, first by Fourier [not the mathematician Fourier] and Saint-Simon and later in a much more sophisticated manner by Marx and Engels. The scientific method provided the socialist doctrines with a ‘sanction’ of the first order. Furthermore, the theses of socialist doctrine thereby acquired the appearance of objectivity and a certain inevitability, being presented as a consequence of immanent laws independent of human will.

In other words, science was used to provide the guise of objectivity, but was not actually used. One of many examples he gives comes from Friedrich Engels and his use of mathematics to justify a position on epistemology. In the forward to Anti-Dühring, Engels states “Awareness of the fact that I have not sufficiently mastered mathematics has made me careful: no one will be able to find me trespassing against the facts.” He then goes on to state several bold claims about mathematics which are either completely false or represent a gross misunderstanding of the subject. Shafarevich then slyly notes:

As for political economy or history, Marx and Engels clearly did not believe that they had ‘not sufficiently mastered’ these subjects; nothing prompted them to be ‘careful’ as with mathematics. One may well imagine how resolutely they operated in these areas.

What I hope to illustrate by this little example is what I worry is a growing trend in the world of public policy, namely the use of SCIENCE not to further a discussion, but to end it. Setting aside Marx, Engels and my grim Cold War paranoia, how often in the news, particularly in relation to the newest public policy do we hear words and phrases like “evidence-based decision making”, “leave it to the experts”, “the latest research”, and “the scientific consensus”? How often are those terms used to placate or rebuke a perturbed public? Science, properly done, should welcome skepticism and scrutiny. Whether it can withstand the scrutiny is what marks it as good science. It should always be able to substantiate bold claims.

The difficulty in going against fashionable ideas is that it can have the appearance of contrarianism or audacity. No layman in his right mind would contradict Hawking in physics, or Chomsky in linguistics. But there is something intrinsically different about fields such as psychology and education. By their direct influence on public policy, these fields are more immediately impactful on our lives than, for example, the latest research in cosmology. Additionally, the nature of data is much different. Setting controls in social science experiments can be vastly complicated. Correlation and causation are not as easily distinguished. Reproducibility is harder to verify. There is a very interesting phenomenon called the “Decline Effect” in which the empirical evidence for a phenomenon gradually disappears over time (but I’ll leave that for another day).

To distinguish true from false, and plausible from implausible is a skill that nobody can possess perfectly, but it is a skill that we all need to some degree. We can’t all be economists or financiers (I’m certainly not), but a decent sense of numbers combined with a prudent degree of caution is an invaluable tool. I do not believe skepticism in science or statistics should be explicitly taught (and definitely not to young children), because that may lead to mistrust and cynicism. But I do believe that actively thinking about the numbers we encounter everyday should be encouraged, both for the mental exercise and the practical effect of encouraging a healthy position of caution. The examples that I gave in this article focused more on economics but it appears everywhere. Is this a good deal on a loan? Do the numbers advertised by that organization asking for a donation make sense? Does my answer to a word problem on my physics assignment sound right? It is a practical, and dare I say, fun daily application of mathematics, but because it is predicated on a certain degree of comfort with arithmetic, it may not be a practical habit to encourage so long as the current state of numeracy amongst students entering high school remains so discouraging.

About Me

I am a current teacher candidate at Brock University for which my teachables are physics and mathematics. I completed by B.Sc and M.Sc in physics at McMaster University. Since graduating, I have stayed on with my supervisor Dr. An-Chang Shi as a researcher studying theoretical soft-condensed matter with a focus on block copolymer self-assembly. Besides polymers, I am interested in physics and math pedagogy, and the question of why people either seem to these subjects easy or insurmountably difficult. My hobbies and interests include classical music, economics, and old books. I love discussing current events and the history of ideas. The name of my blog comes from a veiled criticism a high school teacher once made about my writing style (I thought it was a great expression).