Crocodiles, Zebras, and Hard Exams

Education and mathematical social media has recently been alive with a discussion of a problem which appeared in this year’s Scottish Higher exam.

[Background: for anyone reading this from outside the UK, this exam is taken at the end of the final year of school, by which stage most students (including almost all those taking this paper) will have specialised to about four subjects. Achieving particular results in these exams will be the condition set by many universities for candidates to take up places.]

The paper under discussion can be found here. It’s the second paper in the set, where calculators are permitted. The crocodile question that has attracted particular attention is Q8, and I hope it is ok to reprint it here, though of course copyright lies with the SQA:

CrocodileQI’ll come back to the rest of the paper later, but for now let’s talk about some of the reasons why candidates found this question challenging under time pressure.

The structure of the argument prompted by the question is the following. We are modelling a situation which might arise in the real world involving crocodiles, or in a variety of different contexts. There are a number of ways to get between two places, involving travelling through two different types of routes, at different speeds. Which is the fastest possible route? We describe the routes with a parameter, and after thinking briefly about the range of parameter, we can calculate (in this case using Pythagoras – the river has constant width presumably – and the now very familiar distance-speed-time relation) the time taken for each route. Finally, finding the fastest possible route is an exercise in minimising this function, for which we differentiate.

I’m sure the well-prepared candidates had seen all parts of this argument in various contexts during class and revision. What I imagine made this question seem confusing was that the examiners actually did the first step for the candidates, by telling them how the time taken depended on x, without expecting them to engage with any of the geometry or dynamics implicit in the answer. So all that is left for the candidates is to substitute in two values into the expression, and then differentiate to find the minimum, possibly with a small additional argument to say why this should be a minimum.

Obviously, if they just said “Find T(0) and T(20)”, then part a) would not be meaningful in any way as a test of ability. So you get two marks for correctly interpreting the ‘word problem’, that travelling only in the water corresponds to x=20, and minimising water time corresponds to x=0. Now the paper is out of 70, so this fine, but you can also now see why this caused confusion. Is x actually a variable according to the pre-amble? Look at Q2 (which is all about domains and so on). Students are expected at this stage to be familiar with the technicalities of defining functions and their arguments. Well the phrase “particular point” is used, so if we are going to be literal, then x is not a variable, and T(x) is just one number so given this information about about the time taken under the minimal route, we can’t do part a).

Of course this is deliberately artificial. I did not think what I have just described, and nor would any experienced mathematician. But maybe this contributed to the confusion?

Applied Problems

On a friend’s Facebook post referencing this question, someone had commented, loosely paraphrased: “This sort of question will put students off maths because they tell you the answer. This will make students feel that the maths they have learned isn’t useful.”

I disagree with this sentiment on several levels. Firstly, while no-one wants exams to be crushing for the candidates, nor a tedious regurgitation of routine material, the time to become convinced of the usefulness, relevance and interest of the course material is, well, during the course. It’s hard enough to write questions that test understanding and competence at core material, without adding the constraint that it has to make integration seem important. Hopefully some questions may seem interesting and relevant, but this is neither objective, nor the central ambition of such an exam.

Secondly, interpreting intermediate steps which you are given is a hugely important skill, and in this particular example, the fast-forward mechanism of the question makes it even more clear why the maths they have learned is useful: I know how to differentiate things with square roots, so I can use that new skill to solve problems about crocodiles. While I wonder whether the original plan had been to get candidates to derive T(x), I’m not sure whether this could really be said to be a useful application of maths that they’d learned, since they learned it several years previously.

In some other areas of commentary, it was raised that this is not a realistic model of crocodile mechanics, because it fails to take into account an almost infinite number of other factors. At least the ones that said the zebra would have just run away were joking. Others were not joking, but I feel it outside the remit of this short post to discuss why simple mathematical models can still be valuable.

Crocodile_compressed‘Too hard’? ‘Grade review needed’?

I did think that the exam as a whole was pretty tricky for this level. Q1, for example, requires quite a lot of careful calculation, and it’s a short exam. The question about frogs iteratively climbing up a wall genuinely requires some thought. Perhaps this is a standard type of problem introduced in the syllabus, but in either case it’s a nice question. Thinking about functions and their fixed points and monotonicity, and convergence (or not) towards fixed points is a really useful thing to think about, much more so than calculations like eigenvectors for a 3×3 matrix which certainly used to be popular on some exam boards.

So the crocodile question needs to be seen in the context of a paper where even the strongest candidates would be under a lot of time pressure once they reached the final problems. In other words, it was a hard paper, and I personally believe it seems to have been hard in a good way. Despite this mild confusion for some about what they needed to do on the crocodile question, in general both competence with the course material, and creativity in problem solving should be rewarded by this paper.

So it’s a shame to see all the usual cliches about needing to re-think the grades coming out. I should admit that I haven’t checked the full story, but my impression from the quotations given by the SQA is that they wanted a harder exam, not fewer top grades. I don’t see any reason why performance on this harder exam would be less strongly correlated with the candidate’s understanding of the course material and general problem-solving ability than a more routine exam, such as many were clearly expecting. In particular, I doubt many students performed dramatically differently (in relative terms) on the harder exam than they expected to do on whatever they were expecting. Maybe the variance is slightly higher, since maybe some questions have more of a you-either-see-it-or-you-don’t flavour? But I don’t see any real case for this, and it’s somewhat counter-balanced by the greater variety of questions (if the implication is that you don’t have to do them all to do very well).

I can also sympathise with students who need a top grade to go to university. Traditionally, they would complete most of the questions on the paper because they had prepared carefully and had a good idea what sort of things might come up. Concerns about having done ‘well enough’ would still exist, but at least they knew what the parameters were, roughly, and could be fairly confident they’d done ‘enough’.

However, that’s not a good enough reason not to set harder problems. No test can be a perfect measure of ability, since essentially by definition no such thing exists. I’m afraid it’s automatic that for all the students surprised by the change of tone of some of these questions, there is an equal and opposite set of students who, though perhaps they didn’t realise it at the time, actually thrived on the chance to address more demanding material. And their achievement should be celebrated and stand as a message to next year’s students and beyond that some hard questions doesn’t mean everyone must fail, either to get the grade they want, or to engage with good mathematics.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s