Working inside the Black Box: Assessment for Learning in the Classroom

Classified as probably not a substantial claim. May combine two claims.

The KMOFAP programme confirmed the original proposals and added new practical and authentic findings.

“Whilst this has amply confirmed the original proposals, it has also added a wealth of new findings which are both practical and authentic.”

May combine two claims.

An effect size of 0.3 is equivalent to just under half a level at key stage 2, just over half a level at key stage 3, and just over half a grade at GCSE.

“This is equivalent to just under half of a level at key stage 2, just over half a level at key stage 3, and just over half a grade at GCSE.”

May combine two claims.

Exemplification helped pupils grasp standards of good work, sometimes via whole-class discussion of an example and sometimes by applying criteria to peers' work.

“Pupils came to understand what counted as good work through exemplification.Sometimes this was done through focused whole-class discussion around a particular example; at others it was achieved through pupils using criteria to assess the work of their peers.”

May combine two claims.

Changes in questioning led teachers to learn more about pupils' pre-knowledge, gaps and misconceptions, so their next moves could address learners' real needs.

“teachers learnt more about the pre-knowledge of their pupils, and about any gaps and mis-conceptions in that knowledge, so that their next moves could address the learners' real needs.”

May combine two claims.

Initial fears about how pupils might react to comments-only marking were unjustified, and neither parents nor OFSTED inspectors reacted adversely.

“Initial fears about how pupils might react turned out to be unjustified, and neither parents nor OFSTED inspectors have reacted adversely.”

May combine two claims.

Where changes in questioning have been made, pupils become more active participants and realise learning depends less on spotting the right answer and more on expressing and discussing their own understanding.

“Where such changes have been made, experience has shown that pupils become more active as participants, and come to realise that learning may depend less on their capacity to spot the right answer and more on their readiness to express and discuss their own understanding.”

Classified as probably not a substantial claim.

Project experience indicates that the generic skills developed do transfer across subjects despite subject differences.

“Despite these differences, experience has shown that the generic skills that have been developed do apply across subjects.”

• AssumptionRests on an assumption: The experiences and reports from the KMOFAP project teachers are representative enough to support general conclusions about classroom practice.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Increasing wait time can lead to more pupils being involved in question-and-answer discussions and to longer replies.

“Increasing the wait time can lead to more pupils being involved in question and answer discussions, and to an increase in the length of their replies.”

Classified as probably not a substantial claim. May combine two claims.

The most successful English activities are rich tasks enabling pupils to deepen textual understanding or scaffold writing, often through pair or small-group work that feeds whole-class discussion.

“Those that are the most successful are those rich tasks which provide pupils with an opportunity either to extend their understanding of a concept within the text or to 'scaffold' their ideas before writing.Characteristically these include small group and pair work, the reflections often being fed back into a whole class discussion.”

May combine two claims.

After their science and mathematics teachers' experience, five KMOFAP schools changed policies to permit comment-only marking, and two of these removed marks or grades from homework school-wide.

“Five of the schools in the KMOFAP project have, following the experience of their science and mathematics teachers, modified their policies to allow comment only marking; for two of these the modification was that no marks or grades be given on homework throughout the school.”

Classified as probably not a substantial claim. May combine two claims.

The findings are based on classroom observations and records, meeting records, teacher interviews and writing, and pupil group discussions.

“The findings presented here are based on the observations and records of visits to classrooms by the King's team, records of meetings of the whole group of teachers, interviews with and writing by the teachers themselves, and a few discussions with pupil groups.”

• AssumptionRests on an assumption: Teachers' self-reports, written pieces, and project observations reliably reflect classroom practice and its effects.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The project's general findings will be of value to all teachers, though some details may differ by age group and subject.

“All of this makes us confident that our general findings will be of value to all, although some important details may differ between different age groups and different subjects.”

• AssumptionRests on an assumption: The contexts of the KMOFAP project are sufficiently similar to other subjects, phases, and schools for its general findings to apply to all.

Probably not a substantial claim: may not pass S2 (checkable).

There is room for improvement in formative assessment.

“Here again, the published evidence gave a clear and positive answer, presenting a detailed picture which identified three main problems.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Comments only become useful feedback if pupils use them to guide further work.

“There is, however, more involved because comments only become useful feedback if pupils use them to guide further work, so new procedures are needed:”

Probably not a substantial claim: may not pass S4 (based).

Linking peer- and self-assessment can help develop the skills and detachment needed for effective self-assessment.

“This linkage can help in the development of the skills and the detachment needed for effective self-assessment.”

May combine two claims.

Peer work is valuable because the interchange uses language pupils would naturally use and because pupils learn by taking the roles of teachers and examiners of others.

“Peer work is also valuable because the interchange will be in language that pupils themselves would naturally use, and because pupils learn by taking the roles of teachers and examiners of others (Sadler, 1998):”

• cites Sadler, 1998 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Higher-quality tasks are associated with higher-quality teacher interventions.

“The better the quality of the task the better the quality of the interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The aftermath of tests can be an occasion for formative work.

“The aftermath of tests can also be an occasion for formative work.”

Classified as probably not a substantial claim.

Good feedback can strengthen pupils' learning skills and strategies.

“At the same time it can enhance pupils' skills and strategies for effective learning.”

May combine two claims. The quote only partly states this claim.

Generic formative assessment strategies can only go so far; subject-specific pedagogical content knowledge is needed.

“However, after a while it became clear that these generic strategies could go only so far.”

Classified as probably not a substantial claim.

Pupils' reflection about their understanding can inform future teaching, indicating where more time needs to be spent on some topics and where it can be saved on others.

“Pupils' reflection about their understanding can also be used to inform future teachingtheir feedback can indicate where more time needs to be spent on some topics and where it can be saved on others.”

May combine two claims.

Giving marks or grades has a negative effect because pupils ignore comments when marks are also given.

“Research experiments have established that, whilst pupils' learning can be advanced by feedback through comments, the giving of marks-or grades-has a negative effect in that pupils ignore comments when marks are also given (Butler, 1988).”

• cites Butler, 1988 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Providing comments to pupils helps parents focus on learning issues rather than trying to interpret a mark or grade.

“Indeed, the provision of comments to pupils helps parents to focus on the learning issues rather than on trying to interpret a mark or grade.”

Classified as probably not a substantial claim. May combine two claims.

Peer-assessment is valuable in placing work in pupils' hands, freeing the teacher to observe, reflect, and frame helpful interventions.

“In addition to this advantage, peer-assessment is also valuable in placing the work in the hands of the pupils.The teacher can be free to observe and reflect on what is happening and to frame helpful interventions:”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When pupils do not understand an explanation, they are likely to interrupt a fellow pupil when they would not interrupt a teacher.

“The last sentence of this quotation brings out an important point-when pupils do not understand an explanation, they are likely to interrupt a fellow pupil when they would not interrupt a teacher.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Learning tasks range along a spectrum from closed tasks with one defined outcome to open tasks accepting many outcomes.

“Differences between learning tasks can be understood in terms of a spectrum.At one end are 'closed' tasks with a single well defined outcome, at the other are 'open' tasks with a wide range of acceptable outcomes.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Peer and self-assessment make unique contributions to developing pupils' learning; they secure aims that cannot be achieved in any other way.

“The main point here is that peer and self-assessment make unique contributions to the development of pupils' learning-they secure aims that cannot be achieved in any other way.”

Classified as probably not a substantial claim.

The practices of self- and peer-assessment can be applied to help pupils prepare for examinations.

“The practices of self and peer-assessment can be applied to help pupils prepare for examinations, for example in tackling the following problem:”

May combine two claims.

Good questioning depends on detailed subject knowledge of a pedagogical kind, not advanced academic study; it requires grasp of fundamental principles, awareness of pupil difficulties, and creative question design.

“Choosing a good question requires a detailed knowledge of the subject, but this is not the knowledge that is gained from advanced study in a subject.”

• Also stated in: § EMBARGOED UNTIL JULY 2 nd © Please do not copy or circulate · plain

  “A high level of subject qualification is less important than a thorough understanding of the fundamental principles of the subject, an understanding of the kinds of difficulties that pupils might have, and the creativity to think up questions which can stimulate productive thinking.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

For peer-group work to succeed, many pupils need guidance about how to behave in groups, such as listening to one another and taking turns.

“For such peer-group work to succeed, many pupils will need guidance about how to behave in groups, e.g. in listening to one another and taking turns.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Preparation of test questions calls for and develops an overview of the topic.

“Preparation of test questions calls for, and so develops, an overview of the topic:”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Open mathematics tasks can admit several good solutions, so quality criteria are harder to grasp and may need repeated movement between examples and abstract criteria.

“In more open exercises, as in investigations of the application of mathematical thinking to everyday problems, there may be a variety of good solutions; then an understanding of the criteria of quality is harder to achieve and may require an iteration in discussion between examples and the abstract criteria which they exemplify.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Traffic lights form the basis of a revision plan.

“These traffic lights then form the basis of a revision plan.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Implementing reforms to feedback can change teachers' and pupils' attitudes to written work, making assessment less competitive and summative and more a step in learning.

“Implementation of such reforms can change the attitudes of both teachers and pupils to written work: the assessment of pupils' work will be seen less as a competitive and summative judgement and more as a distinctive step in the process of learning”

Probably not a substantial claim: may not pass S4 (based).

Feedback has a formative effect only when the learner actually uses the information to improve performance.

“We had, after all, stressed that feedback functioned formatively only if the information fed back to the learner was EMBARGOED UNTIL JULY 2 nd © Please do not copy or circulate used by the learner in improving performance.”

May combine two claims.

Peer- and self-assessment is not merely error-checking; it brings normally tacit standards into the open and demands active pupil involvement.

“Engaging in peer-and self-assessment is much more than just checking for errors or weaknesses.It involves making explicit what is normally implicit, and thus requires the pupils to be active in their learning.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Sadler (1989) argued that criteria by themselves cannot guide judgements of quality or progression because too many variables are involved; applying them requires guild knowledge.

“A second important function of peer-and self-assessment was brought out by Sadler (1989), who argued that criteria alone are unhelpful in judging the quality of a piece of work or in guiding progression because there will always be too many variables.The key lies in knowing how to interpret the criteria in any particular case-which involves 'guild knowledge'.”

• cites Sadler (1989) sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The strategies described for linking formative and summative assessment can be used for tests where teachers control setting and marking, but their application may be more limited where teachers have little or no control.

“they can all be used for tests where teachers have control over the setting and the marking, but their application may be more limited for tests where the teacher has little or no control.”

May combine two claims.

Pupils who receive feedback framed as helping learning outperform those told that performance reveals intelligence and grades, with the largest gain among low attainers (Newman & Schwager, 1995).

“Pupils told that feedback ' …will help you to learn' learn more than those told that 'how you do tells us how smart you are and what grades you'll get'; the difference is greatest for low attainers (Newman & Schwager, 1995).”

• cites Newman & Schwager, 1995 sourceNot checked — no full text of the source

May combine two claims.

A broad review found feedback improved performance in 60% of studies; unhelpful feedback was merely judgement or grading without guidance for improvement (Kluger & DeNisi, 1996).

“A comprehensive review of research studies of feedback showed that feedback improved performance in 60% of them.In the cases where it was not helpful, the feedback turned out to be merely a judgment or grading with no indication of how to improve (Kluger & DeNisi, 1996).”

• cites Kluger & DeNisi, 1996 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

For learning to be effective, pupils must be helped to reorganise what they know so that new, more powerful ideas are integrated.

“The key to effective learning is to then find ways to help pupils restructure their knowledge to build in new and more powerful ideas.”

May combine two claims.

Formative assessment's general principles hold across subjects, but they appear differently in each subject.

“Thus, although the general principles of formative assessment apply across all subjects, the ways in which they manifest themselves in different subjects may differ.We have encountered such differences in making comparisons between teachers of mathematics, science, and English.”

• AssumptionRests on an assumption: The experiences and reports from the KMOFAP project teachers are representative enough to support general conclusions about classroom practice.

May combine two claims.

As pupils think of their work in terms of goals, they develop an overview and the capacity to manage and control it at a meta-cognitive level.

“Insofar as they do so they begin to develop an overview of that work so that it becomes possible for them to manage and control it for themselves: in other words, they are developing the capacity to work at a meta-cognitive level.”

May combine two claims.

Mark-based feedback tends to make pupils compare themselves with others, whereas comment-only feedback is seen as improvement-oriented; comment-only pupils perform better (Butler, 1987).

“Those given feedback as marks are likely to see it as a way to compare themselves with others (ego-involvement), those given only comments see it as helping them to improve (task-involvement): the latter group out-perform the former (Butler, 1987).”

• cites Butler, 1987 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

No one can learn on a pupil's behalf; the pupil must do the learning, even when test and examination scores create pressure.

“Put simply, it became clear that, no matter what the pressure to achieve good test and examination scores, learning cannot be done for the pupil; it has to be done by the pupil.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

To be effective, feedback should cause thinking to take place.

“The central point here is that, to be effective, feedback should cause thinking to take place.”

Classified as probably not a substantial claim.

Self-assessment will only happen if teachers help pupils, particularly low-attainers, to develop the skill.

“However, self-assessment will only happen if teachers help their pupils, particularly the low-attainers, to develop the skill.”

May combine two claims.

Under competition, low attainers blame ability and high attainers credit effort; under task orientation, all credit effort and learning improves, especially for low attainers (Craven et al. 1991).

“In a competitive system, low attainers attribute their performance to lack of 'ability', high attainers to their effort; in a task oriented system, all attribute to effort, and learning is improved, particularly amongst low attainers (Craven et al. 1991).”

• cites Craven et al. 1991 sourceNot checked — no full text of the source

May combine two claims.

Assessing pupils' work is how teachers build guild knowledge, enabling them to distinguish grades and grasp progression.

“Teachers acquire this through assessing pupils' work and it is this process which allows them to differentiate between grades and gain a sense of how progression is achieved.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All subjects need guidance for both closed and open learning tasks, though in different proportions.

“Thus, in varying measure, the guidance needed for these two types of learning work will be needed in all subjects.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Formative and summative assessment cannot realistically be kept completely separate by teachers and pupils; the challenge is to create a positive relationship between them.

“However, it is unrealistic to expect teachers and pupils' to practice such separation, so the challenge is to achieve a more positive relationship between the two.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

A numerical mark does not tell pupils how to improve their work, so an opportunity to enhance learning is lost.

“A numerical mark does not tell pupil how to improve their work, so an opportunity to enhance their learning has been lost.”

Classified as probably not a substantial claim.

Asking pupils to traffic light key words or topics stimulates reflection on where learning is secure and where effort is needed.

“The point of this is to stimulate the pupils to reflect on where they feel their learning is secure, which they mark green, and where they need to concentrate their efforts, in amber and red.”

May combine two claims.

Reward or grade feedback can direct pupils toward ability rather than effort, harming low attainers' self-esteem and fostering learned helplessness (Dweck 1986).

“It can focus pupils' attention on their 'ability' rather than on the importance of effort, damaging the self-esteem of low attainers and leading to problems of 'learned helplessness' (Dweck 1986).”

• cites Dweck 1986 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Peer- and self-assessment are intimately linked.

“Again peer-and self-assessment are intimately linked.”

Classified as probably not a substantial claim.

When pupils take an active part in testing, they may come to view testing as helping their learning rather than victimising them.

“By active involvement in the test process, pupils can see that they can be beneficiaries rather than victims of testing, because tests can help them improve their learning.”

Probably not a substantial claim: may not pass S2 (checkable).

Teachers need models of how pupils learn in order to provide feedback that they can know in advance will be useful.

“what the teachers needed was to be able to give their pupils feedback that they knew in advance was going to be useful.To do that they needed to build up models of how pupils learn.”

Classified as probably not a substantial claim. May combine two claims.

These activities let pupils build understanding through talk and let teachers give in-lesson feedback by questioning and guiding.

“Such activities not only provide pupils with a chance to develop their understanding through talk, they also provide the teacher with the opportunity to give feedback during the course of a lesson through further questioning and guidance.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Peer marking of test papers can be helpful, particularly if pupils first formulate a mark scheme, which focuses attention on quality criteria.

“Peer marking of test papers can be helpful, as with normal written work, and is particularly useful if pupils are required first to formulate a mark scheme, an exercise which focuses attention on criteria of quality relevant to their productions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many who have tried to develop self-assessment skills found the first and most difficult task is getting pupils to think of their work in terms of a set of goals.

“Many who have tried to develop self-assessment skills have found that the first and most difficult task is to get pupils to think of their work in terms of a set of goals.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Learning engages the whole person, not only cognitive processes.

“Learning is not just a cognitive exercise: it involves the whole person.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

English tasks tend toward the open end, though some components are closed; mathematics tasks are usually more closed, except applications that call for open-ended evaluation.

“Tasks in English are mainly at the open end, e.g. the writing of a poem, but there are closed components e.g. for grammatical or genre conventions.Tasks in (say) mathematics are more often closed, but applications of mathematics to everyday problems can require open-ended evaluations.”

May combine two claims.

This kind of feedback can improve learning both by prompting effort and by sustaining the motivation to make that effort.

“Such feedback can enhance learning, both directly through the effort that can ensue, and indirectly by supporting the motivation to invest such effort.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Peer- and self-assessment are long-established in English and are central to the subject's aim of developing critical judgement, given its open-ended tasks.

“In English, peer-and self-assessment have a long history.It follows from the nature of the subject and the open outcome of many of the tasks characteristically set, that they are EMBARGOED UNTIL JULY 2 nd © Please do not copy or circulate central to one of its overall aims-which is to enhance the critical judgement of the pupils.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

English teachers must structure tasks enough to support learning while avoiding over-definition that restricts thought.

“Again this type of work is not uncommon in English, the skill being to make the task sufficiently structured to scaffold learning but not so tightly defined as to limit thinking.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Developing self-assessment skill can take time and practice.

“This can take time and practice:”

May combine two claims.

School policies can limit formative assessment; a policy requiring marks or grades on all homework, for instance, blocks serious comment-only marking.

“Policies can actually, or by interpretation, constrain use of formative assessment.A notable example would be a policy that, by demanding that a mark or grade be given on every piece of homework, prevents the serious use of comments.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The desired learning environment demands a classroom culture that can feel strange and unsettling to teachers and pupils alike.

“A second principle is that the learning environment envisaged requires a classroom culture that may well be unfamiliar and disconcerting for both teachers and pupils.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Frequent-review target-setting systems would discourage teaching changes that trade short-term progress for longer-term learning gains.

“Another example would be that a target setting system that required very frequent review would inhibit any change in learning methods which might slow down immediate 'progress' in order to produce medium to long term gains in learning skills.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Coherent planning of support, evaluation and dissemination for assessment-for-learning innovation requires a school leadership team member to hold strategic oversight.

“It follows that support, evaluation, and subsequent dissemination of innovation in assessment for learning will only be planned in a coherent way if responsibility for strategic oversight of the development is assigned to a member of the school leadership team.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Experience indicates that relying on a few enthusiasts is unfair to them, while mandating immediate role and style changes for all staff is unreasonable.

“Our experience supports the view that to realise the promise of formative assessment by leaving a few keen individuals to get on with it would be unfair to them, whilst to do it by a policy requiring all staff immediately to change their personal roles and styles in their classrooms would be absurd.”