Bolton College’s FirstPass Platform

For nearly a century analogue machines and digital computers have been used by teachers to help mediate the delivery and assessment of closed questions. Student responses to yes/no questions, multiple choice questions or drag and drop activities can be processed by computers because they observe two possible states; namely, has the student selected the correct answer to a question – yes or no?

In the Autumn of 2017, colleagues at Bolton College embarked on a tentative journey to discover if a computer could be trained to support teachers to assess student responses to an open-ended question; and if real-time feedback improved the quality of student work when responding to such a question.

We have discovered that if we make use of natural language classification, natural language understanding and other tools a computer can indeed be taught to analyse and assess responses to an open-ended question.

It is also possible to offer textual and graphical real-time feedback to students. Our work also enables teachers to create multiple classification models that can be used to support the formative assessment of numerous open-ended questions.

The FirstPass Platform

In 2022 teachers will have the opportunity to pose open-ended questions to their students. In this situation, the computer is able to process an infinite number of possible states as the student responds using free form text.

Bolton College’s FirstPass platform is the channel which allows for the formative assessment of open-ended questions. In doing so, it radically shifts how formative assessment is conducted through the medium of our networked devices.

The power of crowdsourcing

One of the key traits of the FirstPass platform is its ability to take advantage of crowdsourcing which supports the participative and collaborative model that underpins the training of the subject topic classifiers on the platform.

For instance, business teachers at Bolton College could set up a subject topic classifier that contains labelled sentences about why people chose to set up a business as a sole trader. As business teachers in other schools and colleges observe the creation of this subject topic classifier they can also start to add labelled sentences to this classifier. This accelerates the training of the subject topic classifier and furthers the accuracy of the classifier at identifying student responses to open-ended questions about the topic. Business teachers across the globe who teach their students about the different types of business ownership could also offer their support to train this particular subject topic classifier. All of a sudden, the volume of labelled sentences for this classifier starts to grow exponentially.

However, the growing number of labelled sentences for this subject topic classifier accelerates even further when student responses about this subject topic are added to the classifier. When this happens we envisage that the FirstPass platform could operate with a very high degree of accuracy when providing real-time feedback to students as they respond to the open-ended questions that have been set by their teachers. This iterative improvement continues as each academic year starts, ends and resumes again.

The student experience

A couple of years ago, a student at Bolton College stated that the feedback that was being provided by FirstPass was like having a virtual teacher sat next to him as he composed his answer. If the development of the FirstPass platform is closely aligned to the needs of students and teachers we hope that it will prove to a be a useful tool to aid the formative assessment process.

For students, the platform will offer the following assistance:

• Students will have the opportunity to review, amend and improve on their answers before submitting them to their teacher for final commentary.
• As students respond to a given question their answers can be used to provide additional labelled training data to respective classifiers. As the volume of training data increases FirstPass will become more and more accurate at assigning the correct label(s) to each sentence. The legacy that students leave for future users of the platform is welcomed.
• The transactional distance between student and teacher is narrowed by FirstPass’ ability to offer real-time feedback to each student.
• FirstPass can be described as a complex agent as it supports students and teachers with the formative assessment of open-ended questions. If FirstPass is coupled with other campus datasets and agents its ability to offer hyper-contextualised information, advice and guidance to each student will become more pronounced. With tools such as FirstPass students will increasingly engage with one or more digital agents as they engage with campus services. In this instance, the agents will be supporting the formative assessment process.

Summary

Colleagues at Bolton College would like to conclude that the computer does not displace the teacher when conducting the formative assessment of open-ended questions.

The FirstPass platform is simply a tool that extends the teacher’s ability to offer real-time feedback to his or her students as they compose their answers. Feedback to open-ended questions is incredibly nuanced and a digital machine that is supported by cognitive computing cannot do the task on its own.

Teacher feedback is incredibly important and the human touch must remain when conducting formative assessment of open-ended questions. Digital tools like FirstPass can be very useful for students and teachers; but we must recognise that computers at present can only perform a limited and narrow set of cognitive tasks; and in high stake environments such as our schools, colleges or universities the teacher must remain central when designing, delivering and managing the formative assessment of open-ended questions. As always, we look forward to sharing more about the FirstPass platform as it develops.