OCMT is a mobile learning platform that is used to identify the weaknesses in students’ numeracy skills. Upon completing the diagnostic assessment, students access rich, interactive modules that upgrade their skills before taking a summative assessment to determine their level of mastery.
OCMT was developed by Vretta through collaborative efforts between content and research experts from all 24 colleges and schools in Ontario in partnership with the Ministry of Education and the Ministry of Advanced Education and Skills Development. To view the information brochure, click here.
Through the research conducted by the College Student Achievement Project (CSAP), a custom framework was developed as the foundation for both the assessment items and the remedial modules. The framework comprises of two equally important dimensions which form the basis for test item validity analysis:
Content Topics and Sub-topics
To ensure that the Assessment for Learning program is custom-tailored for the specific requirement of students in Grade 9 Applied math courses, nine mathematical topics and their sub-topics are identified to form the content dimension of the assessment framework.
The items and remedial modules were developed corresponding to the broader goals of mathematics teaching and learning: (A) Knowing (B) Applying (C) Reasoning.
The framework was reviewed by an advisory committee made up of college mathematics faculty and secondary school mathematics consultants.
Read more on the Assessment Framework here (page 10 to 14).
The framework formed the basis for classifying assessment items which had been submitted by college faculty, eliminating duplicates, enhancing quality, and the filling of blank cells in the matrix through creating new items as required.
Both constructed response (written) and selected response (multiple choice) items are included. The former provides a more reliable estimate of student ability than the latter because guessing is eliminated and the use of a computerized format enables a range of acceptable responses to be established for each item.
The item database was fully translated/adapted into French from its original English format.
Variants of many items were developed by Vretta’s proprietary algorithm and then selected on the basis of realism and ease of computation and (after field testing) their psychometric properties.
The items were then pre-tested in secondary schools, in which every item was responded to by between 20 and 100 students and the results given detailed psychometric analysis using both classical and IRT (item response theory) methods. At the same time, each item was reviewed both by the teachers whose classes were used for pre-testing, by the CSAP advisory committee, by officials at EQAO and at the Ministry of Education.
Following these analyses, each item was either eliminated, modified or retained unchanged. This item database was then used as the basis for field trials of the overall assessments.
It is critically important to note that the OCMT is more than just the test item and remedial module database itself. Of equal importance is the test design whereby the items are combined into individual tests. This design ensures that all tests are:
- unique for every test taker;
- equally representative of both content and performance expectation dimensions of the assessment framework;
- long enough (i.e. a sufficient number of test items) to enable reliable estimates of test takers’ abilities but short enough to complete in a reasonable length of time.
Read more on the test design here (page 19 and 20).
The interactive, voice-enabled remedial modules have been customized and aligned to the learning objectives of the framework. They provide students with the following learning experiences:
Topics are broken down to micro-steps for students to visualize and conceptualize and engage with mathematics.
Various types of interactive practice scenarios with feedback mechanisms help students master concepts.
Students can progress through the modules and master concepts at their own pace.
Provides teachers with the ability to use the modules in class, or assign them as homework, to enrich the learning experience.
Student data from the use of the OCMT is transformed into Dashboards from which teachers can monitor students’ progress within the OCMT, their growth in terms of the completed remediation lessons, and their recent activity on the OCMT.