Reliability and validity in e-learning assessment is an area that needs to be updated, re-evaluated and continually developed as the online learning environment continues to grow. According to Markham and Hurst, Reliability is the extent to which an assessment is stable over time and the extent to which the results from an assessment device are reproducible. (p. 2) Validity is the extent to which an assessment device is assessing what it purports to assess and the extent to which the results from an assessment device represent what the learner has achieved given the objectives and outcomes of the unit/course. (p. 3)
From a historical perspective, educational assessments have changed over time. Some testing has had to have modifications made to take into consideration validity across cultural groups. In what I would call a traditional testing assessment, multiple-choice, true/false and open answer tests, I have seen some changes, that I wonder how valid the assessment of one’s knowledge is. For example, I have seen open book tests and allowance of a “Cheat Sheet” during testing. It is true the testing is a timed test. One should know the information before taking the test, and often it has been said that the open book tests are harder. But why offer an open book test where answers can be looked up? “Cheat Sheets” are completely new to me. Professors have allowed notes to be written on a sheet of paper, with certain specifications to the size of the sheet of paper. As the semester progresses, the size of the sheet becomes smaller – yet as the students have said, so does their writing and shorthand notes to accommodate more information. The authors state that, “Knowledge validity is firstly about the extent to which the assessment being submitted is the work of the student – that this is a valid sample of the student’s knowledge.” (p. 11) I have a hard time understanding how the use of open book tests and cheat sheets provide a valid assessment of student learning.
There are several types of assessment that are used in both a traditional classroom and e-learning environment. In addition to a test, essays, projects, presentations and art work are types of assessment. The way in which each type is evaluated needs to differ. It is important to assess each work individually as opposed to comparatively.
Markham and Hurst suggest three areas for establishing reliability and validity: educational, technological and process. In their discussion on Educational Oriented changes, I agree with their suggestion of training students to become better evaluators of information they are receiving. I equate this to what many traditional colleges offer, a required course in the Freshman Year – I’ll call it a Freshman Seminar Class. Many students I have been in courses with have discussed the difficulties they have had in maneuvering the online learning environment, with finding materials, the use of the library resources, etc. At Empire State College, there are several workshops, webinars and seminars that give an overview on topics, yet it still gets met with difficulty. If this was a requirement, students may have an easier transition into an e-learning environment. The concerns that they discuss of plagiarism have been in existence long before e-learning. Pre-Internet days, I recall seeing signs and flyers on college campuses for individuals to write papers for a price. The difference now is, one doesn’t have to look as hard to find them. (p. 13)
Technology Oriented changes include applying technology to help reduce the Knowledge Validity problems. They also suggest that institutions demand additional attention be paid to quality constraints. Some of the constraints include clear deadlines, student workload constraints and return of marked assessment within a specified time frame. (p. 14)
Finally they suggest changes in Teaching/Learning Process. They discuss a structured approach to peer review. When using peer review, it is important to not only be clear with the person doing the review needs to be aware of personalized twists in thinking. In doing so, the individual will be to conduct the assessment not only on the content but also its process. (p. 15)
Markham, S., & Hurst, J. (2009). Re-Assessing Validity and Reliability in the E-Learning Environment. In C. Spratt, & P. Lajbcygier (Eds.), E-Learning Technologies and Evidence-Based Assessment Approaches. Information Science Reference. (1-19)