A Time to Help

Just another WordPress.com site

Afterword: Learning-Centered Focus to Assessment Practices August 31, 2013

Filed under: Uncategorized — Anita DeCianni-Brown @ 2:25 pm

“Many teachers commonly use assessment as the starting point of their teaching activities because they believe that assessment drives learning and teaching activities.”  Using these practices can potentially detract from promoting effective, efficient and engaging learning.  (p. 300)  Referencing the K-12 learning environment in New York State public schools, it is often said that teachers, “teach to the test.”  The learning that happens is very focused on core curriculum standards and this is what they must teach.  Students learn in somewhat of a boxed environment that this is the information that they must know, and how it applies to this situation. 

 

In a traditional college setting, there is a little more openness in instruction.  Yet, there is assessments still drive learning and teaching activities.  In learner-centered environments, such as with Empire State College, instructors act as facilitators of the course, and learners explore the content and find ways to apply that to the world around them.  When attention is paid to the design of the learning experience, the benefits to the student are much greater.  “Learning is a personal and also a social process.”  (p. 301) There is more of a freedom in learning.  Only the learning experiences of learners is able to be designed, not the learning itself.  With a carefully designed course, that engages students, learners have the ability to learn not only from the instruction set forth, but from each other through engagement and interaction with each other. This has been one of the greatest strengths and benefits I have found in the adult learning and online courses. 

 

 

Naidu, S.  Afterword:  Learning-Centered Focus to Assessment Practices.  (2009).  In Lajbcygier, P., & Spratt C., E-learning technologies and evidence-based assessment approaches. Hershey: Information Science Reference

Advertisements
 

Identifying latent classes and differential item functioning in a cohort of e-learning students August 30, 2013

Filed under: Uncategorized — Anita DeCianni-Brown @ 1:55 pm

Sandford, Lajbcygier and Spratt describe a blended learning environment that:

  • Integrated a combination of traditional with web-based approaches
  • Combined media and tools employed in an e-learning environment
  • Combined a number of pedagogic approaches (irrespective of learning technology)

This chapter seemed to most reflect a K-12 learning environment with me. In the analysis of the grading specific evaluation is given to particular questions.  Questions reflected particular learning objectives.  Students needed to have previous knowledge of a information, formula, and be able to apply that to solving the problem.  For example, one question tested a student’s knowledge of binominal option pricing model.  In order for a student to answer it correctly, they needed to know the difference between European and American call and put options, be able to read an Excel spreadsheet set up of binominal model, understand how to construct the terminal prices of the underlying equity asset and apply the correct payoff equation for the specific option.  (p. 203)  Obviously, this goes further than just being able to apply an equation and come up with an answer.  They need to be able to pool from their knowledge of other factors and use that to determine what is needed in order to come up with the correct answer.

Having identified a potential source of DIF allows consideration of corrective responses to existing teaching and assessment.  Just identifying the possibility of a priori knowledge bias makes one more cognizant of avoiding such bias when developing new examination items.  (p. 206)  The reliance on one’s prior knowledge to answer things correctly seems to be an area that can affect learning for international students.  With this in mind, regarding strategies for assessment:

  • It is important that assessment be restricted to the content covered within any one unit.
  • Allow students the option to select items in an assessment.  Optional items could be included that allow students to exercise their own discretion as to the assessment task, and to identify that best suit their educational backgrounds and strengths.

In their conclusion, they state that through the identification of items and students for which biases are more frequent, teaching can be better informed and corrective actions, where necessary, can be effectively directed.  This is also true for students with learning disabilities.  Being able to include other forms of instruction other than lecture or writing on the board, but rather including hands-on learning or other visual aids, can help with the learning process.

 

Lajbcygier, P., Sanford, A., and Spratt, C. (2009). Identifying latent classes and differential item functioning in a cohort of e-learning students. In Lajbcygier, P., & Spratt C., E-learning technologies and evidence-based assessment approaches. Hershey: Information Science Reference

 

Issues in Peer Assessment and E-Learning

Filed under: Uncategorized — Anita DeCianni-Brown @ 10:13 am
Tags: , ,

Learners can become involved in the assessment of group work, with a set of established guidelines.  By determining this in advance, it can address some of the concerns that may arise during the course regarding fair assessment of individual contribution.  In order for this to work effectively, students do need to be prepared for how to assess work.  In doing so, Benson states that, there is a value in peer assessment that helps prepare students for the kinds of highly contextualized learning that are a part of work and life.

The functionalities of e-learning tools and environments are divided into pre-Web 2.0 technologies, Web 2.0 technologies and “other” tools.  The Web 2.0 technologies cover a wide variety of platforms such as wikis, blogs, social bookmarking, social networking services and access to virtual worlds.  Each of these enable an unprecedented online activity between users and between users and content.  In using these platforms, students can share their information, ideas, projects and data with not just an immediate classroom of their peers, but with a wider, broader audience.   The “other tools” include standalone electronic options such as software programs (Word, Excel, Acrobat) to audio and video files.

Opportunities for peer assessment

Opportunities of e-learning for peer assessment offer improved communication between learners.  The benefits of the improved communication include:

  • Quality of what is communicated (forms of assessment responses available in a recorded form)
  • Speed of communication
  • Flexibility of communication

Most students who undertake assessment tasks in e-learning are asked to use are asked to:

  • Submit assessment items (essays, reports, reviews)
  • Automated assessment (could include quizzes where students get automatic feedback)
  • Assessable online discussion
  • Web publishing of assessment items

One of the options discussed in peer assessment is the use of badges in education.  Based on an established criteria, students can award their peers with badges based on their interpretation of levels of mastery completed or participation/contribution.  In using peer assessment, particularly in group settings, there may be more of a collaborative environment, and less opportunity for individuals to sit back and watch others complete the work.

Tucker, R., Fermelis, J., Palmer, S.  Designing, Implementing and Evaluating a Self-and-Peer Assessment Tool for E-Learning Environments.  (2009).  In Lajbcygier, P. & Spratt, Co.  E-learning technologies and evidence-based assessment approaches.  Hershey:  Information Science Reference.

 

Ensuring Security and Integrity of Data for Online Assessment

Filed under: Uncategorized — Anita DeCianni-Brown @ 9:37 am
Tags: , ,

Armatas and Colbert cite Weippl (2007) that dependability is the key requirement for e-assessment, which includes availability, reliability, safety, integrity and maintainability. (p. 98)  The key requirements can face challenges as well.  Technology can be an important piece in assessment, which is broken down into 4 key roles:

  • Identification – authentication (provide proof that your are entitled to take the assessment); verification (that you are the person you say you are)
  • Integrity of data – assessment responses are transmitted properly and data is protected from being altered.
  • Access control – technology is used and involves encryption, separating session and version control.
  • Logging and Auditing Data – academics use a range of purposes related to assessment and teaching and learning.

Mechanisms that can help control integrity of online exams include:

  • Use of random questions offered from a question bank – not all students will be given the same questions on the same test, however, all of the questions pertain to the information covered in the lesson.  Even when students are given multiple attempts at tests, the question bank is utilized.
  • Timed tests – even though some instructors allow for an open book test, using a timed test may alleviate cheating in that students do have to know the material in order to complete the exam.  If they use their books, or search the internet, they are still only given a particular time frame, which can minimize cheating.
  • Validation of user identity including user name/password, biometrics and hardware tokens – although students may be required to use a user name and password and security tokens, in an online environment there is still no complete certainty that the person taking the test is who they are supposed to be.  Biometric technology uses fingerprints (physical) or voice (behaviorial) to verify the user.
  • Smartcards – activating a smartcards require that the user must have both the smartcard and the PIN to access the assessment (but does not guarantee user identity).
  • Web cameras – have been suggested as a use in assessment to verify who the user is.

Issues associated with authentication and verification of students, data integrity and access control and monitoring were important in shaping how online assessment was incorporated at Deakin University.  They used a learning management system (LMS) that delivered the assessment.  Students were able to access it via their user name and password.  It allowed a convenience for students in being able to upload their information and the use of the LMS allowed for the development of review tests.

There is still much to be done in order to keep the integrity of online assessments/tests.  However, the integrity issues also exists in a traditional classroom.  With the use of cellphones, as well as other more “primitive” means (writing on one’s hand, keeping a cheat/crib sheet in a hidden location), paying someone else to write a paper – cheating exists outside of just the online environment.

Reference

Armatas, C. and Colbert, B. (2009). Ensuring security and integrity of data for online assessment. E-Learning Technologies and Evidence-Based Assessment Approaches. Information Science Reference: New York.

 

Introducing Integrated E-Portfolio Across Courses in a Postgraduate Program in Distance and Online Education

Filed under: Uncategorized — Anita DeCianni-Brown @ 2:40 am
Tags: ,

Electronic portfolios have been becoming increasingly popular in the job market and career search.  They focus on “growth and development over time” and provide individuals the opportunity to show evidence of how they are accomplishing meeting standards over the course, or throughout their education.  It’s a collection of work in media formats such as PowerPoint, hyperlinked text, websites, galleries and PDF’s of one’s work.  “Integrating many varied, published definitions, e-portfolio can be understood as a collection of purposefully organized artifacts that support backward and forward reflection to augment and assess growth over time.”  (p. 245)

Bhattacharya states that early e-portfolio practice has been:

  • Individuals rather than the institutions have ownership and control over the e-portfolio.
  • E-portfolios should support reflective practices need for life-long learning.
  • Assessment against curricular rubrics is essential in establishing meaning in e-portfolio artifacts.

There are schools, such as ESC, that claim that the student’s portfolio is the property of the college when it is submitted for academic review.  Bhattacharya also includes several other possibilities, but the one that seems to resonate the most is the portfolio as a reflective practice.  I will using blogging as an example.  When I first began at ESC, the first course I took, Digital Storytelling, introduced me to writing a blog.  Over time, I added other areas, such as photography work from courses and design work to the blogs.  This showed my own personal growth.  Many of the written blogs include some reflection or my connection to material.  I can refer back to them, re-read either what I wrote about a chapter, or topic, and have a refresher and how I was able to connect with some other aspect.

There are three types of E-portfolios:

Showcase E-Portfolio:  Organization occurs after the work has been created.  This would be the examples of digital and visual work.

Structured E-Portfolio:  A predefined organization exists for work that is yet to be created.

Learning E-Portfolio:  Organization of the work evolves as the work is created.  I would say that the blogs I have written during my graduate school work, would be an example of learning e-portfolios.  As I have moved through the courses, and worked on/completed tasks, the portfolio is built.  Last semester I took a course that was to blog about a topic and develop it.  In my case, I selected the need for a particular type of course, I wrote my own rationale for why I felt it would be necessary through to a lesson plan outline.

Bhattacharya, M.  Introducing Integratd E-Portfolio Across Courses in a Postgraduate Program in Distance and Online Education.  (2009)  (p. 243 – 253)

 

Designing, Implementing and Evaluating a Self-and-Peer Assessment Tool for E-Learning Environments August 29, 2013

Filed under: Uncategorized — Anita DeCianni-Brown @ 1:20 pm

Student Self and Peer Assessment (SAPA) is proposed as a solution to motivation, process and assessment issues in team-based assessment.  (p. 172)  Students may contribute to the feedback of group members in the form of qualitative and/or quantitative; formative and/or summative; formal and/or informal and period or one-off.  Those advocating feel that this increases student engagement.  I tend to agree.  If everyone is being assessed based on the work they do within the group, they may be less apt to be a non-contributor.  When the group grade is given to everyone regardless of contribution, one may have the mindset of, “it will get done.”  “The others will do it.”  But when one’s group grade can be affected by their non-contribution, they may be more motivated to be a contributor.

There also is concern with peer assessments in that are individuals given a fair assessment.  One may provide feedback that is too critical or they may avoid honest criticism.  This does not give a fair reflection of the work.  When done properly, if a rubric is created by the instructor indicating what the assessment categories are and how individuals should be assessed, this can lead to fairer assessment.  For example, “To receive an Excellent rating, students must have completed and shown … “

One of the biggest challenges being changes in assessment practices toward more open, performance-oriented types of assessment showing competence in professionally relevant outcomes/qualifications.  One specific problem is aligning summative and formative assessment practices.  They discuss the need for more professional development of instructors.  There is reference in the article that several studies show that teachers do not hold assessment conceptions that fit new approaches to assessment, and that teacher assessment conceptions should change.

Based on the study of assessment problems in the following findings and recommendations were presented:

  • If teamwork assignments are to reflect the type of willing and productive collaboration demanded by professional practice, then the completed assignment can only be assessed as a product of that collaboration, whereas the assessment of an individual’s contribution to the project must focus rather on the process of arriving at that product.  This in particular, is a valid argument for the use of peer assessment.  When working in a team environment, unless the instructor is overseeing every aspect of the project, how do they know the contribution of the individual members.
  • Students find peer assessment to be more manageable and a more accurate reflection of individual contribution when it is continuous throughout a teamwork assignment rather than one-off.  Again, this is a valid argument of the use of continual assessment and peer assessment.  Learners can have a sense of how they are progress and contributing if they are evaluated as they go along in the work.
  • Quality of teamwork as measured in grades increases in problem and project based learning assignments when continuous peer assessment is used to assess individual contributions, such as teacher-only assessment.  This seems to be the most motivational of the assessments.  When one knows their contribution to the team can affect their overall group grade, they will most likely, be more of a contributing member to the team.
  • Students greatly prefer the individualized assessment of their assignment contributions based on online SAPA rather than all team members being allocated the same mark.  Again, as the previous one suggests, learners may be more motivated to contribute.  By the same token, those who normally would do the work (or the bulk of the work) feel there is more of a fairness in the grading.

Gulikers, J., Biemans, H., Wesselink, R. and van der Wel, M.  Aligning formative and summative assessments:  A collaborative action research challenging teacher conceptions. (2013)  Studies in Educational Evaluation.  39.  116 – 124.

 

Designing, Implementing and Evaluating a Self-and-Peer Assessment Tool for E-Learning Environments August 8, 2013

Filed under: Uncategorized — Anita DeCianni-Brown @ 7:56 pm
Tags: , , ,

Teamwork has many advantages and can have a positive affect on the learning environment.  The affects can include development of social behavioral skills, promotion of inclusive participation, development of critical thinking, motivating students from a passive to a more active learner, and/or students can learn from their peers.  Teamwork is one of the skills most sought after by employers.  How well will someone contribute to the job and how well will they fit in with the overall team to create success for the business.  Building teamwork in a classroom can be more beneficial for learners as it is cited as being more representative of real world work. 

 

By the same token, with the advantages of teamwork also comes challenges, primarily being weaker individuals who do not participate and contribute to the group, yet can reap the rewards of the group’s success.  One of the possibilities to address this is by introducing student Self and Peer Assessment (SAPA).  SAPA can motivate learners in that they are being assessed individually as well as being part of the overall group.  Advocates of peer assessment feel that when students know they are being assessed on their contribution to the project, there will be more motivation to contribute and can help in improving one’s own performance.   Assessments can be:

 

  • Qualitative and/or Quantitative
  • Formative and/or Summative
  • Informal and/or Formal
  • Periodic or One-off

 

In the teamwork example cited by Tucker, Fermelis and Palmer, they discuss peer feedback and assessment and individualization of the team mark if there is evidence of unequal contributions from team members.  Throughout the assignment, individuals were required to make regular ratings of their own and peers contributions.  From the description, I found this to be a fair system.  Students know in advance that their contribution to the project or team will count and that should they not contribute, their team grade, not the full team grade, will be affected. 

 

Thinking about this in relationship to online learning vs. face-to-face/traditional learning environment, I have found that while it is possible to do team projects via online learning, it is quite a challenge.  In the face-to-face environments, when I was assigned to a team project, often during the class time we were able to break out into our groups.  We were able to make arrangements either before or after a class to work on it as well.  In the online environment, with everyone checking in at different times to the course, it was more challenging.  In one project, trying to decide on the actual topic took about a full week.  Once that was decided on, and the tasks were divided, it went more smoothly.  This may have been alleviated if we tried to set aside one particular time during the week to chat at the same time.  For the assignment, we did not do individual assessments of our peers, we just needed to provide a general reflection of working as a team and then our own individual contribution.  Because of the way the project was set up, it was actually somewhat easy to tell the work involved from each person’s area.  Each person would have received fair assessments on this particular project, I feel, because it was broken up that everyone had to contribute. 

 

Tucker, R., Fermelis, J., & Palmer, S.  Designing, Implementing an Evaluating a Self-and-Peer Assessment Tool for E-Learning Environments.  In C. Spratt, & P. Lajbcygier, E-Learning Technologies and Evidence-Based Assessment Approaches (218-233). Hershey, PA: Information Science Reference.