In November I facilitated a webinar entitled, “Assessing Intercultural Learning: Beyond Assessment Tools.” Due to the high interest in and positive reception of that session, I decided to also write a blog post on the topic.
In this post, I discuss some of the challenges involved in assessing intercultural learning, and share the webinar slides, which contain practical examples of ways to address these issues.
In this post, I discuss some of the challenges involved in assessing intercultural learning, and share the webinar slides, which contain practical examples of ways to address these issues.
Formative & Summative Assessment
First of all, it’s important to distinguish between two primary types of assessment—formative and summative—and think about the role both play in intercultural learning.
Summative assessment is typically given after the instruction or learning experience is over. It provides information about what has been learned. The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark.
Formative assessment, on the other hand, is given throughout the learning process. It’s used to check understanding and determine how students are progressing toward a certain learning objective. The goal of formative assessment is to monitor student learning to provide ongoing feedback.
Formative assessment is very important in intercultural learning because of the developmental nature of such learning. It’s important to understand where our learners are developmentally and provide them with the support and/or challenge they need to continue their intercultural learning journey.
First of all, it’s important to distinguish between two primary types of assessment—formative and summative—and think about the role both play in intercultural learning.
Summative assessment is typically given after the instruction or learning experience is over. It provides information about what has been learned. The goal of summative assessment is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark.
Formative assessment, on the other hand, is given throughout the learning process. It’s used to check understanding and determine how students are progressing toward a certain learning objective. The goal of formative assessment is to monitor student learning to provide ongoing feedback.
Formative assessment is very important in intercultural learning because of the developmental nature of such learning. It’s important to understand where our learners are developmentally and provide them with the support and/or challenge they need to continue their intercultural learning journey.
Challenges in Assessing Intercultural Learning
Let’s talk about some of the challenges of assessing intercultural learning, and then explore how these might be addressed with some practical examples.
One challenge is that there is often a focus on assessing intercultural learning at a university-wide or high-level, in a summative way, yet without formative assessment to support students’ intercultural learning along the way.
Many colleges and universities these days mention intercultural learning, global competence, or similar in their mission statement. They then ask, “How do we know if students are indeed becoming more interculturally or globally competent?” The next step is often to seek out a reliable assessment tool that can be used pre- and post- to assess whether students are developing in the way the university wants. The challenge is that if you find out that students aren’t making the gains that you’d like, it’s oftentimes too late to remedy that, at least for that cohort of students.
We need more program- and course-level facilitation of intercultural learning that incorporates effective formative assessments along the way, providing students with the type of challenge and support that they need to develop in this area. That simultaneously provides us as educators with helpful information about whether students are on track to achieve what our mission statements set out to achieve.
Another challenge that I’ve encountered—at the extremely practical level—involves grading assignments that are meant to assess students’ intercultural learning and growth. Intercultural learning is developmental. We can’t give a student a C because he or she is not as interculturally developed as others. So how do you grade assessments when the focus is intercultural learning?
Let’s talk about some of the challenges of assessing intercultural learning, and then explore how these might be addressed with some practical examples.
One challenge is that there is often a focus on assessing intercultural learning at a university-wide or high-level, in a summative way, yet without formative assessment to support students’ intercultural learning along the way.
Many colleges and universities these days mention intercultural learning, global competence, or similar in their mission statement. They then ask, “How do we know if students are indeed becoming more interculturally or globally competent?” The next step is often to seek out a reliable assessment tool that can be used pre- and post- to assess whether students are developing in the way the university wants. The challenge is that if you find out that students aren’t making the gains that you’d like, it’s oftentimes too late to remedy that, at least for that cohort of students.
We need more program- and course-level facilitation of intercultural learning that incorporates effective formative assessments along the way, providing students with the type of challenge and support that they need to develop in this area. That simultaneously provides us as educators with helpful information about whether students are on track to achieve what our mission statements set out to achieve.
Another challenge that I’ve encountered—at the extremely practical level—involves grading assignments that are meant to assess students’ intercultural learning and growth. Intercultural learning is developmental. We can’t give a student a C because he or she is not as interculturally developed as others. So how do you grade assessments when the focus is intercultural learning?
Structuring Assessment into Program/Course Development
Before discussing some practical approaches to assessment, I want to emphasize the importance of using backward design so that you are thinking about the role of assessment in your program or course from the very beginning. Also known as reverse engineering, backward design simply means starting with the end in mind and working backward from there. According to backward design, you should develop your learning objectives and decide how you will assess students’ progress toward these objectives before creating the curriculum.
For more information about backward design and developing intercultural learning objectives, see the October 2017 blog post.
Before discussing some practical approaches to assessment, I want to emphasize the importance of using backward design so that you are thinking about the role of assessment in your program or course from the very beginning. Also known as reverse engineering, backward design simply means starting with the end in mind and working backward from there. According to backward design, you should develop your learning objectives and decide how you will assess students’ progress toward these objectives before creating the curriculum.
For more information about backward design and developing intercultural learning objectives, see the October 2017 blog post.
Practical Examples
The following are a few examples of assignments that can be used to assess students’ intercultural learning.
Reflective Journals
Reflective journals are a great formative assessment opportunity. Through journals, you can assess students’ understanding of intercultural concepts, theories, and practices, and their ability to make connections between these and their own personal experiences.
Intercultural learning needs to take into account where students are developmentally and meet them there. When you provide feedback to students on what they’ve written in a journal, you can individualize your response to provide the necessary challenge or support that each student needs.
I suggest providing prompts and ideas about what students might want to address in their journals, yet offering flexibility as well. I would advise against just asking them to keep a blog or turn in a reflection paper without offering any guidance.
Students also need concepts, theories, and processes that they can use to help make sense of the experience. If you’re not providing them with such frameworks to process the experience, the reflection will not be nearly as meaningful.
Grading reflective journals can be a challenge, so I suggest taking a loose approach to grading, such as a check mark or full points for successful completion unless it doesn’t meet certain criteria. The main point is to provide useful feedback that will challenge and/or support students as needed during their intercultural learning journey. I often grade reflective journal entries based on their depth of reflection and the connections students are able to make between their experience and the content.
See the slides for examples from students’ reflective journals and instructor responses.
The following are a few examples of assignments that can be used to assess students’ intercultural learning.
Reflective Journals
Reflective journals are a great formative assessment opportunity. Through journals, you can assess students’ understanding of intercultural concepts, theories, and practices, and their ability to make connections between these and their own personal experiences.
Intercultural learning needs to take into account where students are developmentally and meet them there. When you provide feedback to students on what they’ve written in a journal, you can individualize your response to provide the necessary challenge or support that each student needs.
I suggest providing prompts and ideas about what students might want to address in their journals, yet offering flexibility as well. I would advise against just asking them to keep a blog or turn in a reflection paper without offering any guidance.
Students also need concepts, theories, and processes that they can use to help make sense of the experience. If you’re not providing them with such frameworks to process the experience, the reflection will not be nearly as meaningful.
Grading reflective journals can be a challenge, so I suggest taking a loose approach to grading, such as a check mark or full points for successful completion unless it doesn’t meet certain criteria. The main point is to provide useful feedback that will challenge and/or support students as needed during their intercultural learning journey. I often grade reflective journal entries based on their depth of reflection and the connections students are able to make between their experience and the content.
See the slides for examples from students’ reflective journals and instructor responses.
Digital Storytelling Projects
In a course I designed called “Intercultural Communication and Leadership,” which is taught at CIEE study centers around the world, students create a digital storytelling project for their final.
According to the Digital Storytelling Association, “Digital Storytelling is the modern expression of the ancient art of storytelling. Throughout history, storytelling has been used to share knowledge, wisdom, and values. Stories have taken many different forms. Stories have been adapted to each successive medium that has emerged, from the circle of the campfire to the silver screen, and now the computer screen.”
A digital storytelling project typically consists of a 3- to 5-minute video. It combines mostly still images with a story and sometimes music. It’s produced by someone who is not a media professional and is typically constructed as a thought piece on a personal experience that is important to the author.
One of the great things about digital storytelling is that it provides an easy means through which students can share their learning with others if they’d like.
The slides contain further information about the CIEE assignment. Below you can also see an example of a final project.
Digital storytelling projects can be used as summative, but also formative assessment. They allow us to see how students are making sense of the course content and their ability to connect it to their own lives. This can help us evaluate their learning. But ideally the final project is part of a larger process during which you can assess students’ learning along the way and provide feedback as well.
Similar to the reflective journals, it’s important to keep in mind that grading might have to be based on factors other than the students’ demonstrated level of intercultural competence. Grading also shouldn’t be based on the technical skill demonstrated in creating the project, unless that’s related to one of the learning objectives. Instead, it’s helpful to grade based on things such as level of reflection or analysis, connections made between the concepts learned and real-life examples, etc. In addition, it’s important to provide information to the students early in the process about how they will be graded, perhaps in the form of a rubric (see example in the slides).
Here's one example of a digital storytelling project (google "ICL digital storytelling project" for more examples):
In a course I designed called “Intercultural Communication and Leadership,” which is taught at CIEE study centers around the world, students create a digital storytelling project for their final.
According to the Digital Storytelling Association, “Digital Storytelling is the modern expression of the ancient art of storytelling. Throughout history, storytelling has been used to share knowledge, wisdom, and values. Stories have taken many different forms. Stories have been adapted to each successive medium that has emerged, from the circle of the campfire to the silver screen, and now the computer screen.”
A digital storytelling project typically consists of a 3- to 5-minute video. It combines mostly still images with a story and sometimes music. It’s produced by someone who is not a media professional and is typically constructed as a thought piece on a personal experience that is important to the author.
One of the great things about digital storytelling is that it provides an easy means through which students can share their learning with others if they’d like.
The slides contain further information about the CIEE assignment. Below you can also see an example of a final project.
Digital storytelling projects can be used as summative, but also formative assessment. They allow us to see how students are making sense of the course content and their ability to connect it to their own lives. This can help us evaluate their learning. But ideally the final project is part of a larger process during which you can assess students’ learning along the way and provide feedback as well.
Similar to the reflective journals, it’s important to keep in mind that grading might have to be based on factors other than the students’ demonstrated level of intercultural competence. Grading also shouldn’t be based on the technical skill demonstrated in creating the project, unless that’s related to one of the learning objectives. Instead, it’s helpful to grade based on things such as level of reflection or analysis, connections made between the concepts learned and real-life examples, etc. In addition, it’s important to provide information to the students early in the process about how they will be graded, perhaps in the form of a rubric (see example in the slides).
Here's one example of a digital storytelling project (google "ICL digital storytelling project" for more examples):
Group Projects
The last example I want to mention involves using group projects to assess students’ intercultural learning. Many professors assign group projects, but oftentimes the focus is more on the final product than the process. To assess students’ intercultural competence, the focus should be just as much—if not more—on the process as the product.
We can help students learn to engage effectively across cultural differences by requiring them to actually do it, while providing tools, space to reflect, and support that can help them learn through the process.
To do this effectively, it’s important to assign students to diverse groups rather than letting them choose their own. If you’re thinking that your students aren’t a very diverse group, keep in mind that people don’t have to be from different countries or ethnicities to have different perspectives or approaches to group work.
It’s really important to make sure students understand the project is as much about process as product. They also need to understand why focusing on their ability to work effectively across difference is important. If this is an engineering class, for example, you could provide some examples of the countless times, in your own life as an engineer, you’ve had to work in diverse teams, and how important the team dynamic has been for your success.
It’s also critical that students have some intercultural frameworks—like concepts and processes—to help them work together more effectively. You can’t simply place them on a team and ask them to reflect on the experience, because they may very well just complain about the teammates that they feel aren’t pulling their weight. Students need to have more value-neutral frameworks to reflect on, compare, and contrast approaches, including their own.
Make sure to have students reflect on their own role in the process. What have they noticed about their own values and assumptions surrounding teamwork?
With regards to assessment, the final product can serve as a form of summative assessment. The focus on process throughout can serve as formative assessment.
You want to find ways to provide feedback on both process and product. As far as the formative aspect goes, you can have students respond to written reflection questions during the process, or have in-person meetings or check-ins. This allows you to gauge how they are engaging the process as individuals and as a team, and provide feedback along the way.
Grading should reflect both process and product. You send the wrong message if you tell students process is important, but grade only the final product. Again, you don’t really want to grade based on how interculturally competent they were during the process. Instead, it’s probably better to provide a grade based on their depth of reflection, attempt to apply what they’ve learned, etc.
Concluding Thoughts
To wrap up, I’d just like to summarize a few key points. First, I want to emphasize the importance of formative assessment in intercultural learning. Becoming interculturally competent is a developmental process, and we need to provide feedback that helps students along their journey. In doing so, try to offer both validation and support and also challenge students in developmentally appropriate ways.
Second, while we can assess students’ intercultural learning, we can’t necessarily assign grades based on a student’s level of intercultural competence because it is such a developmental process. Instead, we can grade them on whether they are doing things that will help them develop interculturally—like reflecting deeply, connecting intercultural concepts with personal experiences, listening for understanding, etc. It’s helpful to use rubrics and share these with students ahead of time so they understand how they will be graded.
To wrap up, I’d just like to summarize a few key points. First, I want to emphasize the importance of formative assessment in intercultural learning. Becoming interculturally competent is a developmental process, and we need to provide feedback that helps students along their journey. In doing so, try to offer both validation and support and also challenge students in developmentally appropriate ways.
Second, while we can assess students’ intercultural learning, we can’t necessarily assign grades based on a student’s level of intercultural competence because it is such a developmental process. Instead, we can grade them on whether they are doing things that will help them develop interculturally—like reflecting deeply, connecting intercultural concepts with personal experiences, listening for understanding, etc. It’s helpful to use rubrics and share these with students ahead of time so they understand how they will be graded.
Join the Conversation!
Enjoying the blog? You’re invited to join me and an amazing group of higher education professionals committed to fostering intercultural learning at the next Intercultural Leadership Forum! You'll have a chance to connect with others doing this work and gain new insights as you move toward your intercultural goals.
AUTHOR
