How do we assess Interaction at Pipplet?

At Pipplet, we are often asked, which of these capacities we assess and how. 

Interaction happens when humans communicate ideas and others react to them. It is essential for language use and for the world to work. So, unless you are a monk on a vow of silence or on a self-isolation with no internet access, it is likely that you interact with people every single day, probably dozens if not hundreds of times.

That is why a language test is only complete and comprehensive if it tests interaction. 

Now that we know that interaction only happens when there are two (or more) interlocutors involved, the question that arises is how to test it?

The traditional response to that is you need to have a human examiner facing a test-taker, asking questions, while the test taker reacts to or engages with the questions.. This approach usually involves a topic or prompt the test-taker must respond to. The examiner is required to ask follow up questions and evaluate the test-taker using a grading system

No matter the skill level, the point is to test a person’s ability to comprehend what the examiner says, express their ideas in a way that is understood and demonstrates their language skills.

However, it turned out that the interaction part was not what I expected it would be. My interlocutors kept complete silence when I presented my ideas on the topic, they didn’t even smile at my attempts to make a joke! Already I was feeling uncomfortable, as if I was boring them, or wasting their time. So I wrapped it up, tried to cut the discussion short for their sake, which was clearly the opposite of what the test required me to do. Should I mention that I failed the oral part?

Not only do we need to find subjects that would be interesting enough for the test-taker, they also should be adapted to their language level, and relevant to their interests and/or industry. If you have never driven a car in your life, you will not likely be able to keep a conversation about car maintenance for more than 2 minutes.

Another concern that might arise when we talk about face-to-face language evaluation is an ethical one, primarily around bias. Let’s imagine that a test-taker speaks freely about what is taking place in an image, but says something an examiner is not fond of. To push it even further, imagine that the evaluator is not fond of the way a test taker looks, talks, or acts. Imagine a transphobic person evaluating a non-binary test-taker or a person with a thick accent facing a racist evaluator. It’s hard to notice when the evaluation of language skills turns into an evaluation of the test-taker’s personality traits. Although the language evaluators are trained to deal with these kind of biases, they are never fully eliminated.

To avoid this type of situation, most direct face-to-face language tests are held with at least two language professionals present. This should ensure a better quality control, since the final grade is decided by vote, but does not always remove the bias completely. At the same time, that drives up the cost of that type of test. 

It’s called semi-direct because there is no actual interaction going on between the test-taker and the examiner. The responses are evaluated anonymously and on a deferred basis, which helps with some of the problems I mentioned earlier. 

A big advantage that this method offers is eliminating the factor of non-verbal communication. While it’s considered extremely important to master it in order to be a good communicator, from a language tester point of view, it can be both a blessing and a curse. 

Most of the time we’re not even aware of it, but we pass through a tremendous amount of information with our facial expressions, gestures and mannerisms. However, when it comes to intercultural communicative situations, they can often become an obstacle. Simply think about when you have to talk on the phone, and you need to verbalize everything to pass the right message through. This is often the moment when we all regret not having a good enough grasp in a foreign language.

So while a semi-direct language test is merely an artificial model of a human interaction, it helps to dissect the test-taker’s performance into communication strategies that they employ and their linguistic competencies, which can be assessed separately and in a more objective way.

These things considered, the semi-direct language tests offer this incredible opportunity to evaluate the test-taker’s language skills, and language skills only. Which has the potential to be a fair and cost-efficient way of assessing language levels since there is no way for the evaluator to like or dislike, encourage or discourage the test-taker. 

As you can see, assessing interaction is quite a tricky topic. And as the technologies develop, our approach changes with them. 

Do you have any questions about our language assessment? Feel free to contact us on contact@pipplet.com!