HAVE YOU HEARD OF KOREAN SAT?

Korean SAT is a standardized, high-stakes exam for seniors at a high school to enter the universities in Korea, which determines what colleges they can apply.

The test is held on the specific date (the 4th Thursday, Oct.) of a year and on that day, interestingly, government officers and employees of most companies are encouraged to come to work one hour late in the morning to prevent test-takers from being late due to the traffic congestion.

More details are required to understand the test better, such as the schedule, sections of the test, and the question numbers of each section, etc. The test is initially made by KICE (Korea Institute for Curriculum and Evaluation). It runs almost 10 hours and includes all subjects which test-takers have learned during the high school except Music, Art, P.E., Ethics.

Even though several possible options have been developed for getting into the university without big influences of the test result itself, test-takers are required to meet the criterion up to the certain level of achievement that university requests them to do in the test. Thus, SAT still seems to play an important role regarding the characteristic of this test as a gate-keeper.

Evaluation of Korean SAT
Five principles can be identified and applied to evaluate the test: practicality, reliability, validity, authenticity, and washback.

1. Practicality

The national standardize test needs a vast amount of time, money, and efforts. Nevertheless, SAT is apparently highly practical regarding the procedure of administrating the test and the system of scoring.
The initial stage of administration of the test starts from March, and the later scene ends in December before students apply for universities.

KICE and Education offices of all cities and provinces cooperate with each other to do the administrative tasks. KICE make and develop the question items, print them out, score them, and notice the results. Education departments deal with the applications and administer all details with the test on that day. By doing the whole tasks altogether, they can prepare better in all particulars that concern every possible option. Besides, they develop several plans for disabled students by offering the individual test paper, tools, or schedule.

English test itself, it has 45 questions (17 listening items/ 28 reading items) for 70 minutes. The listening test takes 26 minutes to do, and reading test uses the rest, 44 minutes, including marking the answers. The test consists of multiple-choice items based on test-takers' comprehensive, inferential, or syntactic thinking. That's the main reason that the test has a high level of practicality.

1. Reliability

The reliability of the test is slightly high (but, not that high), because of these reasons below:
All conditions in the test such as the quality of the copies, audio, and classroom are actively controlled. Teachers at a school where the test is held check every detail in advance, covering the whole walls with white paper, adjusting the volume of the broadcasting system, and setting up the hitter at the proper temperature.
Objective scoring procedures leave a little debate about "correctness" of an answer every year. It is one of leading issues whenever the test is opened to the public because the responses of one or two question items seem unexplainable enough as the correct ones even after the thorough process of designing question items. In fact, the only one question might determine someone's potential universities, jobs, and further, their life, so test-takers usually want to make sure why THIS ANSWER cannot be the right one for the question or vice versa. They make formal objections to the questions and answers, and then those opinions can be accepted after discussions among the experts who are from the related fields.

student-related reliability can be a problem of the test every year. Test-takers prepare for the test more than three years in high school, and even some do from elementary school. In fact, they're only required to focus on studying during the whole days, weeks, and years for the test day. No wonder that they're under the higher pressures and tensions as the day comes closer. When the day finally comes, many test-takers feel fatigue, anxiety, and extreme nervousness.

1. Validity

The test has two sections of listening and reading tasks, but it intends to assess four language abilities: listening, reading, speaking, and writing.
Listening ability to comprehend the conversations is directly assessed (12 out of 17) using a simple recall of low-order thinking skills to inferential, synthetic high-order ones. For example, the test includes the tasks such as identifying main ideas, recognizing the purpose of the conversation, distinguishing the relationship between two interlocutors, matching the pictures with conversations, factual understandings which are related to 5W1H, and complex questions
Speaking ability is indirectly assessed by completing the dialogues throughout the ‘fill in the blanks.' (5 out of 17)

Reading test assesses test-takers' interactional comprehensive reading skills, utilizing the reader's background knowledge and cues from the contexts. (22 out of 28) The questions ask about the understanding of purposes and main ideas, guessing a title, interpreting an intention and emotions of the author, analyzing a graph, table or materials, analyzing correspondences between the explanations and the passages. Also, ‘filling in the blank' item is used in the level of a word, phrase (or sentence), and a paragraph.

Writing ability is indirectly assessed how much of the information students can understand from the contexts, summarize them in sentences, and compose a paragraph. (6 out of 28) The questions mainly deal with the flow of the passage, insertion of the phrase, the order of several sections, and summarizing the passages. Grammatical features in language usage and vocabulary words are the part of the writing test.
The test tries to deal with four English abilities, yet it has relatively more attention to listening and reading sections which are receptive skills. On the other hand, speaking and writing skills have less weight on by accounting for only one quarter in the whole test. Moreover, they are assessed by indirect ways through multiple-choice items in listening and reading sections. In fact, speaking and writing abilities are performance-based skills that the outcomes have little correlation with the numerical scores of the test. It could be one reason that the content validity becomes low.

1. Authenticity

In listening section, the language use of the test is not natural enough as it is supposed to be in real world. For example, the script is recorded clearly in normal voices and regular speed, preventing test-takers from missing any details. The whole listening dialogues sound entirely artificial because they include only two exclamations to be expressed interlocuter's surprise without any hesitations (e.g., hmm, well), white noise, and interruptions which can be in the daily life.

All items are contextualized rather than isolated. For example, in listening section, test-takers listen to the conversations and can guess the entire situation as well as some details which are needed to answer the questions. In reading, test-takers read some paragraphs such as stories and episodes and can realize the thematic organizations.
Some topics can be interesting for test-takers, but most of them tend to deal with academic knowledge in depth, or pressing issues such as energy issues, natural evolution process, etc. It makes test-takers feel that the contents of the test are difficult to understand because there are many academic words that students cannot easily guess the meaning throughout the contexts.
Tasks such as notices about the festival, art exhibition are closely approximate to real ones. They have some information that people care about in real world such as fees, parking information, etc.

1. Washback

As described above, SAT is not only a large-scale assessment based on standards of the English curriculum but also a gate-keeping exam for entering the university. After finishing the test, test-takers get a single overall numerical score, which is the sum of each section's score including Korean, Math, and the other parts. In fact, the whole score is the only matter in the entire process dealing with the test results. Subsequently, the test cannot give students useful diagnoses of strengths and weaknesses. In other words, the test cannot have positive washback because the test-takers cannot receive interactive feedback on their achievement in English abilities.

Conclusion

Regarding five principles to evaluate the test, the analysis of Korean SAT shows high practicality and low validity, authenticity, and washback. To increase validity, authenticity, and washback, the Korean SAT need to develop various test items based on balanced language proficiencies and offer students proper feedback on the test. All trials would worth it even if it could decrease the practicality.

Korean SAT has systematic problems in administrating the whole process because it believes that only one-time test can determine student's proficiency. Korea government and education office continuously try to solve this problem by offering several alternatives of a gate-keeping test. It is certain that this national effort to change the problematic system leads to the positive development of the test, which can finally make satisfactory results for students.