Hướng dẫn sử dụng phần mềm phân tích đề thi IATA (Chapter 14_Convert to PDF to Word full)

14.    CHAPTER 14 SUMMARY OF IATA WALKTHROUGHS

The walkthrough examples you have completed provide detailed explanations of the most common statistical procedures that are required to create, implement, and maintain a national assessment system. As you completed these walkthroughs, you learned how to perform the following tasks:

1. Loading response data;

2. Loading item parameter data;

3. Specifying item response keys, item performance levels, and item content classifications;

4. Specifying missing data treatment;

5. Reviewing classical item statistics;

6. Interpreting item response functions and item error analyses;

7. Interpreting factor analysis results;

8. Interpreting summary test statistics;

9. Producing scale scores for reporting;

10. Generating and interpreting differential item functioning analysis;

11. Estimating and defining thresholds for proficiency levels;

12. Selecting subsets of items for specific measurement goals; and

13. Saving results to your computer.

These tasks represent practically all of the normal requirements for test analysis in the implementation of a national assessment. However, simply replicating the examples as they appear in these chapters is not the same as mastering the ability to perform these functions with your own national assessment data. To use this material as a foundation for mastery, the next steps in your learning process should be to review each of the examples several times while following the exact instructions in the chapters.

Once you have mastered the IATA interface, you will be ready to experiment with some of the choices that structure the analysis in IATA. Once again, you should go through each of the walkthroughs, but, instead of following each of the instructions precisely, experiment with the options that are available. For example, what happens to analysis results when the number of test items is very small? What happens to equating results when the set of common linking items includes only very easy or very hard items? It should be stressed that there are many other technical approaches to analyzing test data that are beyond the scope of this volume. However, if you experiment with the IATA sample data and compare your results with those obtained earlier during the various walkthrough exercises, you should get a much broader understanding of which choices are appropriate for different situations.

When you are analysing your own national assessment data, you should first identify the workflow in the IATA menu that is most appropriate to your situation. More than likely, your data analysis situation will be very similar to one of the walkthrough examples presented in one of the previous chapters, which provide a useful reference for your own data analysis. Some situations may require combinations of different workflows, where the results from one workflow or analysis are used as the input data for another.

Ultimately, as your expertise grows, you will appreciate that there are rarely unique and perfect answers or solutions to the problems of national assessment. At best, the statistical methods that we use in modern assessment are able to minimize the influence of errors that inevitably result from the real-world challenges of educational measurement. How we choose and implement these statistical methods is dependent on the goals of the assessment: What are the needs of the stakeholders? What are the consequences of the decisions based on the results? IATA is simply a tool (albeit a useful one) for reducing the burden of these statistical methods and helping you understand the trade-offs of making analytical choices.