Chapter 15 - Measurement in research. The main contents of the chapter consist of the following: Essential characteristics of data, establishing data characteristics, determining objectivity, determining reliability, types of relative reliability,. | Chapter 15 Measurement in Research Essential Characteristics of Data Collected data must have certain characteristics in order for them to be considered acceptable evidence upon which to base conclusions Objectivity Reliability Validity Establish before collecting research data Objectivity The degree to which multiple scorers agree on the values of collected measures or scores rater reliability Important because data should not be dependent on who collected it Reliability Consistency of test scores Scores should reflect a true indication of one’s ability and not change over a short period of time Reliability of test scores is a necessary condition for validity Validity The degree to which interpretations of test scores or measures derived from a measuring instrument lead to correct interpretations Most important characteristic of data Enables researcher to have confidence that conclusions are based on correct information Establishing Data Characteristics Before collecting research data Documented by other researchers Pilot study – situation specific documentation Uniqueness of setting and participation characteristics Sample size does not need to be large Technique generally involves calculating a correlation coefficient Determining Objectivity Two or more scorers or raters should independently score a group of people on the test Intraclass correlation coefficient (R) is calculated as an estimate of the objectivity coefficient ANOVA technique Select appropriate computational formula for R Determining Reliability To determine reliability, each person must have at least two repeated scores, such as. . . Scores on multiple trials of a test within a day Scores from multiple administrations on different days Scores from multiple items on a test Scores on two different forms of a test Types of Relative Reliability Internal consistency reliability Consistency of scores within a day Stability reliability Consistency of scores across days Statistical Techniques . | Chapter 15 Measurement in Research Essential Characteristics of Data Collected data must have certain characteristics in order for them to be considered acceptable evidence upon which to base conclusions Objectivity Reliability Validity Establish before collecting research data Objectivity The degree to which multiple scorers agree on the values of collected measures or scores rater reliability Important because data should not be dependent on who collected it Reliability Consistency of test scores Scores should reflect a true indication of one’s ability and not change over a short period of time Reliability of test scores is a necessary condition for validity Validity The degree to which interpretations of test scores or measures derived from a measuring instrument lead to correct interpretations Most important characteristic of data Enables researcher to have confidence that conclusions are based on correct information Establishing Data Characteristics Before collecting .
