Validity of the data, as discussed in earlier lectures, is one of the objectives of measurement. It is not only important regarding survey instruments, but it is even more vital that the results obtained with the instruments be valid. Validity exists when the data actually measure what they are supposed to measure. If they fail to, they are misleading and should not be accepted.
Determination of whether data are valid is a fundamentally important step. Once they are put into process and emerge as numbers, it is too late to question whether they are really accurate because statistics have a preciseness that connotes accuracy to their readers.
It is far too easy to credit data with accuracy rather than making a sufficient scrutiny of it and of the methods by which it was acquired. The reality of many lazy and hurried data-gathering projects should put every researcher on guard. Marketing researchers have, at times, confirmed that one of their most serious concerns is the errors in survey data submitted to them by the research agencies they employ. The alarming quotation may be substantially justified and makes validation a very important step.
The first of two stages is a thorough review of the methods and quality controls utilized in gathering data. When secondary data are involved, they may be too ancient or unimportant. When contrary conditions pertain, a user of those data may properly request the organization that published them to furnish information on the collection instruments and methods. With primary data this review is important even when the researcher’s own staff gathered the data. When outside agencies are engaged in gathering data, then such investigation becomes mandatory. Many of the best agencies, however, voluntarily furnish validating evidence along with the data.
The problems could turn up in any of the research stages in the gathering of primary data. One is sampling, as samples obtained often vary materially from the sampling plan. Indeed, when research agencies use the term ‘validation’ they tend to refer to checking whether an accurate sample was obtained, for which they routinely check some proportion of interviews. Second, in the conduct of interviews (or a questionnaire mailed in) many errors may have arisen. Researchers should study carefully the questionnaires and the interviewing instructions and procedures to detect causes of errors. A list will aid the people editing the returned forms to be alert for such errors.
