Saturday, January 31, 2009


Difference between validity and reliability? Both are social and impact each other but do NOT determine one another

Reliability is involved with the issue of whether the results are replicable or not (is it consistent) and whether the parts (if there are sub-parts to a measurement) add up to the same results as the whole -> This last part looks into the consistency of the sub-parts relation with the overall measurement.

Validity: has to do with the truth of the measurement. Does the measurement accurately account for the phenomenon it is attempting to quantify (this has to do with interpretation of the results as well). Another way to state it is: How accurately does the measurement account for the phenomenon. Does the measurement fit the task and result in an accurate account of what is being measured?

Potential Problems:
Evaluation of validity requires outside standard to be in place - objective/subjective
Evaluation of reliability requires comparison of the measure with itself - can be answered

Probability : Associated with the null hypothesis and relative to descriptive and sampling statistics. It is the area demarcated by the aforementioned parameters that are set up to determine the certitude of a research hypothesis.

Significance: Related to probability, it is a value region that demarcates the grounds upon which a researcher rejects a null hypothesis or rather a hypothesis that depends on "laws of chance." If x(probability value) falls within this region, then the research hypothesis is thought to be acceptable for reasons other than laws of chance operating.

Empirical Research: systematic research methodology that is planned, data is collected systematically, and reported data and methods with its findings.

A. observes and analyzes activities

B. Projects that are predominately quantitative rely heavily on collecting data that can be

counted and statistically analyzed.

C. Usually seeks to answer

1. What details best describe something such as a person, event, or community?

2. To what degree are two phenomena related to each other?

3. Is there a casual relationship between two phenomena?

Qualitative: (observations/logical processing)

Answers questions about process and description: ethnography, case study, descriptive

-To observe notable features/variables in situation

-No "treatment" or created environment. sample observed as is.

-documentation - written evidence

-archival records: original records - charts maps etc

-interviews/ surveys

-direct observation

-participant observation

Quantitative : (weights/measures)

-Experimental - manipulates variables /aspects/situations and attempts to relate them to one another

random sampling

-Introduction of treatment - predefined relationships of variables

-use of control group for comparing subjects to

-there are also Quasi-groups when sampling isnt random

-focus on isolated variables in a structured situation based in un-natural conditions

Problems with Quantitative : In spite of the perception of the power of numbers to persuade, some audiences distrust numbers; some audiences suspect researchers of bias; some audiences believe that results are not applicable because scientific methods lose sight of the human element; and carrying out an empirical project usually has a high cost in dollars and time.

1 comment:

  1. When we think in terms of validity and reliability, I find it interesting that the schism between the sciences and humanities also finds itself at a crossroads in research methods. You say that some of the problems with quantitative research is that numbers seem suspect to many people, that researcher bias is (I would say) almost expected, and that the human element seems to be lacking. Yet, haven’t we been sort of trained, as a society and a people, to accept what the scientists say? Haven’t their “numbers” been ruling the system for (at least) the last century? The humanists, of course, would argue that something very crucial is missing and that the numbers only work if we contextualize them—otherwise they will almost assuredly be skewed. The scientists argue that if it isn’t quantifiable it isn’t measurable, thus there is nothing we can do with or learn from the information. It may be interesting, in other words, but not valuable. Ah, yes, the bickering continues. May there someday be a truly trasndisciplinary cross-influential mindset between the sciences and humanities in the university…