Four gods for Americans?
The already well-distributed finding from the Baylor survey found four gods that Americans' worship seems decidedly misleading to me.
The authors factor analyzed 29 items that revealed 2 dimensions. Factor analysis is a way of mathematically determining items that "hang together." In personality research, for example, it's been determined that descriptors such as "gregarious" and "assertive" hang together in a broad factor that's been called Extraversion. In the same way, these 29 items were factor analyzed such that 2 groups were clearly evident. Eight items hung well together and were given the name "Belief in God's Engagement" by the authors (p. 26). Six other items hung together well and were given the title "Belief in God's Anger."
In my personality example, I said that certain words descriptive of people tend to hang together to create factors. From this evidence it is posited that there actually exist 5 basic personality factors of human beings. The Lexical Hypothesis states that certain personality factors exist and that words for those characteristics have been invented to describe them. So, by factor analyzing what terms are ascribed to different individuals, one can see that certain terms tend to hang together. Bob tends to be both gregarious and assertive, Jim tends to be neither gregarious nor assertive. Only rarely is a person gregarious and not assertive, because these descriptors correlate well with each other.
Now lets take a look at the original 29 items (actually, I was only able to find 26 items that seemed to ask what God is like, so I must have missed something):
Set 1:
Even if you might not believe in God, based on your personal understanding, what do you think God is like?
1. A cosmic force in the universe
2. Removed from worldly affairs
3. Removed from my personal affairs
4. Concerned with the well-being of the world
5. Concerned with my personal well-being
6. Angered by human sin
7. Angered by my sins
8. Directly involved in worldly affairs
9. Directly involved in my affairs
10. A “He”
How well do you feel that each of the following words describe God?
11. Absolute
12. Critical
13. Distant
14. Ever-present
15. Fatherly
16. Forgiving
17. Friendly
18. Just
19. Kind
20. Kingly
21. Loving
22. Motherly
23. Punishing
24. Severe
25. Wrathful
26. Yielding
Now, since the raw data has not been released yet, I can only speculate as to which 8 items made up the God's Engagment scale and which 6 items made up the God's Wrath scale, but I bet I could guess, and I bet you could too. Remember that an item can load on a factor negatively, that is "Forgiving" could be on the God' Wrath scale, but loaded in a negative direction such that it counted in reverse - people who scored high on the Wrath scale being less likely to endorse "Forgiving" for example.
I can't see how two factors wouldn't emerge from these questions. Synonyms (and antonyms, negatively loaded) will very often load on a single factor, even if you are describing a dog rather than a god: wrathful, punishing, severe...
My problem with this part is that, if you were trying to ascertain how people conceive of God, you would want more items to cover more attributes. I see omnipotence in these items, but not omniscience for example. Where is Wise or All-Knowing. What about Sustaining? It seems to me that these items were assembled based on a hypothesis that Americans differ as to the level of Wrath and Engagement that they espouse in god. It's fine to have a hypothesis, in fact, it is necessary to ensure that you cover the domain with your items (i.e. to ensure that you have enough items about Wrath so that they can hang together meaningfully). But the way that this has been reported, you would think that this pattern emerged from a huge list of attributes. No, indeed, it appears that some people think that God is more engaged and some people think he is less engaged. Great. So long as it is clear that the test was trying to measure Engagment, not that it magically appeared.
Next, the authors put the two dimensions on a graph and divided each in half, creating 4 quadrants. They called the high-wrath, high-engagement group, "Authoritarian God"; the low-wrath, low-engagement group, "Distant God"; the high-wrath, low-engagement group, "Critical God"; and the low-wrath, high-engagement group, "Benevolent God." Now this would be just dandy if the data points tended to concentrate in these four quadrants, and perhaps they do. Cluster analysis of some sort could tell us if this is in fact true. In other words, do people tend to cluster in a zones of low wrath and high wrath with few in the middle? A quick look at a distribution table would tell us if the variable is bi-modally distributed, which would lend some credence to a "4 gods" argument, though it still only covers those 2 dimensions (add omniscience and perhaps we would be told that we live in an 8 god America). If, on the other hand, the data is more normally distributed, then a median split probably isn't appropriate (i.e. every body above 10 gets classified as belonging to a Wrathful God category while people who score below 10 we will call the non-Wrathful category even if most people score between 8 and 12 and very few people score a 2 or a 18).
Now, there certainly were interesting things related to people's membership in each of these four quadrants and the study was very well performed, but it seems to have been reported on poorly. Indeed, the study was more interested in how belonging to one of these categories affected other beliefs and behaviors, such as Church attendance, prayer, attitudes toward abortion, and political opinions. The study does make a strong case that where one lies on the continuum of belief regarding God's morality and God's activity does indeed affect ones behavior.
I have a great deal more to say about the initial results, so stay tuned.
The Codebook for the Survey from the Association of Religion Data Archives
The Baylor Institute for Studies of Religion
American Piety in the 21st Century: Selected Findings from the Baylor Religion Survey - this is the 74 page initial report on the findings - PDF
The authors factor analyzed 29 items that revealed 2 dimensions. Factor analysis is a way of mathematically determining items that "hang together." In personality research, for example, it's been determined that descriptors such as "gregarious" and "assertive" hang together in a broad factor that's been called Extraversion. In the same way, these 29 items were factor analyzed such that 2 groups were clearly evident. Eight items hung well together and were given the name "Belief in God's Engagement" by the authors (p. 26). Six other items hung together well and were given the title "Belief in God's Anger."
In my personality example, I said that certain words descriptive of people tend to hang together to create factors. From this evidence it is posited that there actually exist 5 basic personality factors of human beings. The Lexical Hypothesis states that certain personality factors exist and that words for those characteristics have been invented to describe them. So, by factor analyzing what terms are ascribed to different individuals, one can see that certain terms tend to hang together. Bob tends to be both gregarious and assertive, Jim tends to be neither gregarious nor assertive. Only rarely is a person gregarious and not assertive, because these descriptors correlate well with each other.
Now lets take a look at the original 29 items (actually, I was only able to find 26 items that seemed to ask what God is like, so I must have missed something):
Set 1:
Even if you might not believe in God, based on your personal understanding, what do you think God is like?
1. A cosmic force in the universe
2. Removed from worldly affairs
3. Removed from my personal affairs
4. Concerned with the well-being of the world
5. Concerned with my personal well-being
6. Angered by human sin
7. Angered by my sins
8. Directly involved in worldly affairs
9. Directly involved in my affairs
10. A “He”
How well do you feel that each of the following words describe God?
11. Absolute
12. Critical
13. Distant
14. Ever-present
15. Fatherly
16. Forgiving
17. Friendly
18. Just
19. Kind
20. Kingly
21. Loving
22. Motherly
23. Punishing
24. Severe
25. Wrathful
26. Yielding
Now, since the raw data has not been released yet, I can only speculate as to which 8 items made up the God's Engagment scale and which 6 items made up the God's Wrath scale, but I bet I could guess, and I bet you could too. Remember that an item can load on a factor negatively, that is "Forgiving" could be on the God' Wrath scale, but loaded in a negative direction such that it counted in reverse - people who scored high on the Wrath scale being less likely to endorse "Forgiving" for example.
I can't see how two factors wouldn't emerge from these questions. Synonyms (and antonyms, negatively loaded) will very often load on a single factor, even if you are describing a dog rather than a god: wrathful, punishing, severe...
My problem with this part is that, if you were trying to ascertain how people conceive of God, you would want more items to cover more attributes. I see omnipotence in these items, but not omniscience for example. Where is Wise or All-Knowing. What about Sustaining? It seems to me that these items were assembled based on a hypothesis that Americans differ as to the level of Wrath and Engagement that they espouse in god. It's fine to have a hypothesis, in fact, it is necessary to ensure that you cover the domain with your items (i.e. to ensure that you have enough items about Wrath so that they can hang together meaningfully). But the way that this has been reported, you would think that this pattern emerged from a huge list of attributes. No, indeed, it appears that some people think that God is more engaged and some people think he is less engaged. Great. So long as it is clear that the test was trying to measure Engagment, not that it magically appeared.
Next, the authors put the two dimensions on a graph and divided each in half, creating 4 quadrants. They called the high-wrath, high-engagement group, "Authoritarian God"; the low-wrath, low-engagement group, "Distant God"; the high-wrath, low-engagement group, "Critical God"; and the low-wrath, high-engagement group, "Benevolent God." Now this would be just dandy if the data points tended to concentrate in these four quadrants, and perhaps they do. Cluster analysis of some sort could tell us if this is in fact true. In other words, do people tend to cluster in a zones of low wrath and high wrath with few in the middle? A quick look at a distribution table would tell us if the variable is bi-modally distributed, which would lend some credence to a "4 gods" argument, though it still only covers those 2 dimensions (add omniscience and perhaps we would be told that we live in an 8 god America). If, on the other hand, the data is more normally distributed, then a median split probably isn't appropriate (i.e. every body above 10 gets classified as belonging to a Wrathful God category while people who score below 10 we will call the non-Wrathful category even if most people score between 8 and 12 and very few people score a 2 or a 18).
Now, there certainly were interesting things related to people's membership in each of these four quadrants and the study was very well performed, but it seems to have been reported on poorly. Indeed, the study was more interested in how belonging to one of these categories affected other beliefs and behaviors, such as Church attendance, prayer, attitudes toward abortion, and political opinions. The study does make a strong case that where one lies on the continuum of belief regarding God's morality and God's activity does indeed affect ones behavior.
I have a great deal more to say about the initial results, so stay tuned.
The Codebook for the Survey from the Association of Religion Data Archives
The Baylor Institute for Studies of Religion
American Piety in the 21st Century: Selected Findings from the Baylor Religion Survey - this is the 74 page initial report on the findings - PDF
posted by Kletos Sumboulos at 7:49 PM
3 Comments:
Very well said. Have you ever thought about writing a book on statistics? You explain these concepts in a very readable manner.
I look forward to your further posts on this topic. A huge point about God's wrath emerges for me and that relates to who and what God is wrathful towards. I certainly hope He has wrath towards people like Hitler and Pol Pot and events like the Holocaust and the Khmer Rouge.
I agree with Theophilia. Thanks for the analysis -- I love it when a blogger I follow gives a real behind the scene's look at his field of expertise (at least if I can at all understand it -- and that excludes Josh S and his math).
The point about chosing 10 as a way to just divide a unipolar distribution going from 8 to 12 reminded me of a historical analysis of regional suport for state-building in the Civil War. After mapping the areas of support and opposition in the Union and the Confederacy, the historian (Franklin Bensel) stepped back and showed how the distribution in the Union was bipolar (i.e. really for or else really against) but in the Confederacy it was unipolar, with most reps being pretty much in favor of most measures.
Anyway it was nicely done, I thought.
I found your blog post in Google because I also wondered about how to interpret the "Four Gods" finding. Your point about needing more information as to how the factor analysis was done is valid. But I'm not sure I agree with your assumption that the 29 items they used were mere subitems from (in your example) just two questions. I'm guessing, actually, that they used 29 full-fledged questions from the survey, though admittedly it's not easy from looking at the survey to tell which 29 questions were considered to be about "God's character and behavior." If the researchers were indeed using 29 full-fledged questions, I feel that it's not as easy as you found it to be to psych out which eight questions they found to be associated statistically with God's "engagement" and which six with God's "anger."
I also agree that it would be nice to see a cluster analysis to find out if there really are two bimodally distributed axes here.
I note that the Wikipedia article on factor analysis contains the caveat: "Interpreting factor analysis is based on using a 'heuristic,' which is a solution that is 'convenient even if not absolutely true' ... More than one interpretation can be made of the same data factored the same way." So you could add to your point about the surveyors "trying" to measure engagement and anger from the get-go a second point about the survey team "trying" to derive these two particular factors heuristically after the fact, giving a result which may be statistically convenient but not indicative of any underlying truth.
Post a Comment
<< Home