I’ve lived in Missouri (USA) for about 7 years now, mainly for career reasons. Instead of going on about myself here (my usual approach), I’ve described in my profile’s biography a bit about what I “do” and how I was drawn to skepticism and associated worldviews and movements. I suspect my story’s not unlike that of many others in this forum—though perhaps more mundane.
What prompted me to join this forum tonight was noticing the role that social science appears to have played in the recent Miami, FL case where a judge over-ruled the state’s ban on adoption by same-sex couples. This seems like a (small) legal success for science that PoIers might find interesting, so I’ll try to find an appropriate place to discuss it.
Very neat, PsyStat. Statistics is something I know significantly less about than I should, since I firmly believe it is the root of all good experimental science. I am sure many of us here would be interested in your insights, as well as in whatever you have to say about the case you mention.
Hi, Doug. Although I believe we’d all be better off with some basic statistics literacy, perhaps more important is the sort of critical thinking most scientists bring to a problem—the kind of attitude that pushes skeptics to consider alternative explanations of what they’ve observed instead of rushing to embrace an “obvious” or preferred account. One idea from social-science methodology that I find really useful is the notion of “threats to validity” (a la Campbell, Stanley, Cook, Shadish): basically, an organized list of ways we might draw faulty causal conclusions about empirical data; although developed in the context of “messy” quasi-experimental or observational research where rigorous experimental control isn’t feasible, much of it applies to reasoning about everyday experience.
At any rate, statistics and methodology are just a few useful tools for shedding light on the issues discussed here. What I know about these areas seems dwarfed by how much I don’t understand about philosophy, law, physics, biology, literature, and many other relevant disciplines. There’s a lot I don’t know, and I look forward to learning.
Probably the most thorough and authoritative reference is Chapters 2 and 3 in the following book, which is sort of a third installment following Campbell and Stanley’s (1963) and Cook and Campbell’s (1979) class texts—which I’ve admittedly never read:
Shadish, W. R., Cook T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
One of the better Web intros I’ve found is in Bill Troachim’s online Research Methods Knowledge Base (apparently more thorough hard copies are available):
The above page just overviews the four main types of validity in that framework (statistical conclusion, internal, construct, external); more detailed treatments are given in separate sections, which might make more sense if read in a more systematic tour through the site (depending on one’s familiarity with social-science research methods and some of the idiosyncratic terminology).
Incidentally, some authors have applied similar threats-to-validity ideas to research synthesis (a.k.a. meta-analysis—in fact, there’s a fairly large literature on ways to avoid faulty conclusions in such quantitative reviews):
Cooper, H. M. (1982). Scientific guidelines for conducting integrative research reviews. Review of Educational Research, 52, 291-302.
Oh! Another Missourian! Glad to have you here. We don’t have many fellow Missourians here. North or South Missouri? Well, no matter if it’s Missour-ee or Missoura for you, Welcome. I’ll have to check out your website when I can.
Welcome, P-S. I certainly agree about the need for literacy in statistics, and the problem isn’t limited to the general public or social scientists. I took only a few lower division stat courses more than a half century ago, but I’ve been distressed at how many otherwise decent chemists make obvious statistical blunders. This may not be the case in academia, but I’ve seen it often in industrial settings. In any case, glad to have you here. Don’t be shy about pointing out if any of us make statistically incorrect conclusions in our posts.
Oh! Another Missourian! Glad to have you here. We don’t have many fellow Missourians here. North or South Missouri? Well, no matter if it’s Missour-ee or Missoura for you, Welcome. I’ll have to check out your website when I can.
Thanks; it’s nice to hear from the MO contingent. I lived in Columbia for a few years and have lived in St. Louis for a few more (and Lake St. Louis for a year in between). I’m not sure whether those count as north, south, or neither—depends on the reason for the distinction, I suppose. Perhaps (slightly) more interesting, where I come from in WI “roof” and “root” rhyme with “put” (not with “boot”), and “route” rhymes with “boot” (not with “pout”). Hmmm.
Thanks! I’ll check them out after the long weekend.
Last night I posted an example “social experiment” in the General Discussion forum. Anyone interested in playing with the threats-to-validity framework might try using it to pick apart that study and design a better one.
If I were more ambitious, I might offer a prize for the best study—but then I’d have to judge the entries , obtain a prize, etc. So for now I’m counting on intrinsic motivation or sheer boredom to push folks into taking a whack at this collective study-design exercise.
Hi, skuld. I too am curious what intellectual disciplines CFI folks associate themselves with (in terms of formal training or personal interest), and in particular how many have backgrounds in psychology or other social or behavioral sciences.
I think you’ll find a wide range of backgrounds here - some with training in physical, biological, and social sciences, and quite a few who, while without formal advanced education are quite well self-taught.
Hi, skuld. I too am curious what intellectual disciplines CFI folks associate themselves with (in terms of formal training or personal interest), and in particular how many have backgrounds in psychology or other social or behavioral sciences.
i have a doctorate in psychology and just completed a postdoctorate in medical rehab outcomes research at kessler institution in new jersey.
We don’t have many fellow Missourians here. North or South Missouri? Well, no matter if it’s Missour-ee or Missoura for you, Welcome. I’ll have to check out your website when I can.
, obtain a prize, etc. So for now I’m counting on intrinsic motivation or sheer boredom to push folks into taking a whack at this collective study-design exercise.
