Learning how to be a critical consumer of adoption research

Several years ago while I was cleaning out some files, I came across my adoption papers. Included in my file was a 1969 research report, “OUR Children: A Study of Korean Adoption.” This study report based on interviews with 46 Korean children adopted through Lutheran Social Service of Minnesota and their parents was fascinating, largely because reading it now reveals that little has changed since 1969. This study is one of many that have been conducted on adoptees or foster alum since Sophie van Senden Theis’s groundbreaking 1924 study on the outcomes of foster children.1


As Dr. Myers points out in his column last month, adoption research is often subject to the same binary of “good” and “bad” as other aspects of adoption and outcome and adjustment studies in particular are framed to either validate that adoptee outcomes are positive (yay for adoption, it works!) or that adoptee outcomes are poor (oh boy, we must improve how we practice adoptions). Research also seems intimidating to non-researchers. I have often called research the equivalent of the scene in the Wizard of Oz where Dorothy and friends are quaking in their ruby slippers at the booming voice and larger than life head of the Great and Powerful Wizard of Oz, only to find, thanks to Toto’s curtain-revealing revelation, that the powerful Wizard is just an ordinary man. What appears to be something magical or beyond our understanding is just a person, pushing levers and buttons. Dare I suggest that to some degree research is similar?


Before I started graduate school and long before I started researching adoption, I was just another adult Korean adoptee without any real understanding or knowledge that because of my experience others like me were being “studied” to see “how we turned out.” When I first began to read research, I had a tendency to put research up on a pedestal like the Great and Powerful Oz. I did not realize that a person could question the research findings, the sample used, the methodology or the theoretical basis for the questions that frame the study. I really began to question adoption research when I learned how to do research and understood how much of the research process is critically tied to the researcher and his or her own thoughts and beliefs about adoption, their personal experiences related to adoption, and the framework or worldview in which they exist. I also learned that many of the adoption studies of the past – particularly outcome and adjustment studies, which historian Ellen Herman2 calls the “adoption research enterprise” – are used by adoption professionals to continue with status quo adoption practices because these studies provide “evidence” that adoption as is “works.”


The very first piece of transracial adoption research I came across several years ago was Simon and Altstein’s well-known longitudinal study of 204 adoptive families from 1972 to 1984.3 At the time, there was something I did not like about that study, but I could not wrap my brain around what part or parts of it bothered me. I had the belief, like a lot of people do, that research studies are the ultimate authorities. I was unaware that a person could critique research. I took research to be the “absolute truth.” Both authors and their study have been frequently cited in popular media outlets, such as the New York Times4 and Christian Science Monitor,5 typically as a counter to the National Association of Black Social Worker’s (NABSW)6 position opposing transracial adoption, as evidence that transracial adoption “works.” Yet, most of these same articles do not interview adult transracially adopted persons. It was not until I learned how to critique, analyze, and, most importantly, how to DO research that I understood why I had such a problem with the Simon and Altstein research.


I am not suggesting we discount adoption research. Every study highlights something important from which we can learn. However, I particularly want adoptees to be better able to see the assumptions and humanness that undergirds adoption research; knowing research is not this Great and Powerful thing that we cannot question – that there are real people, with real biases and motivations, and real limitations behind the “curtain” and that all are attempting to better understand this phenomenon of the adoption experience.


I hope this article helps others deconstruct the way research – in particular in the social and behavioral sciences, including sociology, psychology, and social work – perpetuates the belief that research is objective and value neutral when it comes to adoption. Just think of me as your personal Toto.


Understanding the Research News Cycle


Unless you are a student or faculty at a college or university, access to research tends to be limited and inaccessible. Much of what one learns about research is what is reported in the general media. Jorge Cham, creator of the comic, “PhD Comics”,6 made this cartoon about the Science-News Cycle (below), and while it was written to describe what occurs in the hard sciences, it fits for social science research as well. Using this humorous depiction of the way popular media digests and regurgitates research, an adoption study might look like this:


A psychology researcher at university A conducts a study on what factors predict positive outcomes for transracial adoptees. The study’s findings show that under certain conditions, factors A and B are correlated to outcome C (positive outcomes) with statistical significance; however, factors D, E, and F had no statistical significance. The university releases a press release about the study. A news organization then picks up the story and says, “Study proves transracial adoptees have positive outcomes!” Local news outlets interview someone whose experience fits the study’s findings and cite the study as evidence of its truth. Eventually the authors and basis for the study are long forgotten, but when someone says that transracial adoptees do not have a positive experience, a counter-argument is available: “Well, research says that transracial adoptees have positive experiences so you must be an outlier.”


How to critique adoption research


How does someone understand adoption research? Here are the ways adoption researchers push buttons and pull levers, just like the Great and Powerful Wizard of Oz:


1. Who is the author and what is their worldview?


In their articles for this column, both Dr. Myers and Dr. Raible challenged the notion of “objectivity” in adoption research and I echo and agree with their position. Every person who conducts research has a point of view; we do not grow up isolated in a social vacuum. So, what we learn and know about a topic is influenced by where we grow up, the families and communities where we live, and our lived experiences. These contexts shape the questions and assumptions every researcher has about the topic of their study. In short, every researcher has assumptions and biases; the most any scholar can do is be honest about how their assumptions and biases may influence and shape their scholarship.


Does it matter if the researcher is an adoptive parent, first/birth parent, or adoptee or has a relationship with any of the above? What if the scholar was a former adoption worker? It matters because how a researcher approaches a topic is going to be influenced by their relationship to the topic. The questions that an adoptee may ask about an element of adoption might be different than what an adoptive parent or former adoption professional might ask. This does not mean that one perspective is more or less valid than another’s. It does, though, mean that the research is going to be influenced by the researcher’s paradigm or worldview.


I recently came upon an interview published in a small organization’s newsletter in which a very prominent scholar with numerous published research studies on adoption discusses his adopted child in a particular way that shed light on his research studies. No doubt this scholar’s position as both a parent and a researcher increases his credibility, rather than diminishes it, as many of us adoptee researchers find. Many research studies about adoption have been conducted by adoptive parents who occupy dual roles; a parent and academic. This dual role is often considered an asset. Adoptee scholars, on the other hand, are often considered suspect. Our objectivity is questioned. I have yet to hear the criticism of “too personal” used against an adoptive parent researcher.


2. Who is the sample?


When research is about people, the “sample” is the selection of people that are the object of study. Often times the media reports a general explanation of the sample through stating the number of people surveyed, interviewed, or whose case record data was reviewed. How the sample for the study was obtained is also important. If the researchers for a study on outcomes on transracial adoption, for example, contact an adoption agency in search of participants, they will likely only get participants for their study that are still in contact with that agency. That means potential important information from participants who no longer stay in touch with the agency will not be included. This might potentially skew the results; maybe families that are struggling with their adoption are no longer in contact with their agency.


Unlike the hard sciences where one can use controlled settings to minimize any factors other than the ones being tested, human beings are tricky to study. We have moods, motivations that change by the minute, hour, or day, we are influenced by our environments, and we have values and beliefs that impact how we behave or answer a survey. We are also notoriously “wild” in the world – most of the adoption research conducted involves volunteer participation. So, how the researcher finds the people to participate is almost always skewed in some way.


A few years ago the Evan B. Donaldson Adoption Institute published the report, “Beyond Culture Camp,” in which 179 Korean adoptees in the U.S. were surveyed. This is a good-sized sample, always considered a good thing in quantitative research because the larger the sample, the more people think they can generalize the results. But even this sample needs to be taken with a few grains of salt. First, as someone who is quite involved in the Korean adoptee community, I saw this call for participants advertised from numerous Korean adoptee groups. How many Korean adoptees who do not participate in online or in-person Korean adoptee organizations participated? How else did Korean adoptees find out about the study? Did the way this call for participants was promoted increase a specific type of respondent, one who was more likely to socialize with other Korean adoptees? We do not know the answers. At the very least, the report did not say whether it asked how the respondent heard about the study, which would have given more insight; but we have to remind ourselves that sample is important – the size, the way the sample was obtained, and the recognition of who is not included in the sample.


3. What is the unit of analysis?


The unit of analysis is the actual “thing” that is being studied. It could be an individual, a family, or a group. When looking at the sample, be careful to note the cases when the unit of analysis is the adoptee, but it is an adoptive parent or a caseworker who is providing the responses to the questionnaire or survey. One of the aspects of several studies on transracial adoption that disturb me is when the conclusions about adopted children are based on the responses adoptive parents provided on behalf of the child.


4. What are the researcher’s questions?


You can discern pretty quickly the researcher’s worldview by looking at the research questions. Questions are generated by the researcher’s knowledge of the topic and the opinions that are formed after reading the existing literature. When looking at a research study, look to see how clear the researcher’s questions are stated. Is the question, “do transracial adoptees have a positive racial identity?” or, “what factors contribute to a transracial adoptee’s racial identity?” Maybe the question is, “what factors contribute to adoption failures?” You can see where I am going here. The researcher posits the questions they hope to answer and the questions are not value neutral – they typically relate to the researcher’s worldview.


5. What is the methodology being used?


Methodology is how the researcher conducts his or her study. Research studies may be quantitative, qualitative, or mixed methods (using both quantitative and qualitative methods). Different academic disciplines tend to produce different types of studies. Psychology and social work, for example, often use quantitative research while the humanities (history, ethnic/gender/queer studies) are more likely to use qualitative studies. The methodology that a researcher uses is driven by the questions the researcher wants to explore.


Quantitative studies often attempt to explain the who and what and may also try to answer questions about predictions. For example, a common question in adoption research is to look at what “factors” predict a “successful” adoption. The question may be, “Who are the children most likely to succeed in adoption?” Quantitative studies often aim to test a hypothesis – will x happen under condition y ? – with the larger goal of being able to generalize their findings. The goal is to compare the adoptees to a cohort of others and rate them on some bell curve of “normalcy.” Quantitative studies are able to provide some information about patterns of behaviors or factors.


Qualitative studies tend to look more at the history or the phenomenon of an experience. Qualitative studies may look at the culture of something (ethnography), the lived experiences of people (phenomenology) or explore life stories (narrative or oral history). A qualitative study will often include in-depth interviews with the study participants, and may also include observations and analyzing written or other types of documents. Qualitative studies do not begin with a hypothesis, but rather the question, “What has been the meaning of your lived experiences?” The findings are not generalizable, but often are able to get to a much deeper nuance of the complexity around the adoptee’s lived experiences.


Knowing the type of research helps you understand the goal of the research study.


6. What tools for assessment or measurement were used?


Non-researchers often do not realize that the tools that are used to measure or assess behavior, thoughts, values, or beliefs are only as objective as the humans who design, implement, and score these tools. We also need to take a look at how these tools were developed and who they were “normed” on. If the sample used to develop the assessment tool was all middle-class, white college-aged adult men, how would what this group of folks respond to be similar or different to people of other races, gender, age, or education levels?


For example, a study on the “adjustment” of transnational and/or transracial adoptees, the metrics for what constitutes “well-adjusted” are typically criteria, such as mental health diagnoses, behaviors (often reported by adoptive parents or other observer, not self-reported by the adopted individual), whether the adoptee has friends, or scores in a certain range on some ratings scale.


Many of the assessment tools are also somewhat limited because the scales typically ask people to rank themselves (for example on a scale o 1-5) or make them choose from a limited set of answers that a person might not feel accurately describes how they want to answer. At best, these tools offer a helpful “point in time” picture of a person, but do not always factor other limitations, such as if an adolescent adoptee is responding based on whether they want to please the researcher, or if they answer dramatically different because of the atypical situation of being in a research study.


7. Pay attention to the limitations.


Every research study includes a section on the limitations of the articles. Believe authors if they say: their sample was limited or constrained; that they cannot generalize beyond the scope of their study; that they had missing data, a poor measure used to gather the data, or other issues that impacted their study’s conclusions. There may, of course, be more limitations than the authors acknowledge, but at least this section gives some place to begin your analysis of the study.


8. Look for gaps in the analysis.


Numbers, statistics, and interview data are one thing, but in the end, the data is analyzed by the researchers, and here is often where you see some gaps in what an adoptee researcher might ask an adoptee participant compared to a non-adoptee researcher. In the Simon and Altstein study, 66% of the Black adoptees stated that they were proud to be “Black” or “Brown,” yet 73% of these adoptees chose all white friends. Simon and Altstein concluded this disparity was due to the majority white neighborhoods and the majority white schools of these families. While environment plays a big role, I also wonder how much the authors were too willing to attribute these results only to environment, and did not look at whether there was a disconnect between what the adoptees expressed to parents and researchers and how they felt internally given that the researchers are white and have not had the lived experience of transracial adoption. Would it occur to to the authors that there would be any other explanation?


I often see further analysis related to transracial and intercountry adoption identity research is the gap in what we know about human development and identity and what is viewed as typical for adoptees. Many adult adoptees do not feel any racial identity conflict until they are in their mid-life years, after marrying and having children of their own. Most longitudinal studies end with adoptees in their mid-20s, not at an age when identity constructions or self-definitions have solidified. Since most scholars who study or theorize about identity issues (racial or not) designate adolescence and young adulthood as being the times for a person’s identity construction, it is underwhelming for me to read that an 8 year old has “a positive racial identity.” I meet transracial adoptees all the time who only begin thinking about racial identity in their thirties or forties.


There is also a hesitation to explore further instances when there are small findings for potential, emotionally significant, if not statistically significant, findings. While these numbers may be small, they are important. For example, in a study I conducted at the University of Minnesota on adopted children in residential treatment, the number of intercountry adoptees was too small to be able to analyze statistically. Yet, despite the small numbers, there were some important – if not statistically significant – findings. Another example of a gap in the existing research is about abuse of adoptees by adoptive parents. Anecdotally, the number of adoptees I know who were physically, sexually, or emotionally abused by their adoptive parents is shockingly high. Yet, most adoption professionals are extremely unwilling or unable to consider this information credible because there has not been an empirical study that has found statistically significant results – therefore rendering this information “anecdotal.” While the numbers may be small compared to the overall numbers of adoptees who are not abused by their adoptive parents, it does not invalidate that some adoptees are abused by their adoptive parents and that this should be something that needs further exploration. When a reader begins to see certain types of results are never reported, such as, “Were you abused or neglected by your adoptive parent(s)?,” she should ask whether the question was 1) not asked, 2) asked but not reported, and 3) why.


Conclusion


Adoptees are often quite skeptical of research that is conducted on them and for good reason. Often times, the researchers are not asking questions that are relevant to the adoptees, or they are making conclusions that adoptees feel are not accurate. My own evolution into becoming a researcher was a direct result of feeling that the existing research did not seem to accurately represent my experiences as a Korean adoptee. In addition, I thought there were a lot of gaps in terms of what existing researchers were studying. I am encouraged by the growth in adoptees who are doing research; adoptee researchers will be offering additional information about the adoptee experience based on questions non-adopted researchers may never have even considered asking.


I would like to offer a challenge to any researcher who may be reading this, who has or is currently, or plans in the future to conduct research on adoptees. As part of the adoptee community, I hear from many adoptees that they feel exploited and used when approached to participate in yet another study about adoptees. Since 2006, I have been keeping track of calls for participants for research studies on transracial and intercountry adoptees. I currently have 55 studies in my log and those were just the ones in which I had personally seen a call for participants. I am sure there are many more, particularly by students for the undergraduate or graduate thesis research projects. Many of these studies never get published, and many adoptees say that after their participation they never hear from the researcher or get to read the resulting thesis or articles. Adoptees have been generous in taking the time to participate in studies, and I would like to see more researchers who benefit from the participation of adoptees for their research do a more thorough job of disseminating their research. Those who do publish in academic peer-review journals, remember that most adoptees do not have access to these journals.


Dorothy herself was an orphan, “adopted” by her Aunt Em and Uncle Henry. She travels with her friends to the Emerald City to seek help from the Great and Powerful Wizard of Oz, only to find out that the Wizard is just a regular man after all. Likewise, there is no mystery to research. Adoptees often think of research as the absolute truth, without realizing that behind the numbers and statistics and academic jargon, there is just a human being trying to make sense out of his questions about the adoption experience. Hopefully, you now have some tools to be your own Toto, to pull back the curtain, and learn to critique adoption research. Or, perhaps this article may have inspired some of you to join the growing number of adoptee scholars and researchers working to grow and expand the world’s understanding about adoption.


~ JaeRan Kim


References


1. van Senden Theis, Sophie (1924). How foster children turn out. Retrieved from Adoption History Project, http://pages.uoregon.edu/adoption/studies/TheisHFCTO.htm


2. Herman, Ellen (2008). Kinship by design: A history of adoption in the modern United States. Chicago: University of Chicago Press, p. 190.


3. Simon, Rita and Howard Altstein (1977). Transracial adoption. New York: Wiley.


4. See: New York Times articles “De-emphasis on race in adoption is criticized” by Ron Nixon, May 27, 2008 at http://www.nytimes.com/2008/05/27/us/27adopt.html?pagewanted=1&_r=0&sq&st=cse%22%20minority%20&scp=1%22Evan%20B.%20Donaldson%20Adoption%20Institute; “Debate on race and adoptions is being reborn” by Charisse Jones, October 24, 1993 at http://www.nytimes.com/1993/10/24/nyregion/debate-on-race-and-adoptions-is-being-reborn.html?src=pm; and “Parent and child: Beyond ‘Losing Isaiah’: Truths in shades of gray” by Lena Williams, March 23, 1995 at http://www.nytimes.com/1995/03/23/garden/parent-child-beyond-losing-isaiah-truth-in-shades-of-gray.html?pagewanted=all&src=pm


5. Christian Science Monitor


http://www.csmonitor.com/1999/0414/p1s5.html/(page)/3


6. See “National Association of Black Social Workers ‘Position Statement on Transracial Adoption’ September 1972 at Adoption History Project, http://pages.uoregon.edu/adoption/archive/NabswTRA.htm and the NABSW’s “Preserving Families of African Ancestry” at http://www.nabsw.org/mserver/PreservingFamilies.aspx


7. Cham, Jorge. PhD Comics: Piled Higher and Deeper. Retrieved from http://www.phdcomics.com/comics.php?f=1174


8. McGinnis, Hollee; Susan Livingston Smith, Dr. Scott D. Ryan, and Dr. Jeanne A. Howard, (2009). Beyond culture camp: Promoting healthy identity formation in adoption. New York: Evan B. Donaldson Adoption Institute.