I am still reviewing the scientific literature on child custody evaluations. I’m at the third or fourth tier of research right now. In my review of the literature to date, which is fairly extensive, I am deeply disturbed by what I’m finding, or actually NOT finding.
From what I see…
There is absolutely no scientifically based foundation to the practice of child custody evaluations. Zero. None.
Child custody evaluations are little more than exceedingly expensive guesses. As far as I can tell, the recommendations produced by child custody evaluations are no more valid that looking in a crystal ball or reading the entrails of a goat.
I have found no scientific research supporting the validity of the conclusions and recommendations produced by child custody evaluations. In fact, I’ve found no scientific research that has even TRIED to support the validity of the conclusions and recommendations of child custody evaluations.
And even the theoretical foundations from clinical and developmental psychology that might be relied on for making the guesses that occur in child custody evaluations appear to be absent or deeply flawed.
Let me be clear on this statement:
Based on my review of the research literature, there is no scientific research or scientifically derived data to support the validity of the conclusions and recommendations reached by child custody evaluations.
The Construct of Validity
The scientific construct of validity essentially means that the conclusions we reach as a result of our assessment or research are true… that they are valid.
According to a standard textbook on scientific methodology (Cozby, 2009):
“Validity refers to “truth” and the accurate representation of information” (p. 85).
The scientific construct of validity refers to the degree to which the findings of our assessment or research are true.
So, for example, the validity of an intelligence test means the degree to which the intelligence test actually measures the construct of “intelligence.”
A test of puzzle solving ability might represent a valid measure of intelligence, depending on how “intelligence” is defined, but a test of a person’s ability to count from 1 to 10 is not likely to be a valid test of intelligence (depending on how the construct of “intelligence” is operationally defined).
I don’t want to become too technical on this point, but I do want to establish that this isn’t me, this is standard scientific methodology, so again, turning to the textbook definition of validity:
“Construct validity refers to the adequacy of the operational definition of variables. To what extent does the operational definition of a variable actually reflect the true theoretical meaning of the variable?
“In terms of measurement, construct validity is a question of whether the measure that is employed actually measures the construct it is intended to measure.”
“Applicants for some jobs are required to take a Clerical Ability Test; this measure is supposed to predict an individual’s clerical ability. The validity of such a test is determined by whether it actually does measure this ability.” (Cozby, 2009, p. 96)
Validity is a central construct in scientific research and assessment.
The scientific method contains several defined approaches of establishing validity for an assessment instrument or procedure. Again, according to Cozby (2009) these scientifically established methods for determining validity include:
Face Validity: The content of the measure appears to reflect the construct being measured.
Content Validity: The content of the measure is linked to the universe of content that defines the construct.
Predictive Validity: Scores on the measure predict behavior on a criterion measured at a time in the future.
Concurrent Validity: Scores on the measure are related to a criterion measured at the same time (concurrently).
Convergent Validity: Scores on the measure are related to other measures of the same construct.
Discriminant validity
Scores on the measure are not related to other measures that are theoretically different.
Cozby, 2009, p. 97
Again, I don’t want to get too technical in this blog post, I just want to highlight that this isn’t me.
The construct of validity is a standard scientifically defined construct regarding whether something we assert is true, and there are standard scientifically defined approaches to establishing an assessment procedure’s validity.
Regarding the validity of the conclusions and recommendations produced by child custody evaluations (i.e., are they true), no effort has even been made to establish the scientific validity of the conclusions and recommendations reached through the process of child custody evaluations. Much as I try (and I’m trying), I cannot find a single research study examining the scientific evidence for the validity of child custody evaluations.
I want to be clear on this, I’m not saying that the scientific data on the validity of child custody evaluations is weak… I’m saying it is NON-EXISTENT.
There is absolutely NO scientifically established foundation for the validity of the conclusions and recommendations produced by child custody evaluations. None. Zero. Not one study. Ever. Nothing.
There is no scientific support whatsoever for the validity of the conclusions and recommendations produced by child custody evaluations. Might as well cast tarot cards or have a monkey throw darts at a dartboard.
There is no scientifically based support for the validity of the conclusions and recommendations produced by child custody evaluations. None.
Child custody evaluations are essentially, “junk science” and “voodoo assessment.”
That’s a strong statement.
Yet I would challenge any proponent for the practice of child custody evaluations to cite for me one research study that even seeks to establish the scientific validity of the conclusions and recommendations produced by child custody evaluations. No one has even tried to establish the scientific foundation for the validity of child custody evaluations.
Even more to the point, however, I would challenge the proponents for the practice of child custody evaluations to cite me the research support demonstrating the validity for the conclusions and recommendations of child custody evaluations, the face validity, content validity, predictive validity, concurrent validity, convergent validity, and/or discriminant validity.
There is none. Zero. Nothing. There is NO scientifically established foundation for the conclusions and recommendations produced by the practice of child custody evaluations. None.
The systematic collection of data provides the APPEARANCE of scientific rigor, but the conclusions and recommendations are 100% guesswork. There is no scientific support for the validity of the conclusions and recommendations produced by child custody evaluations. None.
The conclusions and recommendations of child custody evaluations are essentially “junk science” – “voodoo assessment” – rattle some beads, perform some rituals of data collection, recite some incantations, and just make up some recommendations based on the whims and prejudices of the moment.
Despite the apparent rigor involved with the systematic collection of data, there are NO scientifically described or established criteria in any of the literature for linking the conclusions and recommendations made in child custody evaluations to the data collected. As far as I can tell, it is pure, unadulterated, guesswork that has no defined linkage to any theoretical or scientifically established foundation.
Might as well read the entrails of a goat.
Operational Definitions
As noted by Cozby, the key to establishing the scientific validity for any assessment procedure is to “operationally define” the construct being assessed.
If, for example, we are going to create an assessment for “intelligence,” we first need to “operationally define” what we mean by “intelligence.” Is it the amount of vocabulary the person knows? Is it some form of problem solving ability? Is it a combination of both? Are there different types of “intelligence?”
How do we define the construct of “intelligence” that we are going to be assessing? The operational definition for the construct provides the foundation for the scientific validity studies that will follow. If we don’t have an operational definition for the construct, then we cannot collect scientific data on the validity of the construct because we haven’t defined what the construct means.
Once we define what we mean by a given construct, such as “intelligence,” other people may then disagree with our definition, and a lively debate and dialogue ensues regarding the definition of the construct. And different approaches to assessment will emerge based on different approaches to defining the construct.
However, if we don’t ever define the constructs we’re assessing, then no debate or discussion ever occurs. Everyone just makes up their own definitions based on whatever they need the construct to mean in order to justify what it is that they want to do.
In one case, the “best interests” of the child are factors xyz. In another case, they’re factors abc. In a third case, they’re factors qrs. There is no defined standard for determining what the “best interests” of the child are.
For evaluator A, the child’s “best interests” might be x. For evaluator B, the child’s “best interests” might be y. Without an operational definition for the construct, the “best interests” of the child become whatever I want them to be in order to justify my decision.
The “best interests” of the child becomes a fluid and malleable construct that I can define in any way I want based on whatever it is that I want to do. If I want to recommend xyz, I simply emphasize xyz as being in the “best interests” of the child and I minimize the importance of qrs. If, on the other hand, I want to do qrs, then I simply define qrs as being in the “best interests” of the child, and I minimize the importance of xyz. The construct becomes a means to justify whatever decision I want to make.
My decisions aren’t based on the best interests of the child. In fact, it’s just the reverse, the “best interests” of the child are based on my decision. Whatever I decide, I then use the construct of the “best interests” of the child to justify this decision.
Q: But aren’t your conclusions and recommendations based on the data?
A: Naw, not really. I collect a lot of data, but then I can interpret and weight the data in any way I want. I can make this thing more important than that. Or I can ignore this data and highlight that data. I can do that in any way I want, because nothing is defined, there are no operational definitions for any of this. It’s all based on however I define the constructs based on my desires, whims, and prejudices. So I just decide what I want my conclusions and recommendations to be, and then I interpret the data accordingly, weighting this and discounting that.
Q: But what about all that data collection you do? Doesn’t that mean anything?
A: That’s just show. It’s a ritual we go through to give the appearance of scientific rigor.
By putting in so much effort and collecting so much information it looks like our conclusions and recommendations must be based on the application of “scientific principles” to the thorough collection of data. But that’s just a show for the audience. If we didn’t collect all that data, no one would give our conclusions any credibility. So we have to do it to establish our credibility.
But when it comes down to it, there’s no established principles or guidelines for how we INTERPRET that data, and it’s the interpretation of data that really matters. So we can pretty much do whatever we want in terms of coming up with our conclusions and recommendations, we can reach any conclusion we want or offer any recommendation, without any restriction or limitation imposed by whether our conclusions or recommendations are accurate or correct.
Generally it’s best to stay in the mid-range with recommendations. If you don’t take a stand, you can’t really be attacked. Just kind of go with the way things are, maybe a little nudge here and there. And if there’s any unresolved issues, just recommend therapy.
Oh, and here’s the best part, because child custody evaluations are kept protected by the court, no other mental health professionals ever review our work for the accuracy of our interpretations, conclusions, and recommendations. We can pretty much do whatever we want And let me tell you, all that time spent collecting data, and then report writing, is pretty lucrative.
Providing operational definitions for a construct allows professional psychology to discuss and debate the accuracy of this definition. New ideas emerge and the understanding for the construct deepens and improves through professional dialogue and debate, which ultimately leads to better assessment procedures and improved methodologies.
For example, in the field of intelligence assessment, developing an operational definition for the construct of intelligence has created tremendously robust professional dialogue and disagreement. There’s Spearman’s proposal for a general intelligence factor (“g”), there’s Thurstone’s set of primary mental abilities, there’s Cattell and Horn’s proposal for fluid and crystallized intelligence, there’s Howard Gardner’s (different Gardner) proposal for eight distinctly different types of intelligence. With each proposal regarding an operational definition for “intelligence” our understanding for and assessment of the construct improves.
Absence of Professional Discussion
Where is the corresponding robust debate and dialogue regarding the constructs used and assessed by child custody evaluations?
What do we mean by the construct “best interests” of the child? How are we operationally defining this construct of “best interests?”
As important as our operational definition for this construct, what is the scientific evidence that supports our operational definition of the “best interests” of the child as being the factors we identify? Where is the professional dialogue and debate?
What do we mean by the construct of “parental capacity?” How are we operationally defining the construct of “parental capacity?”
As important as our operational definition for this construct, what is the scientific evidence that supports the factors we’re using in our operational definition of “parental capacity?” Where is the professional dialogue and debate surrounding the key factors in parenting?
Where is the robust debate and dialogue within professional psychology surrounding what factors define the “best interests” of the child, or what factors define “parental capacity?” There is none. It is totally absent. Doesn’t anyone else in mental health find that spookily disturbing? That we have NO professional dialogue or debate about such central tenets of child custody assessment?
Q: How is it we have NO discussion or debate around defining these constructs?
A: There’s no disagreement because we just let everyone make up whatever definition they want. No definition. No debate.
Try as I may, I cannot find a single operational definition for either of these key and central constructs for the assessment conducted in child custody evaluations. I find general guidelines, such as for the “best interests” of the child:
1.) the child’s wishes,
2.) any history of abuse,
3.) the parents’ wishes,
4.)each parents’ ability to share the child with the other parent, and
5.) the environment that best promotes the development of physical, mental, and spiritual faculties.
Current statutes. (2003). In Handbook of forensic psychology: Resource for mental health and legal professionals. Oxford, United Kingdom: Elsevier Science & Technology.
But these general guidelines for domains of information to consider lack the specificity needed to be reliable operational definitions for the construct of “best interests” of the child.
How should we interpret the child’s expressed wishes? How much weight do we give them relative to other factors?
Debate: And if we consider the child’s expressed wishes, won’t we then be turning the child into a “prize to be won” by the parents, and won’t this lead to efforts by the parents to influence the child’s choice and preferences (“Choose me, Choose me. If you come live with me I won’t make you do homework. If you come live with me I’ll buy you a new gaming system.”). Won’t this turn the child into a battleground for the parents’ spousal dispute as a “prize to be won” by the “best parent” (by the parent who best appeases or most intimidates the child)?
How will considering the child’s wishes affect, or be affected by, the parent’s ability to share the child with the other parent? Aren’t we making it harder for the parents to “share the child” by making them competitors for the child’s affection?
And how are we operationally defining the last construct of “the environment that best promotes the development of physical, mental, and spiritual faculties” of the child? What are the criteria by which we are making this determination?
In all of my efforts to date, and they have been considerable, I have yet to find an operational definition for either of the key and central constructs of child custody evaluations; the “best interests” of the child and the “parental capacity” of the parent. I see these terms used, I just haven’t located an operational definition for what these terms mean.
Without operational definitions for either of these key and central constructs of child custody assessments, then there can be no scientifically established basis for the assessment. The child custody evaluation becomes nothing more that “making it up as we go” by defining “best interests” or “parental capacity” in whatever way we want in order to justify whatever we decide to do.
I find a whole lot of guidelines for what data to collect, and for how the data should be collected. But that’s not the same thing as operational definitions for how to INTERPRET and use the collected data to reach a conclusion and recommendations. It’s this second part, regarding the criteria by which the clinical data obtained during the custody evaluation should be interpreted, in which the professional silence is deafening.
Not my Fault
The emperor has no clothes. Sorry. He’s naked. That’s not my fault.
To my professional colleagues… don’t get mad at me. Somebody needs to say it. Child custody evaluations have no operational definitions for the key and central constructs they use in their assessment, and they have no scientific support for the validity of the conclusions and recommendations they make.
Child custody evaluations are scientifically naked. The emperor has no clothes. Sorry. Not my fault. I’m not the tailor, I’m only the kid standing on the parade route, watching the (naked) emperor go by.
Child on the parade route: “Look mommy, that custody evaluation has no clothes on.”
Mommy: “Shhh, don’t say that, you’ll get in trouble.”
Don’t blame me, I’m not the tailor. I’m just the kid watching the naked emperor go by.
There is no scientifically established basis for the conclusions and recommendations reached by child custody evaluations. They are “junk science” comprised of “voodoo assessment” – rattle some beads, perform some rituals, recite some incantations, and make up some pronouncement based on whatever whim, motive, or prejudice moves you.
Secrecy of the “Insiders”
Child custody evaluations are secret reports guarded and protected by the court. Since their release is restricted, they are not subject to critical professional review and scrutiny. They represent the judgement of one person, operating alone, without consultation or review. When they are subjected to review and scrutiny, it is typically by other forensic child custody evaluators to see if the “procedures” of the child custody evaluation process were followed, not regarding the accuracy of clinical data interpretation and the validity of the recommendations.
Any critical review of the child custody report is not about the clinical interpretation of the clinical data, or the validity of recommendations that were derived from the interpretation of the clinical data. Instead the review is about the “procedures” employed in the custody evaluation; did the custody evaluator rattle the proper beads and perform the proper rituals to appease the tutelary spirits of child custody? Were collaterals interviewed? Were home visits made? Were the proper test instruments employed?
And the best way to stay out of trouble is to make middle-of-the-road recommendations. And by all means, DON’T IDENTIFY PARENTAL PATHOLOGY (even if identifying the parental pathology is in the best interests of the child).
The rare professional reviews of child custody evaluations that do occur do not typically involve a critical analysis regarding the accuracy of the clinical psychology interpretations made regarding the clinical data collected, nor do they involve a critical analysis of the appropriateness from a clinical psychology framework regarding the recommendations made based on the interpretation of the clinical data.
In my role as an expert consultant in legal cases, on multiple occasions the court has made available for my review child custody evaluations. I have had the opportunity to review the clinical data reported in the custody evaluations, as well as the professional interpretation of this clinical data and the recommendations that were made based on this interpretation of the clinical data. As a clinical psychologist, I am deeply appalled by the extraordinarily poor interpretations of the clinical data that I have found in the child custody evaluations that I have professionally reviewed.
As a clinical psychologist, it is bad. VERY bad.
Statement to the Court
I have tremendous respect for the courts and our legal system.
My father, an attorney, worked for the federal court system for 30 years. He was with the State Bar of California and served as a magistrate within the court system. He was a man of great integrity. I have a deep respect for him, and for the court system in which he served.
Out of my deep respect for the justice system and for the Court, and from my professional integrity as a clinical psychologist serving children and families, and from my professional background in CLINICAL PSYCHOLOGY (not forensic psychology), my understanding of child and family psychotherapy, and my professional knowledge of child development, I wish to respectfully offer to the Court my extremely deep and troubling concern about the QUALITY of the clinical interpretations made in forensic child custody evaluations.
The secrecy in which these child custody evaluations are held prevents their professional review regarding the level of professional competency and therefore accuracy of the clinical interpretations of the clinical data collected in these forensic evaluations. These forensic evaluations do an exceptional job of collecting data, but the clinical interpretations of the clinical data is, in the cases I have reviewed, deeply flawed and deeply troubling.
I am concerned that an inherent conflict of interest exists within forensic psychology that prevents an adequate critical analysis within professional psychology regarding the practice of child custody evaluations, and that this inherent conflict of interest prevents relevant information from being made available for the Court’s consideration regarding the absence of scientifically established validity for the recommendations provided by child custody evaluations and the poor quality of clinical interpretations contained within these custody recommendations.
The field of child custody evaluations is currently within an echo chamber of like-minded forensic psychologists that prevents an appropriately critical professional oversight and review of the interpretations and recommendations made in child custody evaluations, and of the absence of scientifically established foundation for the interpretations and recommendations made by child custody evaluations.
Based on my review of the clinical interpretations made in the multiple child custody evaluations that the Court has allowed me to review as an expert consultant to my clients, I wish to respectfully offer to the Court my deep concern as a clinical psychologist regarding the level of professional accuracy contained in the CLINICAL interpretations of the clinically relevant data contained in child custody evaluations, which adversely affects the conclusions and recommendations reached in these child custody reports.
Recommendation to the Court
The recommendations I would respectfully offer to the Court are:
1.) Consulting Psychologist: I would recommend that the Court allow each parent (if they choose) to select a consulting psychologist in addition to the court-appointed forensic evaluator, thereby creating a panel of three psychologists surrounding the custody evaluation; one psychologist representing the court, and a psychologist representing each of the parents, much in the same way as each parent is represented by legal counsel in the courtroom.
2.) Review and Consultation: I would recommend that these consulting psychologists be empowered to review with the court-appointed psychologist the clinical data once it is collected by the court-appointed custody evaluator, and that they provide professional consultation to the court-appointed custody evaluator regarding the interpretation of the clinical data, and the potential conclusions and recommendations to be derived from the clinical data.
3.) Dissenting Opinion: I would also recommend that these consulting psychologists be allowed to write a “dissenting opinion” if they choose regarding the interpretation of the clinical data and the recommendations made by the court-appointed psychologist, which would be appended to the final report of the court-appointed custody evaluator.
This oversight and consultation by independent professionals is warranted by the tremendous importance of the decision and recommendations provided by the child custody evaluation and the complete absence of scientific foundation for the validity of the conclusions and recommendations reached by child custody assessments.
Professional Psychology
I would also call on professional psychology to critically examine and consider the theoretical and scientific foundations for the practice of child custody evaluations. My concerns are based on the following.
1.) Absence of Scientific Foundation: The complete absence of any research examining and supporting the scientifically established validity of the conclusions and recommendations reached by child custody assessments. There is no supporting scientific evidence for the face validity, content validity, predictive validity, concurrent validity, convergent validity, or discriminant validity of the conclusions and recommendations produced by child custody assessments. In the absence of such supportive scientific evidence, the recommendations offered by child custody assessments are little more than “junk science” and “voodoo assessment.”
2.) Absence of Operational Definitions: The complete absence of established operational definitions for the key and central constructs of the child’s “best interests” and the parent’s “parental capacity” that are central to the child custody assessment. Given the incredible importance of the recommendations being rendered by child custody evaluations in influencing the Court regarding the lives of children and families, there needs to be a much more engaged and vigorous professional discussion regarding the specific factors defining the constructs of the child’s “best interests” and the parent’s “parental capacity” to meet those interests (similar to the robust discussions generated surrounding the construct of “intelligence”).
3.) Cultural Considerations: Any assessment of parenting and family processes is necessarily embedded in a cultural context. A robust and vigorous discussion needs to be engaged regarding the influence of culture on the process of child custody assessment and the formation of recommendations, particularly around the standard employed in assessing parenting practices and the establishment of family values.
4.) Conflict of Interest: The current practice of conducting child custody evaluations is financially lucrative. The ability of forensic psychology to critically evaluate itself is therefore compromised by an inherent conflict of interest. As a consequence of this inherent conflict of interest in meeting the needs of clients, it becomes even more essential to ensure that a deeply critical independent analysis be conducted regarding the scientific validity for the interpretations and recommendations reached by child custody evaluations.
5.) Secrecy and Oversight: Procedures need to be established to provide reasonable professional oversight regarding the validity of the clinical interpretations made by a child custody evaluator, especially given the complete absence of scientific support for the validity of child custody assessments and recommendations.
Conclusions
Based on my review of the scientific literature surrounding the conclusions and recommendations provided by child custody evaluations, I have reached the conclusion that the practice of child custody evaluations as currently structured represents little more than “junk science” and “voodoo assessment” which does not merit consideration in court proceedings.
I am certain that this conclusion will generate considerable disagreement. My response is to request a citation to any scientific article that even assesses the face validity, content validity, predictive validity, concurrent validity, convergent validity, or discriminant validity of the conclusions and recommendations produced by child custody evaluations.
To take this incredibly low bar just a tad higher, I would request a citation to any research demonstrating the face validity, content validity, predictive validity, concurrent validity, convergent validity, or discriminant validity of the conclusions and recommendations produced by child custody evaluations.
Also, I would request a citation to any “operational definition” for the constructs of “best interests” of the child and “parental capacity” of the parent which are the central tenets of the assessment.
You don’t need to reference me to the general professional guidelines regarding what information to collect, or how to collect it. I am asking for the reference to the actual operational definitions for what these constructs mean that can be applied to the collected data in interpreting and formulating the conclusions and recommendations of the assessment (i.e., an operational definition for what factors in the collected data indicate the “best interests” of the child or the “parental capacity” of the parent, and regarding an operational definition for what factors in the data indicate “non-best interests” and “non-parental capacity” of the parent.
Until the scientific foundation for the conclusions and recommendations of child custody evaluations is established, I must conclude that child custody evaluations are little more than “junk science” and “voodoo assessment” that do not merit court consideration. Rattle some beads and read the entrails of a goat.
Or offer me a citation for the scientifically established validity of the conclusions and recommendations derived from child custody evaluations.
Craig Childress, Psy.D
Clinical Psychologist, PSY 18857
References:
Cozby, P. C. (2009). Methods in Behavioral Research: Tenth Edition. New York, NY: McGraw-Hill.
Current statutes. (2003). In Handbook of forensic psychology: Resource for mental health and legal professionals. Oxford, United Kingdom: Elsevier Science & Technology.
Dr. Craig Childress: Attachment Based "Parental Alienation" (AB-PA) 