Post: Brain Gym --where is the REAL research?

Posted by L.L. on 1/29/05

More | Report Post

With Brain Gym, one may not understand or be able to fully
assess the
neurological and physiological underpinnings of the
educational
kinesiology theory; however, testing the results/outcomes
of
implementing "Brain Gym" techniques by employing solid,
well-designed
research is rather straightforward. Here are my concerns
about
the "literature" for Brain Gym---and some suggestions for
improving
the literature so that, if Brain Gym IS/IS NOT effective,
we can trust
the data behind those conclusions:

Type “Brain Gym” into www.google.com and a dearth of
testimonials,
pseudo-research, anecdotal evidence, and hype is found on
a multitude
of websites purporting Brain Gym as being the “cure all”
for learning
problems. People have sadly been lured into the belief
that it is
effective and frankly, nearly miraculous. Including us.

As I scanned through the literature available, as well as
many
websites containing “information” about Brain Gym, I
noticed several
consistent flaws in the research pertaining to Brain Gym
or
Educational Kinesiology.

1.Studies that purported Brain Gym’s efficacy generally
contained:
a.Small sample sizes (they were case studies, anecdotal
research,
testimonials—these studies all had too small of a sample
size to
adequately trust the data when generalizing it to the
population
at large.)
b. Studies were often of a very short duration—thereby
reducing or
eliminating long-term predictive validity of results, as
well as
generalization of outcomes to other populations.
c. The definitions of measures used to describe baseline
data
and/or outcomes were not properly operationalized. (In
other
words, the description of what was to be measured—such as
rating
scales used to report growth, teacher opinion, student
behavior,
etc. were not specific enough—opening the window for
inaccurate
assessment and significantly confounding results. They
also were
not designed to exclude many other possible extraneous
variables
that could affect study outcomes. )
d. Placebo effect had not been eliminated as a variable.
e. Hawthorne effect had not been eliminated as a variable.
f. Effects of subjects’ attention (Attention Effect) to
treatment
condition vs. control group were not controlled.
g. Motivation effects were not controlled.
h. Outcome bias was not controlled.
i. None contained double-blind studies, or studies
containing the
observations/assessments conducted by evaluators who were
not
previously aware of which students received treatment.
j. In all but the Ferree’s study (see below), control
groups were
not properly designed. There was either the treatment
group,
or a control group that got no treatment. There were no
groups that got an equal, but different, treatment (e.g.,
such
as a control group that engaged in “generic” physical
movement, such as unscripted exercise, with which the
differences between prescribed Brain Gym activities could
be
accurately compared. This would help one understand
whether
general exercise/activity is beneficial to better academic
outcomes versus specifically Brain Gym—and whether one has
more impact than another. Even more problematic was the
fact
that studies lacked design that would enable us to
determine
if the outcomes were due to Brain Gym, exercise, or the
result
of one or more of the aforementioned concerns stated in a.
through i.)
k.Baseline and Outcome measures (pre-intervention and post-
intervention measures/tests/observations) were conducted
using
unreliable tools. These may include:
i.Assessments that are not capable of reliably measuring
small academic gains—such as what would be required
for these types of studies
ii.Measures that are not valid for the use the
investigators intended
iii.For those studies containing observations and/or
rating scales, the measures were not specified or
provided in the studies. This means that we have no
way of evaluating the quality of the scales used
(i.e., item quality, item bias, technical adequacy, or
reliability/validity of the tool)
iv.How staff was trained before the study was started is
not mentioned.
v.Observations (pre and post intervention) were not
conducted in a systematic and reliable manner (or if
they were, no data are provided that would indicate
such thing). As a result, the data cannot be
sufficiently trusted.
vi.Subjects who were evaluated using these measures were
not generally randomized for selection or assignment
to control groups.
vii.No data are provided for how teachers (or those
providing pre-and-post test data) were trained in the
administration/scoring of the tests used to determine
outcomes.

These are just a few examples of the multiple problems
associated with
the “research” that has been discussed on the internet.
(Generally
speaking, the academic community most often regards
research that
exists only on the internet--without also being published
in scholarly
journals—as potentially unreliable reliable because it has
not gone
through the rigors of the peer-review process).

For those who are not familiar with research and how it is
conducted
and published: Investigators who conduct studies—and do so
well—are
continually motivated to submit their findings to
scholarly journals
for review, acceptance, and ultimate publication when
those studies
are deserving. Ethical researchers WANT to have their
studies
published for public review and consumption—they are
continually
motivated to educate others about their findings and want
to advance
the field. They want to educate others on “best practice.”
They
appreciate good research design and earn the respect of
colleagues and
the public by generating such research.

Even more skepticism…

What I found to be most suspicious about Brain Gym is the
preponderance of “research” cited only on the Brain Gym
website, or on
related sites purporting Brain Gym’s benefits. It is
extremely odd
that “Brain Gym research” studies are almost exclusively
published in
the “Brain Gym Journal,” which appears to not a peer-
reviewed or
respected journal within the research and academic
communities. This
journal, which is funded and published by Brain Gym
(another reason
for suspicion), is not publicly accessible (i.e., not in
most journal
databases or subscribed to by university libraries). If
you want to
read most of their “studies,” you must pay a fee. I find
it
incredibly “interesting” that one cannot read their
studies online, at
their website. What company/program would develop a
website for their
business—but not want to publicly provide hard data and
solid research
that describes the utility of the tools they are
promoting? Why would
one have to PAY to read information—advertised on their
website--that
supports the use of their own program and tools? This
makes me think
that the “research” contained inside those pricy documents
is
something they do not want viewed by anyone other than
desperate
individuals who do not understand the components of good
research.
Desperate parents and/or those unfamiliar with the rigors
of true
research come to mind…

Without question, if their research was truly useful and
of quality,
they would either be submitting it to peer-reviewed
journals so that
the masses could obtain copies and read about the efficacy
of such
interventions (and critique their research designs fairly
before
putting faith in the findings), or publishing it on their
website for
all to read. Why wouldn’t Brain Gym want to do this? It
would only
further promote their cause and enable their business to
grow! The
fact they hide their self-conducted “research” in a self-
published “journal” available only for purchase, thereby
limiting
public access, is deserving of our deepest concern and
skepticism. To
even further exacerbate the situation, there is
unfortunately plenty
of pseudo-research on the web that “sounds good” to those
who do not
have experience or training in quality research design.
The hype, the
testimonials, and the claims made are entirely
untrustworthy without
empirical evidence.

Even so, thousands of dollars are spent on the empty
promises Brain
Gym offers.

Posts on this thread, including this one

Brain Gym --where is the REAL research?, 1/29/05, by L.L..

Re: Brain Gym --where is the REAL research?, 6/21/05, by Dave Gow.

Re: Brain Gym --where is the REAL research?, 6/21/05, by LL.

Re: Brain Gym --where is the REAL research?, 7/21/05, by 10-a-C.

Re: Brain Gym --where is the REAL research?, 9/24/06, by mary mcbride.

Re: Brain Gym --where is the REAL research?, 9/24/06, by Penny.

Re: Brain Gym --where is the REAL research?, 9/26/06, by Michelle.

Re: Brain Gym --where is the REAL research?, 9/26/06, by sophie.

Re: Brain Gym --where is the REAL research?, 10/15/06, by J. Jennings.

Re: Brain Gym --where is the REAL research?, 11/13/06, by Charlotte.

Re: Brain Gym --where is the REAL research?, 12/08/06, by OTDrew.

Re: Brain Gym --where is the REAL research?, 12/08/06, by OTDrew.

Re: Brain Gym --where is the REAL research?, 12/08/06, by OTDrew.

Re: Brain Gym --where is the REAL research?, 12/05/09, by Robert.