In Part I, we examined a 2003 article that proposed 5 different “styles” that can influence how science is conducted and perceived.

Since Part II, we’ve been unpacking a long and stellar 2003 piece by Paula and Keith Stanovich, “Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research To Make Curricular & Instructional Decisions.”

Today in Part IV, we continue onward deeper into the article to examine the oh-so very science-y aspects of experimental design.

The Logic of the Experimental Method

So here’s two words that will cause an immediate and negative gut reaction to many involved in education: manipulation and control. Yet according to the Stanovichs, these sinister concepts are essential to experimental design:

“The heart of the experimental method lies in manipulation and control. In contrast to a correlational study, where the investigator simply observes whether the natural fluctuation in two variables displays a relationship, the investigator in a true experiment manipulates the variable thought to be the cause (the independent variable) and looks for an effect on the variable thought to be the effect (the dependent variable) while holding all other variables constant by control and randomization.”

It is interesting to think that there is a certain cold calculation to effective experimental design that may in fact present a barrier to executing more controlled studies in real schools. RCTs, for example, are incredibly difficult to do in schools because no one in a school in their right mind wants to withhold an intervention or resource from kids who are not currently receiving it that may be more effective. Which then makes having a control group really hard to sustain.

There’s a lot related to these concepts of manipulation and control that brings us back to causal inferences and the importance of attempting to falsify our hypotheses, which we briefly discussed in Part III. Frankly, I find all these seemingly simple concepts hard to fully grasp. It’s made me feel slightly better to see real scientists posting deep explorations of the complexity of making accurate causal inferences on my Twitter timeline.

I think in education we do have some grasp and application of these ideas. For example, concepts of team inquiry, problem-solving, and data-based decision-making are oriented around looking at student data and coming up with some kind of problem of practice and theory of change or hypothesis, then taking action steps to address it.

The Stanovichs point this out at the close of this paper, in fact:

Effective teachers engage in scientific thinking in their classrooms in a variety of ways: when they assess and evaluate student performance, develop Individual Education Plans (IEPs) for their students with disabilities, reflect on their practice, or engage in action research.

This assessment cycle looks even more like the scientific method when teachers (as part of a multidisciplinary team) are developing and implementing an IEP for a student with a disability. The team must assess and evaluate the student’s learning strengths and difficulties, develop hypotheses about the learning problems, select curriculum goals and objectives, base instruction on the hypotheses and the goals selected, teach, and evaluate the outcomes of that teaching. If the teaching is successful (goals and objectives are attained), the cycle continues with new goals. If the teaching has been unsuccessful (goals and objectives have not been achieved), the cycle begins again with new hypotheses.

Yet perhaps an area where we fall most short in terms of aligning more directly to the real scientific method is that we typically move confidently and brashly towards action without either strategically creating a group of students who do not receive the supports we think will be most effective, or considering in advance what evidence would both most clearly prove or falsify our theory of change. Instead, we often seem to default towards the notion that if we’ve taken action, we’ve achieved success. Everyone in education likes a good celebration, especially those in more political positions, but “confronting the brutal facts” in normed or standardized data seems to be a rarer exercise, other than as media headlines when the latest NAEP results come out. (For a model of an educator making this shift towards confront the brutal facts in literacy outcome data, please read this blog from The Right to Read Project).

In the tech world, A/B testing is a well-known and common undertaking. Maybe we need to keep simple study designs like this in mind and start smaller in our change endeavors. Maybe it’s not always about one whole-sail theory of change, but rather about having a few different theories that we can keep testing in a simple ways, with different students, classes, or groups. Or maybe the flipside is better? I’ve explored the idea that coherence within and beyond a school may actually matter more than “research-based” practices elsewhere, so it’s an open question as to whether putting a firm directive out and asking everyone to get on board (more typical), versus opening up a space for teacher teams to try out different ways to meet a goal, is ultimately more effective in impacting collective outcomes for students.

The Need for Both Correlational Methods and True Experiments

To this end, the Stanovichs highlight a great point that there’s a need for multiple approaches to experimental design. We then must look across findings from these divergent approaches for converging evidence:

It is necessary to amalgamate the results from not only experimental investigations, but correlational studies, nonequivalent control group studies, time series designs, and various other quasi-experimental designs and multivariate correlational designs, all have their strengths and weaknesses. For example, it is often (but not always) the case that experimental investigations are high in internal validity, but limited in external validity, whereas correlational studies are often high in external validity, but low in internal validity.

Convergence increases our confidence in the external and internal validity of our conclusions.

The Role of Case Studies and Qualitative Investigations

I found this section of the paper, in which the Stanovichs explain how case studies and qualitative studies operate in conjunction with quantitative research, especially fascinating.

In their explanation, case studies and qualitative investigations are most useful as explorations of a problem. Once a theory is more fully formed, however, ensuring it can be stress tested in full to “rule out alternative explanations” requires the “comparative information” provided by quantitative experiments.

Where qualitative investigations are useful relates strongly to a distinction in philosophy of science between the context of discovery and the context of justification. Qualitative research, case studies, and clinical observations support a context of discovery where, as Levin and O’Donnell (2000) note in an educational context, such research must be regarded as “preliminary/exploratory, observational, hypothesis generating.”

In education, however, investigators sometimes claim to be pursuing Objective B but slide over into Objective A without realizing they have made a crucial switch. They want to make comparative, or quantitative, statements, but have not carried out the proper types of investigation to justify them.

Case studies and qualitative description lack the comparative information necessary to prove that a particular theory or educational practice is superior, because they fail to test an alternative; they rule nothing out. Take the seminal work of Jean Piaget for example. His studies were critical in pointing developmental psychology in new and important directions, but many of his theoretical conclusions and causal explanations did not hold up in controlled experiments.

Woah.

My thinking is that while case studies can be a good way to get an investigation of a problem started, I suspect they can also serve as a great bookend to the journey as well. In other words, I suspect that once a theory has some solid backing from converging evidence, drawing up case studies and gaining fuller context of its application in the real world with qualitative description can also be really beneficial.

So: a qualitative-quantitative-qualitative sandwich! I don’t know, I’m not a researcher. But people seem to like case studies, so it seems like a good communication and educative tool, in addition to the beginning of an exploration.

Teachers and Researchers: Commonality in a “What Works” Epistemology

The Stanoviches close this tour-de-force exposition with some connections between the thinking necessary for science and how it can be leveraged by teachers:

Drawing upon personal experience is necessary and desirable in a veteran teacher, but it is not sufficient for making critical judgments about the effectiveness of an instructional strategy or curriculum.

Teachers need creativity, but they also need to demonstrate that they know what evidence is, and that they recognize that they practice in a profession based in behavioral science.

One of the pitfalls in teaching is that we often rely too heavily on personal observation, rather than systematic empiricism that can move us past our subjective assumptions and more accurately surface the underlying causes of what we see.

I suspect this is a why running records are such a strongly held practice, even in the midst of shifts to universal screening instruments and systematic phonics programs.

Yet the Stanovichs end on a hopeful note that there is a researcher-teacher partnership that can be forged:

Teachers believe in a world that is predictable and controllable by manipulations that they use in their professional practice, just as scientists do. Researchers and educators are kindred spirits in their approach to knowledge, an important fact that can be used to forge a coalition to bring hard-won research knowledge to light in the classroom.

This was a lot to unpack, and it sure took me some time to pick away at it, but I feel like I came away with a deeper understanding of research and how to consider it in the context of education.

The crazy thing is, I feel like there’s still a lot I just touched the surface of, or missed entirely. But I’ve got other things calling to me to write about, and it’s time to move on. Thank you, Keith and Paula Stanovich, for sharing your wisdom in these gems — and I hope, dear reader, I’ve inspired you to take either article up and read them, or re-read them, as the case may be, to dig into them on your own.

Over and out.

#research #Stanovich #science #scienceofreading #SOR #empiricism #reading

In Part I, we examined a 2003 article by Keith Stanovich that proposed 5 different “styles” that can influence how science is conducted and perceived. In that article, we learned that in education there may be a tendency to lean towards “coherence” in narratives or the “uniqueness” presented by silver bullet fads. These tendencies can and do subvert science-based reading practice.

In Part II, we began our analysis of yet another stellar 2003 piece by Paula and Keith Stanovich, which lays out the importance in drawing on the cumulative base of scientific findings on reading, rather than on gurus, personal agendas, and politics, as the field of education so often tends to. We learned that while peer reviewed research may not be a guarantee of quality, it is at the very least a minimum criterion that establishes such research as a part of the accumulating “public” realm of scientific knowledge.

Today in Part III, we continue onward with the article from Part II, “Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research To Make Curricular & Instructional Decisions,” as it is a lengthy one and there’s quite a bit more left to unpack. For example, the importance of an empirical approach to reading practice . . .

Research-based Practice Relies on Systematic Empiricism

There’s a lot of talk in education-related policy about research-based or evidence-based practice, but what does that mean? According to the Stanovichs, research-based practice is grounded in systematic empiricism.

What is empiricism? Empiricism is knowledge derived from evidence; this is the basis of the scientific method. All empirical theories must be tested by real world observations, rather than drawn from philosophical musing or heart felt intuition.

Basic, right? But the field of education is a realm as subject to political, bureaucratic, and ideological whims and sophistry as it is to taking action based on data from the students in front of us.

Empiricism thus starts with observation, but according to the Stanovichs, it’s more than that:

Observation itself is fine and necessary, but pure, unstructured observation of the natural world will not lead to scientific knowledge. . . Scientific observation is termed systematic because it is structured so that the results of the observation reveal something about the underlying causal structure of events in the world. Observations are structured so that, depending upon the outcome of the observation, some theories of the causes of the outcome are supported and others rejected.”

It’s worth unpacking the term “causal” (easily mistaken for casual by casual readers) here, as it’s one of those academic terms used frequently by researchers and not so frequently by teachers.

When we observe events in the real world, we primarily see the interwoven effects of many underlying factors. In order to disentangle and identify the specific causes of effects, researchers design tests to isolate variables that can allow them to make inferences—causal inferences—about complex and interrelated phenomenon.

When it is claimed that one form of instruction is better than another (e.g. phonics vs. whole word instruction) this is a causal claim that can be tested systematically and empirically.

When testing such claims using systemic empiricism and looking at the evidence base, there must be a space allowed for being wrong. According to the Stanovichs, this is called the “falsifiability criterion“:

A scientific theory must always be stated in such a way that the predictions derived from it can potentially be shown to be false.

A brief digression here on this concept of falsifiability: there are many within the “science of reading” camp—as with any other tribe in education—who can be overly strident in their claims about specific research-based practices. Examples are: sound walls, decodables, and multisensory instruction. Despite the coherence with existing converging evidence offered by each of these approaches, each has yet to be empirically proven as better than other approaches (I should add there is great debate about all of this, of course). This doesn’t mean they aren’t better — just that the peer reviewed evidence is not quite there to state more unequivocally that they are.

We’ve examined related controversy on phonological awareness instruction here on this blog in the past.

While this can be frustrating to those of us who wish to live in a clearly defined black-and-white world of proven approaches based on the “science of reading,” the deeper beauty of science is that it is completely agnostic as to anyone’s preferred outcomes (as with nature). The scientific truth lies on the ever shifting dune-face of peer reviewed evidence. This thus requires intellectual humility and the willingness to shift one’s beliefs in the face of a slowly accumulating evidence base.

Proponents of an educational practice should be asked for evidence; they should also be willing to admit that contrary data will lead them to abandon the practice. True scientific knowledge is held tentatively and is subject to change based on contrary evidence.

So what does this mean for teachers? It doesn’t mean we shouldn’t test out sound walls or use decodables or go all in on multisensory instruction — it means that when we do, to recognize them more humbly, and honestly, as tests of a hypothesis. Teachers are quite familiar with the phenomenon of what works one period with one group of students bombing completely with the next one. What works is dependent on any number of given variables. It’s certainly worth strategically trying different ones until outcomes can cumulatively demonstrate the intended effect. This is what science is all about.

Objectivity and Intellectual Honesty

Philosopher Jonathan Adler (1998) teaches us that science values another aspect of open-mindedness even more highly: “What truly marks an open-minded person is the willingness to follow where evidence leads. . . Scientific method is attunement to the world, not to ourselves.”

As the Stanovichs point out, scientists are also flawed human beings. They are individually no more objective than the rest of us. Yet, the greater endeavor of science countervails this fallibility with a “process of checks and balances.” This process is inherently social, in which scientists engage in peer critique of one another’s assumptions, biases, and conclusions.

“Purveyors of pseudoscientific educational practices fail the test of objectivity and are often identifiable by their attempts to do an “end run” around the public mechanisms of science by avoiding established peer review mechanisms and the information-sharing mechanisms that make replication possible. Instead, they attempt to promulgate their findings directly to consumers, such as teachers.”

This is an important caution to bear in mind. We have an over-abundance of “direct to consumer” products and pitches in our field. Furthermore, even when something is research-based via clinical studies, there is still the issue of translation and implementation in the complex and complicated world of real schools and classrooms.

Either way, in classrooms and schools we also need to rely heavily on our own peer review process: a social process of checks and balances delivered by our peers. Through dialogue and shared analysis of student data in teams, and via peer classroom intervisitations and feedback, we can get our assumptions and hypotheses checked.

The Principle of Converging Evidence

Research itself is hardly pristine, either. All individual studies are imperfect in their own way (and each should clearly outline their limitations) — but when taken collectively, they can provide robust conclusions.

“Scientists do not evaluate data from a single experiment that has finally been designed in the perfect way. They most often evaluate data from dozens of experiments, each containing some flaws but providing part of the answer.”

This idea of converging evidence is critical, because research takes time to conduct, write up, and publish, and it’s easy to get taken up with the latest finding and lose sight of a wider body of evidence, most especially ones that were established by a previous generation. And it’s furthermore difficult to connect the dots between different journals and bodies of knowledge. Like any human endeavor, there are geographical, institutional, and social networks and gaps between scientists that make cross-disciplinary connections over time harder to make. Some of the best peer reviewed articles provide a comprehensive overview of the extant research, and meta-analysis can also be invaluable for sifting through the bricolage and collate the wheat from the chaff.

This makes me think relatedly about how important it is to consider a variety of sources of information about our students — including talking directly to them and their families. I used to coordinate and write Individualized Education Plans (IEPs) for students in my building, and it was only after examining a students’ social history, multiple years of academic performance, multiple test scores across domains, considering the students’ behavior and performance across classrooms, examining writing and work samples, interviewing the student, and speaking with all of his or her teachers and service providers and family that I began to feel like I was getting somewhere in understanding their strengths and needs. One data point will tell you very little about a student. But with multiple forms of data, both qualitative and quantitative, we can tell a story from the converging evidence.

More to Come

Can you believe that we still aren’t done exploring Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research To Make Curricular & Instructional Decisions by Paula and Keith Stanovich?!

Nope. There’s that much good stuff in there to unpack.

Before we wrap up, let’s review what we’ve covered thus far since Part I:

• The way we think about and perceive the world influences how we think about science.
• Some ways of thinking about and perceiving the world lend themselves more readily to science.
• Having different lenses for viewing the world can also be beneficial to science and we need to be flexible to allow for the complexity of the world.
• Our beliefs are compelled by stories and we prefer to have our beliefs fit together coherently, but we need to also be able to change our beliefs to match the evidence. And scientists need to be able to tell better stories about what the evidence shows.
• We are also easily compelled by flashy, unique new findings that can present a magic bullet, rather than by a firm evidence base.
• Resolving disputes scientifically means being willing to resolve differences via mediated collaboration and peer reviewed, published findings.
• Teachers need to become more familiar with peer reviewed findings rather than rely on the self-proclaimed expertise of gurus.
• Peer review ensures a modicum of checks against pseudoscience and establishes the research as part of the realm of “public” knowledge.
• The peer reviewed evidence base needs to converge in its findings across multiple imperfect studies to become established as robust research-based practice.
• Systematic empiricism is the basis for testing hypotheses and establishing what is “research-based practice.”
• We need to have the flexibility and intellectual humility to change our beliefs based on the converging evidence.

To be continued in Part IV of our series. Thanks for joining me.

#Stanovich #science #scienceofreading #SOR #research #empiricism

Discuss...

In Part I, we examined a 2003 article by Keith Stanovich that proposed 5 different “styles” that can influence how science is conducted and perceived. In that article, we learned that in education there may be a tendency to lean towards “coherence” in narratives or the “uniqueness” of silver bullet fads. These tendencies can subvert science-based reading practice.

In Part II, we will look at yet another stellar 2003 piece by Paula and Keith Stanovich titled, “Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research To Make Curricular & Instructional Decisions.” A tip of the hat to the Reading League for this source.

In this dense and lengthy article, the Stanovichs examine how classroom teachers can leverage an understanding of the existing evidence base and of the scientific process at large to inform their practice. It’s so lengthy that I’m going to break it up into more than one post!

There’s a ream of great pull-quotes in this piece, such as the following:

“As professionals, teachers can become more effective and powerful by developing the skills to recognize scientifically based practice and, when the evidence is not available, use some basic research concepts to draw conclusions on their own.”

I will readily admit that I am not very scientifically literate. The methods section of a research paper makes my eyes glaze over, and you’ve lost me at “dependent variable.” As a result, I find much of what is put forward by the Stanovichs in this paper enlightening. They outline what I’m sure are basic stuff for some, but I appreciated these fundamentals, and I also noticed that the authors suggest — here and in their other paper — that scientists themselves can have trouble with these principles.

In the introduction to the paper, the Stanovichs make a strong argument for the need for greater scientific understanding amongst teachers. It is a matter, they claim, of “professional autonomy,” as in an environment in which “anything goes,” the result is “a fertile environment for gurus to sell untested educational ‘remedies’ that are not supported by an established research base.”

Yep. The field of education is most definitely a fertile environment for both self-proclaimed and widely venerated “gurus.”

“The ‘anything goes’ mentality actually represents a threat to teachers’ professional autonomy. It provides a fertile environment for gurus to sell untested educational “remedies” that are not supported by an established research base.”

Education has therefore become run by political disputes and personal agendas rather than scientific results:

“A vast literature has been generated on best practices that foster children’s reading acquisition…Yet much of this literature remains unknown to many teachers, contributing to the frustrating lack of clarity about accepted, scientifically validated findings and conclusions on reading acquisition.”

“The field’s failure to ground practice in the attitudes and values of science has made educators susceptible to the ‘authority syndrome’ as well as fads and gimmicks that ignore evidence-based practice.”

The Stanovichs therefore recommend training teachers to apply some basic scientific criteria for evaluating knowledge, which “could easily be included in initial teacher preparation programs.”

These criteria include:

1. “the publication of findings in refereed journals (scientific publications that employ a process of peer review),
2. the duplication of the results by other investigators, and
3. a consensus within a particular research community on whether there is a critical mass of studies that point toward a particular conclusion”

These may seem disarmingly simple, but the reality is that even the basic gate of evaluating current evidence on reading practice from the standpoint of peer review could do a lot to clear up a few misconceptions out there.

Publicly Verifiable Research Conclusions

We know the peer review process ain’t bullet proof, but it is a minimal guarantee that information will not be pseudoscience.

“Peer review is a minimal criterion, not a stringent one.”

Furthermore — and here I think the Stanovichs illuminate what is the most powerful aspect of peer review — it removes the status of any “special” knowledge attributed to such “gurus” aforementioned.

“Research-based conclusions, when published in a peer reviewed journal, become part of the public realm, available to all, in a way that claims of “special expertise” are not.”

What the Stanovichs mean by “public” is not accessibility to the general public so much as that the research has been published as part of the greater effort of scientific consilience to build a body of knowledge that can be tested and verified.

Many educators believe that knowledge resides within particular individuals—with particularly elite insights—who then must be called upon to dispense this knowledge to others. . .

Science, however, with its conception of publicly verifiable knowledge, actually democratizes knowledge. It frees practitioners and researchers from slavish dependence on authority.

Science can democratize knowledge. What a beautiful idea. It can free us from the Lucy Calkinses of the ed world and allow any and all to understand it — so long as we make the effort to grasp and verify it.

“Empirical science, by generating knowledge and moving it into the public domain, is a liberating force. Teachers can consult the research and decide for themselves whether the state of the literature is as the expert portrays it.”

This made me think of the work of journalist Emily Hanford, who has done our field the invaluable service of synthesizing decades of reading science into more readily digestible forms for the general public and for educators. Interestingly, her name has become almost synonymous with “the science of reading,” and, like the term, can raise a few hackles. But don’t read her as an “expert” on the matter. Review the extensive footnotes provided by her pieces–and beyond–and then decide for yourself.

But how do we know, beyond peer review, what makes something “research-based”? What the heck is science, anyway?

We’ll examine the Stanovich’s answers to these questions in Part III! Stay tuned.

#reading #research #literacy #empiricism #peerreview #science #scienceofreading #SOR #Stanovich

Discuss...

For some, the term elicits eager head nodding — it’s even become incorporated into the sales pitch of many a vendor of education products. For others, the term elicits a gut reaction akin to disgust.

There’s a lot wrapped up in how someone may think of “science” at large that then influences their reactions to the term of the “science of reading.” But don’t just take my word for it. Keith and Paula Stanovich penned some really insightful pieces about this in the early 2000s, and outlined how educators can understand and leverage science to inform their own instructional practice. I’m going to tackle two pieces by them in two different blog posts. In Part I here, we will tackle the “Styles of Science,” from a 2003 piece by Keith Stanovich titled “Understanding the Styles of Science in the Study of Reading” in the Scientific Studies of Reading.

A warm hat tip to Chris Schatschneider for this article!

In this article, Stanovich lays out 5 “styles” of science that colors how the overall term is understood.

1. Correspondence vs Coherence
2. Analytic Reductionism vs Holism
3. Probabilistic Prediction vs Case-based Approach
4. Robust-Process Explanations vs Actual-Sequence Explanations
5. Consilience vs Uniqueness

Not the pithiest breakdown in the world, but there’s some great quotes in here, as well as some useful frames for understanding perspectives on science.

Correspondence vs Coherence

This may be the most useful distinction.

A correspondence view is the truly scientific one: it means using objective data to form and test theories, which is the basis for the scientific method. A coherence view, on the other hand, is our innate predilection to create and respond to compelling narratives about the world.

When it comes to reading research, consider how initial theories were coherence based — they established a narrative that fit the observational data that reading is primarily a “visual” skill in which whole words are recognized. But counterintuitively, empirical testing that better corresponded to reading has clearly demonstrated that reading is primarily a rapid connection of letter and letter-groups to their sounds.

Ken Goodman’s entire oeuvre was based on a coherence approach — and his narratives continue to retain a strong grip on educators. And arguments about advanced phonemic awareness continue apace as we speak — much of which is currently based more on an overarching theory and synthesis of research rather than on corresponding empirical studies.

On what a “correspondence” perspective entails:

“Simply, there is a real world out there that exists independently of our beliefs about it, researchers form theories about this world, and the theories that track the world best are closer to the truth and are thus a better basis for action. This is why planes don’t fall out of the sky, why bridges rarely collapse, and why my headache medication works more often than not.”

On what a “coherence” perspective entails:

“many in the qualitative research communities emphasize constructivist principles that put stress on the requirement that beliefs fit together in a reasonably logical way—the so-called coherence theories of truth.”

. . . “numerous authors have written about how the coherence doctrine, by linking itself with ecumenical notions such as tolerance and personal validation, obscures its uglier aspects. What has been obscured is how indiscriminate belief validation, with no check in external reality, creates a world that most of us would consider a nightmare. In this world, the witnesses and evidence in a jury trial are not sifted as to credibility because any piece of evidence put forward is equal to any other for the reason that all are valid by someone’s perspective.”

And on the resulting clash between these two views:

“. . . an extreme adherence to a correspondence theory of truth often necessitates the frustration of the strong human need for narrative coherence in explanation.”

“the explanatory frameworks that are generated by scientists working in the correspondence framework may not seem plausible to those who value coherence more, particularly the type of coherence that resonates with the narratives inherent in folk psychology.”

Analytic Reductionism vs Holism

While Stanovich makes it clear that correspondence more clearly aligns with science than coherence, some of these other styles outlined rely on a healthy balance, such as here in analytic reductionism vs holism. Both views have their downfalls, and both are needed as a complement to the other.

Analytic reductionism means breaking things down into small parts that can be studied individually — this is the bread and butter of empirical research. Holism, on the other hand, means viewing small parts within their greater context and whole. I’ve spent a fair amount of time arguing the need to view individual schools from a more holistic view, and I currently spent much of my time (in my day job) arguing about the need to view individual student data from a more holistic view, as well. But as Stanovich argues here, the translation of reading science to the classroom benefits from both approaches applied judiciously:

“Gone are the days when such investigations were couched as if comparing a disembodied mind interacting with a disembodied orthography. Investigators in this area appreciate the necessity of adding the learning environment and instructional context as interacting factors in the model of orthography and achievement links that is being developed. This area displays an additive holism rather than the subtractive holism that has soured so many scientists on that end of the analytic and holistic continuum.”

Probabilistic Prediction vs Case-based Approach

I found this distinction really useful as well, and very much related to the correspondence vs coherence dynamic. Researchers who conduct empirical studies will be more accustomed to probabilistic thinking, but for most of us laypeople, we are compelled by case studies.

“We in the behavioral science of the study of reading are so accustomed to probabilistic explanation and prediction that we are prone to forget how alien it seems to the layperson, to teachers, and indeed to some working in our own field.”

Stanovich makes a useful distinction in terms of understanding reading research, which is to understand that probabilistic prediction applies to aggregate data and decision-making, and that outlier cases will always be found — which can lead to some educators writing off the implications of aggregate data.

That said, both approaches have their utility and need to be wielded strategically:

“In many cases though, in the complicated field of reading—in its domains of application—it really is unclear whether we should be in probabilistic or case-based mode”

Robust-Process Explanations vs Actual-Sequence Explanations

This is another one in which both styles have an important function in translating reading science into practice. Robust-process explanations are based on theoretical principles which can apply across different examples, whereas actual-sequence explanations apply to specific cases that have happened.

“Those of us studying the psychology of the reading process are often after robust-process explanations, whereas we often address audiences who are interested in and oriented toward actual-sequence explanations. Teachers often want to know how this particular child reached this level of school achievement or this level of reading difficulty, as the case may be.”

Both of these are useful ways of getting at the truth.

“By subdividing robust-process explanations we get closer to actual-sequence explanations, and by aggregating actual-sequence explanations we get closer to a robust-process explanation. The two work in concert.”

Consilience vs Uniqueness

E.O. Wilson defines consilience as the “unification of knowledge by the linking of facts and fact-based theory across disciplines to create a common groundwork of explanation.” Such efforts are important in reading research, as robust theories draw upon different fields such as “connectionist modeling, cognitive neuroscience, and classroom studies of effective practice.”

Uniqueness, on the other hand, refers to a tendency to look for the flashy new thing that stands out and excites people.

“This concern for consilience contrasts with the faddish tendencies in the field of education to search for magic bullets and miracle cures deriving from theories that do not cohere with the knowledge being developed by allied disciplines. The quest for a magic bullet always tempts education to stray from valuing consilience.”

Like correspondence vs coherence, Stanovich positions consilience as far more firmly on the side of science.

What styles are scientific?

Somewhat unscientifically, since none of these “styles” are based on quantitative research, Stanovich guesstimates how these different styles can be applied in varying ways and still be considered scientific. As you can see, the two styles that present that greatest challenge, in his view, to the application of reading science are the tendency of many in the education field to lean into coherence and uniqueness in their views of reading.

That said, Stanovich is not trying to enforce a rigid perspective of science here — he believes these other styles can work together, and he also warns against rigidity — which is a warning I think many who would consider themselves “Science of Reading” folks would do well to heed:

“I do fear that some would prefer rigid rules and bullet points on a PowerPoint presentation rather than the story of science in its full complexity—including the complexities of certain styles that are neither right nor wrong but represent continua (better viewed as parameters that we are constantly adjusting so as to facilitate the process). Science is a delicate epistemological game.”

“Science’s real uniqueness comes from its self-correcting nature. Its unique epistemic power comes from a very un-Promethean characteristic: its constant fiddling with things—with theory, experimental setups, techniques, and its styles of the type I have discussed. And we are not afraid to readjust—which itself, recursively, is one of science’s characteristic features—we are not afraid to implicitly admit previous error when we make a readjustment.”

“I fear that this flexibility, this juggling, this self-corrective mindset will be lost if we too rigidly reduce science to a set of rules. Do not misunderstand me, though—I am wholeheartedly in favor of instructing teachers and other educational personnel in what science is and is not and what are the unique features that underlie its epistemic power. But these features should not become a prison.”

There’s been a lot of recent “scholarly disputes” about reading research, and Stanovich raises a procedure that would be nice to see used more: “adversarial collaboration.” In this process, the disputants agree to an arbiter, who then works with the disputants to design experiments that can test the theories under dispute. This research is then published by the arbiter with the disputants as co-authors.

What a brilliant way to tackle some of the current debates about phonemic awareness!

“In the middle of the heat of the internecine warfare that sometimes takes place in our field, it is important to be able to differentiate legitimate scientific criticism that derives from background assumptions from the opposite ends of the style dimensions I have discussed here—and criticisms deriving from critics who are not playing the game of science at all.”

“Willingness to accept offers of adversarial collaboration might be a tool to use in distinguishing who is playing science from who is not.

This approach of adversarial collaboration has been used recently to advance the research into cognition. Why don't we do more of this in literacy research?

#Stanovich #research #empiricism #literacy #reading #science #scienceofreading #SOR

Discuss...