| CIA: Action Research and Using Data |
Using data to make good decisions
A critical aspect of school reform efforts is using good, appropriate data to make decisions. It is at the heart of NCLB's emphasis on looking at disaggregated assessment data, and having high expectations for results. There is an expectation that educational practices are based on sound scientific data. The Iowa Professional Development Model begins with collecting and analyzing student data, and the use of data is critical at every step of the model.
As one might expect, with all of this emphasis on using data, there are a number of different approaches available to help educators in this process. A Google search for "data driven decision making in education" will result in over 800,000 hits, and a brief scan of the first few hundred shows that most of them are relevant. There is so much information available on using data that the knowing where to start can be overwhelming!
Another thing that makes this even more complicated is that there are two fundamentally different, but complimentary processes that can be used for gathering and using data. These two processes relate to the two basic approaches to reasoning: deductive and inductive processes. In the context of research, a deductive process is one in which we begin with theory, develop a hypothesis based on the theory, gather data to test the hypothesis, and end up with conclusions based on the data. The state of Iowa's Data Driven Leadership (DDL) process is an example of a deductive process. An inductive process is one that begins by gathering data, looking for patterns in the data, then developing tentative hypotheses and finally theory to explain the observed patterns. Inductive processes can be especially useful when a discovery process is called for, such as when one does not have enough information to form clear questions. Inductive processes are often seen in qualitative research.
A good description of these two approaches and how they relate to each other can be found in Bill Trochim's Knowledge Base, a very good on-line textbook on research methods for the social sciences. Click here for his discussion of deductive vs. inductive processes.
AEA staff are available to provide assistance to districts in understanding what kind of approach would be most useful for specific needs, and using both kinds of approaches in making educational decisions.
Action research
Another source of data to improve the quality of what we do in education is through action research. NCREL defines action research as "...inquiry or research in the context of focused efforts to improve the quality of an organization and its performance. It typically is designed and conducted by practitioners who analyze the data to improve their own practice."
An extensive collection of links to sites that deal with action research can be found at the University of Colorado at Denver. A brief journey through these links will show that there is no single "method" to be found in doing "action research," nor any clearly agreed upon definition. Concepts that emerge in the literature include the emphasis on using data for improvement, on reflection on practice, and on collaboration. One does not find an emphasis on adhering to specific kinds of research methodology, the use of statistics, or other similar things that would be emphasized in "traditional" research. There is much in common with program evaluation, which might be conducted for purposes that are similar to those of a practitioners engaging in action research.
What are the differences, then, between action research and "traditional" scientific research? This is a difficult question to answer. For one thing, despite the NCLB definition of "scientifically-based research," researchers recognize that there is no single method for conducting "scientific" research. There isn't even a single philosophical basis for research, which is why we find both qualitative and quantitative researchers.
In part, the difference between scientific research lies in the tools and methods used. "Formal" researchers use methods designed to ensure that their conclusions are valid and trustworthy, and include some formal methods that can provide answers to difficult questions, like whether one thing actually causes another to happen. Action research is about gathering data to see whether we are improving, and help us plan for changes to improve. Whether we have definitive evidence to conclude that doing "A" caused result "B" in students is less important than simply getting result "B." The measurement is aimed clearly understanding what results we are getting, rather than a complex design to determine causality.
Because the assumptions and purposes for "action research" differ from those of classical research, it may be unfortunate that they both include the word "research." This can be confusing to both traditional researchers and practitioners engaging in gathering data to make more informed decisions. Traditional researchers may view action research as sloppy research (which it is, if it is being used for the purposes of traditional research), and the danger for practitioners is to think that because they have engaged in reflective data gathering and use, their conclusions are as valid (using definitions from classical research) as they might have been with a carefully designed research study. There may be a tendency to over generalize the conclusions, and then it does becomes nothing more than sloppy research. We need to keep in mind that the fundamental purpose for action research is to gather useful data to make educational decisions in a specific context.
Some action research sites from around the world:
Refereed on-line journals for action research
- Educational Action Research A refereed international journal concerned with exploring the unity between educational research and practice. Site contains many full text articles
- Action Research International A refereed on-line journal of action research, sponsored by the Institute of Workplace Research Learning and Development (WoRLD) within the School of Social and Workplace Development (SaWD) and by Southern Cross University Press.
For more information, contact Dr. Jon McKenzie, Assessment Consultant
Last updated Tuesday, August 30, 2005
|
