07/27/20010
Over half the states have now agreed to implement the Common Core standards as their own. I think it is now time to think about how quickly we can expect these standards to be met.
We are told that Common Core is a high quality, rigorous set of standards for K-12 education, far more challenging and explicit than what most states have been using as their own standards. I am going to accept that that is true.
If states could immediately begin to implement Common Core in the classroom (i.e. if we had the curriculum, unit plans, materials and lesson plans ready for September), there would still be problems. There is no way that today’s students are ready for those more rigorous standards. Today’s 12th graders obviously are not up to the high bar that Common Core sets for 11th grade work, because last year they were only operating at the level set by their states’ lower standards. So, next year’s graduates will not be up to the level of Common Core.
Similarly, next year’s 11th graders are not ready for Common Core’s highly rigorous 11th grade work, because their 10th grade work was only at the level of their states’ less rigorous old standards. And if they can’t do 11th grade Common Core level work this year, then they will not be able to do Common Core 12th grade work the following year. So, the high school graduating cohort of 2012 will not be up to Common Core’s standards.
By induction — yes, even we former ELA teachers can resort to math and logic every now and then — this means that it will be 13 years until we get Common Core graduates. After all, how can 4th graders do Common Core level work without having been prepared up through Common Core’s highly rigorous grade 3 standards?
And let’s be honest. This work can’t begin until we have curricula based on the Common Core standards. And then we will need unit plans, materials and lesson plans. That’s going to take a couple of years at least, especially if we consider that teachers themselves will need to get used to their new curriculum, units, materials and lessons. That means it will be more like 15 or more years before we get Common Core level high school graduates.
*****************
Some might argue (i.e. some surely will argue) that it won’t take 13 years for everyone to catch up. They will argue that given a few years, we can raise everyone to the Common Core level — a level on which they can then proceed at Common Core’s pace.
This means that people think that for the next few years, students and educators will work extra-hard to get up to the Common Core standard. Sure, they may be well behind this year, but next year they will be less behind, and the year after that a bit less behind Common Core’s levels. And eventually, they will catch up.
In other words, for the next few years, we would expect all students to do more than a full year of learning — as Common Core defines it. This means that September’s 8th graders (i.e. students and educators, both) will have to work especially hard, because these kids and their teachers have been slacking for 8 years already (i.e. kindergarten and then grades one through seven). And we probably expect those who are not even in high school next year to be able to graduate from high school passing the Common Core aligned tests in five years.
I am sorry, but I think that this is crap. Total crap. Here’s why:
* If students and educators truly are capable of working at such a pace as to be able to catch up like that, then Common Core is not that rigorous. If they are capable of working at such a pace then Common Core coddles them, allowing them to engage in far less learning per year than they are capable of achieving. If we can get next year’s 8th graders to meet Common Core’s complete high school standards in just five years, either Common Core’s K-7 standards are irrelevant or its high school standards are far too simplistic. I have yet to hear anyone make that claim, even defenders of the most rigorous preexisting state standards.
• If today’s 8th graders can learn the 60-75% more per year through high school that catching up to a Common Core would require (assuming that Common Core demands 25% more learning each year), why would our rigorous new standards not demand that greater amount of learning? If students and educators can work at the pace for the years we need to catch up to Common Core, shouldn’t they be able to work at that pace even after everyone has caught up?
* Some might argue that most students are already quite close to Common Core’s grade-by-grade standards. But those people would be saying that Common Core is really just a tweak, and not really a big reform that does not ask for very much to be changed. Or…
* Some might argue that most students perform well above their state’s current standards, and Common Core is just aimed at those few students, schools or districts who do not. These people would be granting that state standards are irrelevant to most students and educators, who already operate well above them.
And yet, Race to the Top is being used to strong-arm states into adopting Common Core. Lots of money is going into developing Common Core aligned assessments (i.e. tests). And both politicians and the public expect these tests to be passed soon, and will have no tolerance for talk of waiting fifteen years for high passing rates. But if it will be possible for students to catch up to Common Core’s standards in just a few years, then Common Core’s standards do not push students and educators nearly as hard as they could take, meaning the standards are not really that rigorous – certainly not much more rigorous than existing standards.
What do you think is going to happen? Assuming that you try to adopt a more thoughtful approach to examining Common Core as a powerful lever for school reform, what do you expect?
Tuesday, April 9, 2013
Charter Schools Are Still Not Public Schools
03/31/2010
Last week, I explained why I no longer think that charter schools are public schools and asked for comments from GothamSchools readers. I’ve given a lot of thought to the ideas that others have presented.
First, no one has come close to rehabilitating the argument that charter schools are public schools simply because they accept public funds. Many organizations have their operations paid for — in whole or in part — by public funds, and not all of them are public in the way that “our public schools” are. If charters are public schools, this is not why.
Second, I raised the issue of democratic accountability. To what degree do elected officials and their appointees have authority over arbitrary aspects of charter school operations and staffing? For example, years ago Mayor Bloomberg required all schools to hire parent coordinators. Under mayoral control, Bloomberg can mandate curriculum and spending decisions, and any spending not controlled by existing contracts. Generally, elected officials and their appointees can even remove principals and other administrators for arbitrary — though not discriminatory — reasons. (Because New York City principals have a union contract, this authority is severely constrained. But this is unique to the city and could be negotiated out of the contract.)
There have been many arguments raised against this point, but they generally fall into two camps. One was that there are other public institutions that are led by people who cannot be so removed, but these responses have been based on admitted ignorance (e.g. how are members of the NY Board of Regents appointed and how can they be removed?, what are the laws regarding removal of charter board members? etc.). The other response has been that elected officials and their appointees can pressure charter schools to remove a principal. My understanding is that this pressure only comes in the form of withdrawing a school’s charter. I am sorry, but I do not think this argument works. By this argument, major accounts of a private company have the authority to fire salesmen at that company because they can threaten to move their account if the company does not accede to their demand. Going back to one of my examples, the private construction firm who builds schools for a district could be threatened with losing the job if some kind of site supervisor is not fired. That does not make the company part of a district or a public entity.
To state this more plainly: Demands that only have power if backed by the threat to pull a charter or contract do not equate to the sort of ongoing democratic oversight that fits into to my understanding of public schools
Third was my most important criteria, the obligations to educate all comers. One reader, Gideon, wisely pointed out that this would imply that our so-called public universities would be excluded. I do not have a problem with that. I think that the distinction between public and private higher education is rather thin and am therefore happy to say that public universities are not public at all like our public schools are.
Another response, perhaps the most common one, has been that many public schools do not serve all comers. There are exam schools, for example. And the kind of school choice model we have in New York City means that few, if any, schools serve all comers. However, I anticipated much of this argument before writing last week. School districts have a responsibility to educate all comers. It is our public schools who collectively – in the form of districts – meet that obligation. District offices are obliged to figure out how to do this and may not direct children to charter schools as part of their solutions
Let me be clear: Charter schools are not district schools. Virtually the entire purpose of charter schools is to free them from districts and their authority. Any argument that states that charter schools are part of the public schools because they are part of the mélange of schools that educate our children applies equally to unquestionably private schools. If those who advance this argument cite the use of public funds, they would have to claim that private schools that accept publicly funded vouchers are also public schools, an argument that I do not think they want to make.
However, I am also willing to concede that in most meaningful ways, selective so-called public schools are really not public schools. And I would further say that meaningful public status is certainly questionable for any school that students cannot attend simply by following the standard normal procedures that all students/families must follow — including my own high school.
Ken Hirsh and others have raised the point that charter schools are subject to government oversight, including inspections and perhaps various well or lesser known state and federal legislation. The mere fact of regulation and inspection however, does not a public entity make. Meatpacking plants are subject to federal inspection. Restaurants are subject to government inspection. Most organizations are subject to regulation in one form or another, though the degree of regulation often varies from industry to industry.
In fact, there are many regulations that only apply to those who receive public funds. For example, the City of New York enforces much of its own legislation by requiring compliance as a condition of contracting with the city. The fact of regulation does not make these entities public.
*****************
So, what am I getting at? I think that public schools must be both responsive to and responsible for the public.
There is no question that charter schools — like many private organizations and entities — are somewhat responsible to the public (as expressed in the form of democratic government). They certainly are more responsive to the public than traditional private schools, but it is not at all clear that they are more responsive than other private entities (i.e. other than traditional private schools) that accept large portions of their operating budgets from the government. By design, they are less responsive than traditional public schools, even if they are more responsive than traditional private schools
Clearly many charter school operators do personally feel responsible for the public and its children. Many charter school leaders work hard to build a school culture that will outlive them and that is infused with that sense of responsibility. I, therefore, understand why some who work in charter schools think of their schools as public schools. However, they build this culture voluntarily; it is not intrinsic to charter schools generally or even a requirement of their charters. The very fact that they are only required to select a student body from among those who apply in the first place makes for a qualitative difference from public schools. Districts cannot place additional students in charter schools when all district schools are overcrowded, nor can they enroll students whose families failed to take part in the normal school selection process in charter schools. In this respect, charter schools are more like traditional private schools than they are like traditional public schools.
And so, while charter schools are clearly not traditional private schools, by design they are not like traditional public schools, either. Even if we acknowledge that there are differences between different charter schools, and between charter school laws, neither of these terms seem appropriate. Those who insist that they are “public schools” or “private schools” clearly have some sort of agenda and some idea other than a full examination of the meaning these terms carry. This leaves us with a need for a third term, as neither “public” or “private” would be appropriate.
Luckily, we already have the term “quasi-public” from other sectors. I do not love this term — or even really like it — but it is surely better than either of the others.
Last week, I explained why I no longer think that charter schools are public schools and asked for comments from GothamSchools readers. I’ve given a lot of thought to the ideas that others have presented.
First, no one has come close to rehabilitating the argument that charter schools are public schools simply because they accept public funds. Many organizations have their operations paid for — in whole or in part — by public funds, and not all of them are public in the way that “our public schools” are. If charters are public schools, this is not why.
Second, I raised the issue of democratic accountability. To what degree do elected officials and their appointees have authority over arbitrary aspects of charter school operations and staffing? For example, years ago Mayor Bloomberg required all schools to hire parent coordinators. Under mayoral control, Bloomberg can mandate curriculum and spending decisions, and any spending not controlled by existing contracts. Generally, elected officials and their appointees can even remove principals and other administrators for arbitrary — though not discriminatory — reasons. (Because New York City principals have a union contract, this authority is severely constrained. But this is unique to the city and could be negotiated out of the contract.)
There have been many arguments raised against this point, but they generally fall into two camps. One was that there are other public institutions that are led by people who cannot be so removed, but these responses have been based on admitted ignorance (e.g. how are members of the NY Board of Regents appointed and how can they be removed?, what are the laws regarding removal of charter board members? etc.). The other response has been that elected officials and their appointees can pressure charter schools to remove a principal. My understanding is that this pressure only comes in the form of withdrawing a school’s charter. I am sorry, but I do not think this argument works. By this argument, major accounts of a private company have the authority to fire salesmen at that company because they can threaten to move their account if the company does not accede to their demand. Going back to one of my examples, the private construction firm who builds schools for a district could be threatened with losing the job if some kind of site supervisor is not fired. That does not make the company part of a district or a public entity.
To state this more plainly: Demands that only have power if backed by the threat to pull a charter or contract do not equate to the sort of ongoing democratic oversight that fits into to my understanding of public schools
Third was my most important criteria, the obligations to educate all comers. One reader, Gideon, wisely pointed out that this would imply that our so-called public universities would be excluded. I do not have a problem with that. I think that the distinction between public and private higher education is rather thin and am therefore happy to say that public universities are not public at all like our public schools are.
Another response, perhaps the most common one, has been that many public schools do not serve all comers. There are exam schools, for example. And the kind of school choice model we have in New York City means that few, if any, schools serve all comers. However, I anticipated much of this argument before writing last week. School districts have a responsibility to educate all comers. It is our public schools who collectively – in the form of districts – meet that obligation. District offices are obliged to figure out how to do this and may not direct children to charter schools as part of their solutions
Let me be clear: Charter schools are not district schools. Virtually the entire purpose of charter schools is to free them from districts and their authority. Any argument that states that charter schools are part of the public schools because they are part of the mélange of schools that educate our children applies equally to unquestionably private schools. If those who advance this argument cite the use of public funds, they would have to claim that private schools that accept publicly funded vouchers are also public schools, an argument that I do not think they want to make.
However, I am also willing to concede that in most meaningful ways, selective so-called public schools are really not public schools. And I would further say that meaningful public status is certainly questionable for any school that students cannot attend simply by following the standard normal procedures that all students/families must follow — including my own high school.
Ken Hirsh and others have raised the point that charter schools are subject to government oversight, including inspections and perhaps various well or lesser known state and federal legislation. The mere fact of regulation and inspection however, does not a public entity make. Meatpacking plants are subject to federal inspection. Restaurants are subject to government inspection. Most organizations are subject to regulation in one form or another, though the degree of regulation often varies from industry to industry.
In fact, there are many regulations that only apply to those who receive public funds. For example, the City of New York enforces much of its own legislation by requiring compliance as a condition of contracting with the city. The fact of regulation does not make these entities public.
*****************
So, what am I getting at? I think that public schools must be both responsive to and responsible for the public.
There is no question that charter schools — like many private organizations and entities — are somewhat responsible to the public (as expressed in the form of democratic government). They certainly are more responsive to the public than traditional private schools, but it is not at all clear that they are more responsive than other private entities (i.e. other than traditional private schools) that accept large portions of their operating budgets from the government. By design, they are less responsive than traditional public schools, even if they are more responsive than traditional private schools
Clearly many charter school operators do personally feel responsible for the public and its children. Many charter school leaders work hard to build a school culture that will outlive them and that is infused with that sense of responsibility. I, therefore, understand why some who work in charter schools think of their schools as public schools. However, they build this culture voluntarily; it is not intrinsic to charter schools generally or even a requirement of their charters. The very fact that they are only required to select a student body from among those who apply in the first place makes for a qualitative difference from public schools. Districts cannot place additional students in charter schools when all district schools are overcrowded, nor can they enroll students whose families failed to take part in the normal school selection process in charter schools. In this respect, charter schools are more like traditional private schools than they are like traditional public schools.
And so, while charter schools are clearly not traditional private schools, by design they are not like traditional public schools, either. Even if we acknowledge that there are differences between different charter schools, and between charter school laws, neither of these terms seem appropriate. Those who insist that they are “public schools” or “private schools” clearly have some sort of agenda and some idea other than a full examination of the meaning these terms carry. This leaves us with a need for a third term, as neither “public” or “private” would be appropriate.
Luckily, we already have the term “quasi-public” from other sectors. I do not love this term — or even really like it — but it is surely better than either of the others.
Are Charter School Public Schools? I’m Afraid Not.
03/26/2010
In the past, I have been very careful to talk and write about “charter schools” and “traditional public schools,” the latter of which is often abbreviated TPS. I have even tended to favor others’ claims that charter schools are, in fact, public schools. But I’m afraid that I was wrong. I have tried to be more thoughtful about this question and I simply cannot find a compelling argument that they are, and can find too many that they are not.
First, let’s acknowledge what charter schools are. They are publicly financed schools that are run by private – usually nonprofit – organizations. Sometimes they are independent, and sometimes they are part of larger charter school organizations or chains.
The primary argument that charter schools are public schools is that they are paid for out of government funds. While they do get most of their budgets from tax dollars, that is not enough to render them public schools. There are many other organizations that pay for operations with public funds but are still private organizations. Defense contractors receive enormous sums of money from the government to provide design and manufacturing of weapons systems, but they remain private corporations. Blackwater provided labor, training and services to the Department of Defense and the State Department, but it remained a private organization.
If a construction firm is hired by a school district to build a school, it remains a private firm. If a new firm is formed to bid for a school construction job, and wins the project, it still remains a private firm. Even if that firm does such a good job that it wins future bids and does all the district’s construction work, it remains a private firm.
Frankly, I’ve not heard any other arguments that charter schools are public schools. Meanwhile, there are lots of ways in which they most definitely are not public schools
I think that it is pretty clear that at least some level of oversight of the day to day operations of our public agencies must rise up to our elected officials. Public agencies and offices must have some level of democratic oversight. Abuses, mismanagement and bad policy are subject to review by elected officials or those they appoint. Policies can be changed, budgets cut and/or senior personnel removed as a direct consequence of this oversight. This is quite different than contracted services. So long as the terms of the contract are met, the government cannot reach into the management of a contractor and force changes. They may try to embarrass the contractor, but they do not have authority to require changes. On the other hand, public schools are accountable to elected school boards, legislatures and/or mayors. There may be changes in who is ultimately responsible for a district, but it always is an elected official.
Charter school principals cannot be removed by elected officials. Their board members are not subject to removal by public elections. The executives of charter management organizations are not accountable to the government for their jobs.
More important, however, is the difference in moral mission. It is the responsibility of the public schools to educate every child who shows up. All children who live in a school district have a right to attend a district school. Furthermore, no public school can in good conscience “counsel out” a student. Private schools are well known to engage the practice of “counseling out” when a student does not seem to fit in or is too disruptive or the school believes that it cannot well meet that student’s needs. As the student has the public schools to fall back on, the moral import of this practice is surely debatable. But the public schools must find another placement for students whose needs they cannot meet, because they – in the form of the district – have a moral and a legal obligation to educate every child that shows up.
Charter schools do not have that obligation, either legally or morally. To the extent that many charter schools are oversubscribed, it would be difficult or impossible for them to do so. While the public schools have to cram in more students – hopefully, eventually, leading to more classrooms and even schools – charter schools only have to serve as many students as they specify. Charter schools are free to say that they do not offer support services for English language learners or autistic children, but the public schools must provide schooling for every child. Charter schools are free to “counsel out” students.
Charter school employees do not work for the government; they are not public employees. While the government has contracted with charter schools to provide a service, they do not act as the government when the provide it. Their operations are not subject to democratic or public oversight; rather their contracts (i.e. their charters) come up for review for possible extension periodically.
If you can make a more thoughtful argument about why charter schools are public schools, I would love to hear it. If you agree or disagree, please share your own thinking as to why.
[Note: I have posted a follow up piece, and the conversation has continued there.]
In the past, I have been very careful to talk and write about “charter schools” and “traditional public schools,” the latter of which is often abbreviated TPS. I have even tended to favor others’ claims that charter schools are, in fact, public schools. But I’m afraid that I was wrong. I have tried to be more thoughtful about this question and I simply cannot find a compelling argument that they are, and can find too many that they are not.
First, let’s acknowledge what charter schools are. They are publicly financed schools that are run by private – usually nonprofit – organizations. Sometimes they are independent, and sometimes they are part of larger charter school organizations or chains.
The primary argument that charter schools are public schools is that they are paid for out of government funds. While they do get most of their budgets from tax dollars, that is not enough to render them public schools. There are many other organizations that pay for operations with public funds but are still private organizations. Defense contractors receive enormous sums of money from the government to provide design and manufacturing of weapons systems, but they remain private corporations. Blackwater provided labor, training and services to the Department of Defense and the State Department, but it remained a private organization.
If a construction firm is hired by a school district to build a school, it remains a private firm. If a new firm is formed to bid for a school construction job, and wins the project, it still remains a private firm. Even if that firm does such a good job that it wins future bids and does all the district’s construction work, it remains a private firm.
Frankly, I’ve not heard any other arguments that charter schools are public schools. Meanwhile, there are lots of ways in which they most definitely are not public schools
I think that it is pretty clear that at least some level of oversight of the day to day operations of our public agencies must rise up to our elected officials. Public agencies and offices must have some level of democratic oversight. Abuses, mismanagement and bad policy are subject to review by elected officials or those they appoint. Policies can be changed, budgets cut and/or senior personnel removed as a direct consequence of this oversight. This is quite different than contracted services. So long as the terms of the contract are met, the government cannot reach into the management of a contractor and force changes. They may try to embarrass the contractor, but they do not have authority to require changes. On the other hand, public schools are accountable to elected school boards, legislatures and/or mayors. There may be changes in who is ultimately responsible for a district, but it always is an elected official.
Charter school principals cannot be removed by elected officials. Their board members are not subject to removal by public elections. The executives of charter management organizations are not accountable to the government for their jobs.
More important, however, is the difference in moral mission. It is the responsibility of the public schools to educate every child who shows up. All children who live in a school district have a right to attend a district school. Furthermore, no public school can in good conscience “counsel out” a student. Private schools are well known to engage the practice of “counseling out” when a student does not seem to fit in or is too disruptive or the school believes that it cannot well meet that student’s needs. As the student has the public schools to fall back on, the moral import of this practice is surely debatable. But the public schools must find another placement for students whose needs they cannot meet, because they – in the form of the district – have a moral and a legal obligation to educate every child that shows up.
Charter schools do not have that obligation, either legally or morally. To the extent that many charter schools are oversubscribed, it would be difficult or impossible for them to do so. While the public schools have to cram in more students – hopefully, eventually, leading to more classrooms and even schools – charter schools only have to serve as many students as they specify. Charter schools are free to say that they do not offer support services for English language learners or autistic children, but the public schools must provide schooling for every child. Charter schools are free to “counsel out” students.
Charter school employees do not work for the government; they are not public employees. While the government has contracted with charter schools to provide a service, they do not act as the government when the provide it. Their operations are not subject to democratic or public oversight; rather their contracts (i.e. their charters) come up for review for possible extension periodically.
If you can make a more thoughtful argument about why charter schools are public schools, I would love to hear it. If you agree or disagree, please share your own thinking as to why.
[Note: I have posted a follow up piece, and the conversation has continued there.]
Misusing Language
01/16/2010
We all know about strong and weak regulation. In this era of economic crisis we debate stronger regulations, and decry the weakening or loosening of financial industry regulations.
We all know what that means. Stronger regulation means some combination of more laws, more controls, closer scrutiny, stricter limits, etc.. Strong price controls would be strict, and weaker regulation of the airline industry has lowered prices dramatically.
And yet, many charter school proponents have twisted the language around 180 degrees. They think that having few charter schools is “weak,” and having multiple authorizers (each of whom might have its own criteria) is “strong.” They think that freedom from preexisting school regulations makes a “strong” charter school law, and many believe that imposing strict requirements on schools as a condition for them to remain open after their initial characterization period is “weak.”
On Friday, The Quick and the Ed’s Chad Aldeman complained that Iowa’s strict charter school law is “one of the weakest charter laws in the country.” The Center for Education Reform’s annual report ranks states from “strongest” (DC, Minnesota & California) to “weakest” (Kansas, Virginia & Iowa). Note, of course, that CER‘s URL is charterschoolresearch.com — clearly marking them as charter school proponents. The National Alliance for Public Charter Schools uses the same language, as do many many others.
Of course, I have heard countless people use this kind of language at academic conferences and around schools of education.
Shouldn’t a “strong” charter school law be one that is very stingy in who can open a charter school, and provide for close oversight of charter school operations and strict accountability (in terms of whether or not they can stay open)? Without addressing the question of whether this kind of strength is good or bad for education, this would clearly be more consistent with how we generally speak about regulation.
How did this twisting of language happen? Why do charter proponents claim that permissive laws are “stronger” stricter laws? I do not ask why they claim such laws are better, because that is an entirely different issue. Does not this smack of the dishonesty of doublespeak?
Dumb Arguments for Stupid Ideas
12/24/2009
The reauthorization of NCLB should require states that accept Title I money (i.e. all of them) to require all public school teachers to get buzz cuts. Seriously*. This would benefit our schools and our students.
Think about it. With buzz cuts, teachers could get ready for work faster in the morning, and spend less time touching up their hair each day. That might give them an extra 10-30 minutes each day (fact**), time they could spend meeting with students, giving students better written feedback or creating better lesson plans. Not only that, but it would actually be like giving them a pay raise!!
Think about it. Our nation’s most efficient public service is the military (fact***). It requires all recruits to get a buzz cut (fact***). It is the strongest military in the world (fact). We want our schools to be the best and most efficient in the world, right? Why not follow the military’s model? This also returns us to traditional values for teachers, in this case discouraging their dating (fact). The time they are not spending on dating, preparing for dates and thinking about dates could then be poured into their teaching, as it should have been in the first place (fact***).
As for the pay raise, this is the genius part. The average amount of money public school teachers in this country spend on a hair cut – including tip — is $32.47 (fact**), and the median number of hair cuts per year is 11 (fact**). Plus, the average public school teacher in this country spends $157.32 on hair products each year (fact**). Requiring teachers to get buzz cuts would put an extra $500+ in their pocket each year. Moreover, this is all after tax dollars (fact), and when one takes the local, state and federal tax rates of the average teacher into account, this is the equivalent of more than a $1000 raise (fact**).
(* OK. Not seriously.)
(** = Not really a fact, but filling in the actual fact would not change the value of the argument.)
(*** = Not really a fact, but it suits my argument to say that it is.)
*****************
Last week, Corey Bunje Bower offered the world the most ironic piece that I know of to have come out of Vanderbilt’s School of Education. He argued against Kim Marshall’s recent commentary in Education Week on merit pay for teachers by claiming that “Ms. Marshall” presented a paucity of facts to support her case, and decried the lack of “discussion[s] of merit pay…[that are] based on facts rather than conjecture and [that] approach[] the topic in an unbiased way.”
The irony of this piece stems from its own amazing lack of facts. First, Kim Marshall is a man, not a woman. (This fact is very easy to ascertain. Google “Kim Marshall” and click on the first link. You’ll get a picture. Once I pointed this out to Mr. Bower, he fixed the pronouns.) Second, Marshall claims that all merit pay programs for teachers are collectively based, rather than individually based – though he leaves himself an out. As Mr. Marshall is arguing against individual merit pay programs, it is hard to understand why Mr. Bower thought it appropriate to argue with him. In fact, Mr. Bower writes that he knows of no such programs, but still argues against Mr. Marshall, decrying conjecture and Marshall’s lack of facts.
I do not write this to point out how wrong Mr. Bower is – though on individual merit pay I think he is very very wrong – but rather to note the flaws in his approach. He claims to be offering “Thoughts on Education Policy,” but is he not doing so very thoughtfully.
Yes, conjecture (i.e. “an opinion or conclusion formed on the basis of incomplete information”) is often a problem in policy discussions and education discussions (e.g. Mr. Bower’s post, ironically enough). No question about that. But the opposite of conjecture is not “facts” or even “research.” In my ridiculous argument above, I offer any number of facts – or place holders for facts – and they do not actually make for a strong argument. It remains a ridiculous argument, regardless of how many facts it purports to present.
You see, the opposite of conjecture is informed analysis, something that Mr. Marshall offered and something I strive to offer myself. Mr. Brower closes by calling for “sober analysis of research,” which is a good thing, but it is not enough and might not even be required. Some issues do not need to researched (e.g. buzz cuts for teachers). Some ideas can be dealt with well without research, though research can be useful and at times can be necessary.
A well designed thought experiment can tell us everything we need to know. Let us look at the potential elimination of free Metrocards for students to get to and from school in New York City. The Bloomberg administration has been encouraging the move away from neighborhood schools in favor of greater use of school choice in NYC. Without the numbers in front of us, I think that we could agree that students probably travel a lot further to school today than they did 30 years ago. We do not need research to tell us that there are many families in the city for whom buying Metrocards for their multiple children would be an incredible burden. A single parent with two kids in school, making three times the minimum wage would have to pay 5% of his/her take-home pay to get a paid for Metrocard ten months out of the year. ($870/week before taxes, $666 after. $1780 total for the Metrocards, $35,000 total take home pay.)
We do not need to research the policy to know that it is a bad idea. We can tell that a lot of kids will not get Metrocards. Lower income families will not be as able to take advantage of school choice. And we can easily predict that many kids will be absent or grossly tardy due to a lack of money to pay for the bus or subway. We do not need to do research, or to soberly analyze the results, to thoughtfully examine this proposal.
I will address the basic problems with individual merit pay proposals another time – I do think that Mr. Marshall left out some critically important issues. For now, however, I just want to urge everyone to look hard at the quality of the thinking behind the arguments people make, without decrying them simply because they do not happen to cite “research” as an academic would.
The reauthorization of NCLB should require states that accept Title I money (i.e. all of them) to require all public school teachers to get buzz cuts. Seriously*. This would benefit our schools and our students.
Think about it. With buzz cuts, teachers could get ready for work faster in the morning, and spend less time touching up their hair each day. That might give them an extra 10-30 minutes each day (fact**), time they could spend meeting with students, giving students better written feedback or creating better lesson plans. Not only that, but it would actually be like giving them a pay raise!!
Think about it. Our nation’s most efficient public service is the military (fact***). It requires all recruits to get a buzz cut (fact***). It is the strongest military in the world (fact). We want our schools to be the best and most efficient in the world, right? Why not follow the military’s model? This also returns us to traditional values for teachers, in this case discouraging their dating (fact). The time they are not spending on dating, preparing for dates and thinking about dates could then be poured into their teaching, as it should have been in the first place (fact***).
As for the pay raise, this is the genius part. The average amount of money public school teachers in this country spend on a hair cut – including tip — is $32.47 (fact**), and the median number of hair cuts per year is 11 (fact**). Plus, the average public school teacher in this country spends $157.32 on hair products each year (fact**). Requiring teachers to get buzz cuts would put an extra $500+ in their pocket each year. Moreover, this is all after tax dollars (fact), and when one takes the local, state and federal tax rates of the average teacher into account, this is the equivalent of more than a $1000 raise (fact**).
(* OK. Not seriously.)
(** = Not really a fact, but filling in the actual fact would not change the value of the argument.)
(*** = Not really a fact, but it suits my argument to say that it is.)
*****************
Last week, Corey Bunje Bower offered the world the most ironic piece that I know of to have come out of Vanderbilt’s School of Education. He argued against Kim Marshall’s recent commentary in Education Week on merit pay for teachers by claiming that “Ms. Marshall” presented a paucity of facts to support her case, and decried the lack of “discussion[s] of merit pay…[that are] based on facts rather than conjecture and [that] approach[] the topic in an unbiased way.”
The irony of this piece stems from its own amazing lack of facts. First, Kim Marshall is a man, not a woman. (This fact is very easy to ascertain. Google “Kim Marshall” and click on the first link. You’ll get a picture. Once I pointed this out to Mr. Bower, he fixed the pronouns.) Second, Marshall claims that all merit pay programs for teachers are collectively based, rather than individually based – though he leaves himself an out. As Mr. Marshall is arguing against individual merit pay programs, it is hard to understand why Mr. Bower thought it appropriate to argue with him. In fact, Mr. Bower writes that he knows of no such programs, but still argues against Mr. Marshall, decrying conjecture and Marshall’s lack of facts.
I do not write this to point out how wrong Mr. Bower is – though on individual merit pay I think he is very very wrong – but rather to note the flaws in his approach. He claims to be offering “Thoughts on Education Policy,” but is he not doing so very thoughtfully.
Yes, conjecture (i.e. “an opinion or conclusion formed on the basis of incomplete information”) is often a problem in policy discussions and education discussions (e.g. Mr. Bower’s post, ironically enough). No question about that. But the opposite of conjecture is not “facts” or even “research.” In my ridiculous argument above, I offer any number of facts – or place holders for facts – and they do not actually make for a strong argument. It remains a ridiculous argument, regardless of how many facts it purports to present.
You see, the opposite of conjecture is informed analysis, something that Mr. Marshall offered and something I strive to offer myself. Mr. Brower closes by calling for “sober analysis of research,” which is a good thing, but it is not enough and might not even be required. Some issues do not need to researched (e.g. buzz cuts for teachers). Some ideas can be dealt with well without research, though research can be useful and at times can be necessary.
A well designed thought experiment can tell us everything we need to know. Let us look at the potential elimination of free Metrocards for students to get to and from school in New York City. The Bloomberg administration has been encouraging the move away from neighborhood schools in favor of greater use of school choice in NYC. Without the numbers in front of us, I think that we could agree that students probably travel a lot further to school today than they did 30 years ago. We do not need research to tell us that there are many families in the city for whom buying Metrocards for their multiple children would be an incredible burden. A single parent with two kids in school, making three times the minimum wage would have to pay 5% of his/her take-home pay to get a paid for Metrocard ten months out of the year. ($870/week before taxes, $666 after. $1780 total for the Metrocards, $35,000 total take home pay.)
We do not need to research the policy to know that it is a bad idea. We can tell that a lot of kids will not get Metrocards. Lower income families will not be as able to take advantage of school choice. And we can easily predict that many kids will be absent or grossly tardy due to a lack of money to pay for the bus or subway. We do not need to do research, or to soberly analyze the results, to thoughtfully examine this proposal.
I will address the basic problems with individual merit pay proposals another time – I do think that Mr. Marshall left out some critically important issues. For now, however, I just want to urge everyone to look hard at the quality of the thinking behind the arguments people make, without decrying them simply because they do not happen to cite “research” as an academic would.
Report Cards and Test Scores
11/17/2009
Did you see that the state wants to evaluate teacher training programs by their graduates’ students’ test scores? Of course, the usual sources think that this is a great idea.
Did you see Michael McCurdy’s post this week in the Community section on what a kindergarten report card — at least for a G&T student — looks like? Apparently, there are nearly 100 criteria — I counted 98— each which gets rated on a scale from one to four.
Of course, high school students’ report cards are far simpler. Just one grade for each subject, something like six to nine marks total. It’s amazing that in nine years schools gets so much simpler and less complex that reporting can be simplified by a factor of ten or more.
Of course, this proposal would rate teachers’ programs even more simply than that, at least high school teachers’. Some teachers would be rated on a single test, and others rated on no test whatsoever (i.e. art, gym, electives). So, a truly a great way to judge teacher preparation programs, no? Remarkable how as we get older, our tasks get so simplified that others need look at less and less data to evaluate how well we are doing, isn’t it?
Did you see that the state wants to evaluate teacher training programs by their graduates’ students’ test scores? Of course, the usual sources think that this is a great idea.
Did you see Michael McCurdy’s post this week in the Community section on what a kindergarten report card — at least for a G&T student — looks like? Apparently, there are nearly 100 criteria — I counted 98— each which gets rated on a scale from one to four.
Of course, high school students’ report cards are far simpler. Just one grade for each subject, something like six to nine marks total. It’s amazing that in nine years schools gets so much simpler and less complex that reporting can be simplified by a factor of ten or more.
Of course, this proposal would rate teachers’ programs even more simply than that, at least high school teachers’. Some teachers would be rated on a single test, and others rated on no test whatsoever (i.e. art, gym, electives). So, a truly a great way to judge teacher preparation programs, no? Remarkable how as we get older, our tasks get so simplified that others need look at less and less data to evaluate how well we are doing, isn’t it?
What is a “Good Student”?
11/13/2009
In Horace’s Compromise, Ted Sizer problematizes* our understanding of what constitutes a good high school. He questions how much learning is really going on in our good schools (i.e. our suburban schools that send kids to fancy colleges). The compromise the title refers to is the deal between students and teachers, in which the students agree not to give the teachers any real trouble, so long as the teachers don’t give them any real trouble – or maybe it is the other way around. Schools don’t really challenge students with truly difficult lessons, and students agree to jump through the hoops that have replaced real and deep learning. In Doing School, Denise Pope makes clear what this looks like from students’ perspectives. She shows different strategies that teenagers might take to be a good student.
* Sorry. I really like that word.
But what really makes for a good student? Sizer and Pope make clear what our current system has worked so hard to teach kids about what it means to be a good student. But what if we could start from scratch? What if we considered the kinds of goals we really want our schools to aim for? Principally, what if we made learning – rather than achievement or attainment – the center of schooling?
*****************
I had a “cardiac event” this week. Chest pains, shortness of breath, numb left arm. A few other symptoms, too. The hospital did a lot to work me up to see what had happened and what might be done: blood work, EKGs, echocardiogram, MRI, cardiac angiogram, cardiac cathaterization. It was a lot of stuff, or at least it felt like a lot of stuff to me.
This was a very frustrating experience for me. The doctors – and physician assistants, nurses and technicians – clearly became very frustrated with me. They obviously felt like I was not being a very cooperative patient.
When you get a medical test or procedure, there are these forms to sign. By signing, you attest that you’ve been informed about the test or procedure, its purpose, the potential side effects, possible alternative treatments, and that you have had your questions answered. The doctors (or other medical personnel) come in to answer your questions – or so they say.
But it turns out that they do not really want to answer questions. Sure, they will answer a couple of questions. But they do not want to be pressed with lots of questions. Prior to one procedure – the one where they stuck a tube into my heart (via my groin!!) – I had forty-five minutes of questions, when they only expected a few minutes.
Later, with another procedure, I was actually told “It will go better if you don’t ask any questions.” Of course, that came after they told me that I should ask them any questions I had about the procedure, even during the procedure.
It will go better if you don’t ask any questions.
*****************
Have you ever seen a teacher get frustrated with a student’s questions? Ever seen a teacher at a loss for how to answer a question? Ever felt like your students’ questions were detracting you from the important lesson you were trying to teach?
Have you ever heard teachers complain about pushy parents? Ever wondered how much of that is being forced to explain or defend their actions? I mean, just a product of seemingly endless questioning – perhaps pointed – by non-educators.
*****************
Have you ever seen an administrator who claims to have an open door policy react poorly when teachers ask him/her about why s/he took a particular action?
A common thread that comes up in the comments here on Gotham School is the issue of the relationship between school administrators and teachers. How collaborative should leadership be? How much power and authority and decision-making should rest in the hands of principals – especially new principals?
*****************
It is not just doctors and teachers. It is lawyers and other professionals, too. Perhaps it is most obvious with the legendarily poor help desk personnel. When we are there to support others, to help them, to work for them, to best enable them to do whatever it is they do, when they are the final decision-maker or do the core work of the organization, do we really tolerate a relationship that best supports them?
What is a good patient, from a doctor’s point of view? One who follows (doctors’) instructions and doesn’t ask too many questions?
What is a good client? One who follows (lawyers’) instructions and doesn’t ask too many questions?
What is a good parent? One who follows (schools’) instructions and doesn’t ask too many questions?
What is a good student? One who follows (teachers’) instructions and doesn’t ask too many questions?
What is a good teacher? One who follows (administrators’) instructions and doesn’t ask too many questions?
What do you think?
Who the Heck Is Ted Sizer?
10/23/2009
Ted Sizer was a critic and trouble maker. He looked at our schools, all of our schools, and said in essence, not good enough and we are doing it wrong.
He didn’t mean the bad schools. And he didn’t mean achievement gaps. He meant all schools. He meant the good schools too, even the best schools.
So, who the hell was Ted Sizer? He was a visionary educator and critic of our schools, a real giant who was influential enough to get a 1000+ word obituary in yesterday’s New York Times and numerous other tributes and articles this week.
His doctorate was in the history of education, and I believe his dissertation was about how the high school credits thing evolved. Forty years ago he was the Dean of the Harvard Graduate School of Education. After that he was the Headmaster at Philips Academy in Andover, Massachusetts. Then a professor of education at Brown. He also helped found a charter school in the middle of Massachusetts, and late in his life was co-principal of it with his wife Nancy. He had credibility in the most powerful of circles.
In 1983, the famous report A Nation at Risk was released by the Reagan administration. It warned of “a rising tide of mediocrity that threatens our very future as a Nation and a people and declared that “If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war.” You see these quotes all over the place, and it is easy to say that this report marked the beginning of the standards and reform movement.
Ted’s Horace’s Compromise was published the next year. He neither defended the status quo nor focused on our obviously failing schools. His critique was nothing like that of A Nation at Risk. Rather, he attacked the very foundation of how our high school works, looking at the basic compromise between teachers and students, an agreement that if students do not create trouble for teachers that teachers will not create trouble for students. This compromise infects what is taught, how it is taught, and the expectations for what learning really is.
What is education? “The worthy residue that remains after the lessons have been forgotten.” When the students forget the explicit contents of today’s lesson – and we know that they will – what is left? Anything? What happens after they forget the difference between atomic number and atomic mass? What is left after they forget the difference between the Declaration of the Rights of Man and the Universal Declaration of Human Rights? After they forget the rhyme scheme and meter of a Shakespearean Sonnet or the relationship between sin, cos and tan?
I read this stuff and was amazed. Someone else out there saw what I saw, the essential hypocrisy in “schooling” in America! Even at our allegedly best schools (e.g. The Thomas Jefferson High School for Science and Technology, Stuyvesant, Andover) we were not doing the right thing, and yet people wanted the other schools to be more like the “best” schools.
But how to provide an education that remains meaningful beyond graduation? Twenty-five years ago, Ted Sizer founded the Coalition of Essential Schools, a voluntary association open to any school that wanted to be a member. CES was built around these link: ten principles, though there was no exam or inspection for Coalition schools.
I was enormously lucky. I got to study with Ted. I got to talk with him for dozens hours about the design, aims and goals of the American high school. (He once called something I said “Quotable!” and I cannot begin to tell you how great that made me feel.) I already knew about Deb Meier’s work, but he gave me enormous new insights and understandings. Though Ted, I learned so much about the idea of teaching Habits of Mind rather than skills or knowledge.
I do not think that most members of the Coalition even come close to Ted’s vision, and I know nothing about Ted’s school, the Parker Essential Charter School. But I recognize that it is amazingly difficult to overturn decades or centuries of understanding about what school should look like and aim towards, and that even falling short of his vision can constitute a huge step forward for our students. And so, Ted Sizer gave us all an ideal of what meaningful schooling could mean, something to work towards, even while the forces around us push us to schooling as the most structured, reductive, temporary value, baby-sitting and crowd control. I know that I’ve been known to be critical of aspirational goals, but every step towards his vision constitutes real improvement and additional life long value for students.
Ted Sizer was a critic and trouble maker. He looked at our schools, all of our schools, and said in essence, not good enough and we are doing it wrong.
He didn’t mean the bad schools. And he didn’t mean achievement gaps. He meant all schools. He meant the good schools too, even the best schools.
So, who the hell was Ted Sizer? He was a visionary educator and critic of our schools, a real giant who was influential enough to get a 1000+ word obituary in yesterday’s New York Times and numerous other tributes and articles this week.
His doctorate was in the history of education, and I believe his dissertation was about how the high school credits thing evolved. Forty years ago he was the Dean of the Harvard Graduate School of Education. After that he was the Headmaster at Philips Academy in Andover, Massachusetts. Then a professor of education at Brown. He also helped found a charter school in the middle of Massachusetts, and late in his life was co-principal of it with his wife Nancy. He had credibility in the most powerful of circles.
In 1983, the famous report A Nation at Risk was released by the Reagan administration. It warned of “a rising tide of mediocrity that threatens our very future as a Nation and a people and declared that “If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war.” You see these quotes all over the place, and it is easy to say that this report marked the beginning of the standards and reform movement.
Ted’s Horace’s Compromise was published the next year. He neither defended the status quo nor focused on our obviously failing schools. His critique was nothing like that of A Nation at Risk. Rather, he attacked the very foundation of how our high school works, looking at the basic compromise between teachers and students, an agreement that if students do not create trouble for teachers that teachers will not create trouble for students. This compromise infects what is taught, how it is taught, and the expectations for what learning really is.
What is education? “The worthy residue that remains after the lessons have been forgotten.” When the students forget the explicit contents of today’s lesson – and we know that they will – what is left? Anything? What happens after they forget the difference between atomic number and atomic mass? What is left after they forget the difference between the Declaration of the Rights of Man and the Universal Declaration of Human Rights? After they forget the rhyme scheme and meter of a Shakespearean Sonnet or the relationship between sin, cos and tan?
I read this stuff and was amazed. Someone else out there saw what I saw, the essential hypocrisy in “schooling” in America! Even at our allegedly best schools (e.g. The Thomas Jefferson High School for Science and Technology, Stuyvesant, Andover) we were not doing the right thing, and yet people wanted the other schools to be more like the “best” schools.
But how to provide an education that remains meaningful beyond graduation? Twenty-five years ago, Ted Sizer founded the Coalition of Essential Schools, a voluntary association open to any school that wanted to be a member. CES was built around these link: ten principles, though there was no exam or inspection for Coalition schools.
- Learning to use one’s mind well
- Less is More, depth over coverage
- Goals apply to all students
- Personalization
- Student-as-worker, teacher-as-coach
- Demonstration of mastery
- A tone of decency and trust
- Commitment to the entire school
- Resources dedicated to teaching and learning
- Democracy and equity
I was enormously lucky. I got to study with Ted. I got to talk with him for dozens hours about the design, aims and goals of the American high school. (He once called something I said “Quotable!” and I cannot begin to tell you how great that made me feel.) I already knew about Deb Meier’s work, but he gave me enormous new insights and understandings. Though Ted, I learned so much about the idea of teaching Habits of Mind rather than skills or knowledge.
I do not think that most members of the Coalition even come close to Ted’s vision, and I know nothing about Ted’s school, the Parker Essential Charter School. But I recognize that it is amazingly difficult to overturn decades or centuries of understanding about what school should look like and aim towards, and that even falling short of his vision can constitute a huge step forward for our students. And so, Ted Sizer gave us all an ideal of what meaningful schooling could mean, something to work towards, even while the forces around us push us to schooling as the most structured, reductive, temporary value, baby-sitting and crowd control. I know that I’ve been known to be critical of aspirational goals, but every step towards his vision constitutes real improvement and additional life long value for students.
Standards: Understanding the Appeal
10/22/2009
Despite everything I have written about the intrinsic weaknesses of big state or national standards, I understand the appeal of standards. I really do.
Imagine this:
Well, that’s the vision. And the vision is the appeal. I get it. What’s not to like?
Now, imagine this:
Moreover, no one has actually explained how new or more rigorous standards actually improve education. I’ve given six reasons why I don’t think that they really can. I’m looking for a “theory of action,” an explanation of the process and mechanism by which this proposed solution actually leads to the desired result.
I have absolutely no doubt that education in our schools happens in the interactions between children and educators, children and each other, children and themselves, and children and the materials, activities and environments that educators set up. If someone can explain to me how new standards will impact those interactions, in spite of all the obstacles I have previously laid out, I might get on board with a serious push for new standards.
Until then, however, I will continue to think efforts to improve, and strengthen standards a big waste of time, money and attention that would be better spent on other purposes. Diane Ravitch just said of another reform, “It may be a distraction from the serious issues that confront our students and our schools.” That is precisely my concern here.
Despite everything I have written about the intrinsic weaknesses of big state or national standards, I understand the appeal of standards. I really do.
Imagine this:
A country in which all students are well educated. High quality curricula, outlined in rigorous standards, taught by smart, thoughtful expert teachers using differentiated instructional techniques that address multiple modalities, learning styles and intelligences. Students easily surpassing the low bar of standardized tests and building a lifelong love of learning. Even students who move or are in the lowest SES communities are well educated because standards ensure that all schools are appropriately ambitious for their students and focused on the core content that really matters.What’s not to like?
Well, that’s the vision. And the vision is the appeal. I get it. What’s not to like?
Now, imagine this:
You own a home appliance store, and you’ve got this friend who makes lousy chocolate chip cookies. You and your friend share a vision of him/her taking flour, butter, brown sugar, white sugar, baking soda, salt, vanilla, eggs and chocolate chips, combining them and pulling fresh hot and yummy smelling cookies out of the oven. But some of the butter is rancid. Some of the flour is moldy. The brown sugar is dehydrated and hard as a rock. There’s no baking soda, just baking powder. There’s no table salt, just kosher salt — which neither of you understand. The vanilla and eggs are fine, but there’s not enough chocolate chips. S/he has an oven, but it’s a little old. You, however, own a home appliance store. So, you give him/her a brand new fancy Viking oven.The problems with our educational system do not stem from a lack of standards or a lack of rigor in our standards any more than your friend’s cookie problem stems from a bad oven. The fact that the great vision includes standards (or an oven) does not mean that a better oven/set of standards will cover for the other problems we actually face. Every state has standards already. Even if they are insufficient — and I have no doubt that many are — they do not comprise even one of the top ten problems facing our educational system.
Moreover, no one has actually explained how new or more rigorous standards actually improve education. I’ve given six reasons why I don’t think that they really can. I’m looking for a “theory of action,” an explanation of the process and mechanism by which this proposed solution actually leads to the desired result.
I have absolutely no doubt that education in our schools happens in the interactions between children and educators, children and each other, children and themselves, and children and the materials, activities and environments that educators set up. If someone can explain to me how new standards will impact those interactions, in spite of all the obstacles I have previously laid out, I might get on board with a serious push for new standards.
Until then, however, I will continue to think efforts to improve, and strengthen standards a big waste of time, money and attention that would be better spent on other purposes. Diane Ravitch just said of another reform, “It may be a distraction from the serious issues that confront our students and our schools.” That is precisely my concern here.
Standards: Demystifying, Debunking and Discrediting
10/8/2009
We are decades into the Education Standards Movement. Standards have been all the rage for quite some time, and they are getting all kinds of attention today. Right now, there is all kinds of work on national standards going on.
But I say, “Feh!” Standards do not matter — particularly national standards — even if we dearly want them to.
What Are Standards, Anyway?
Standards prescribe and specify what should be done in school. In that, they are similar to curricula and lesson plans. In fact, the line between standards and curricula can be hard to distinguish — as can the line between curricula and lesson plans. As a rule, however, standards are the least specific of the three, and focus on what should be taught, rather than how it should be taught.
So, standards documents describe the goals of a course or a subject. They are the bar or the target, depending on your preferred metaphor. They declare what should be taught, what students should learn and/or what they should be able to do by their course’s end.
When I was teaching in New York not that long ago, each of the English teachers in my school was required to have a poster of the ELA standards up in their classrooms. The contents were probably just a couple of pages long, and they specified what students should learn in their high school English classes.
The widely publicized Common Core draft ELA standards, released last month, can be found in a 47 page document, of which six pages comprise the standards and the rest are explanations and examples to help the reader make sense of them.
Who Creates Standards
Anyone may write standards, and can try to publicize them and get others to pay attention. Rarely, however, will such efforts be successful. Standards simply cannot have more power and authority than the organization that publishes them. NCTM (the National Council of Teachers of Mathematics) has been publishing standards for decades, and their expertise and credibility have given them a lot of weight.
However, the most important standards have been created by the states. In our system, the federal government has no constitutional role in education, in theory leaving the entire enterprise to the states. In the last fifty years, though the federal role has grown, standards have still remained the province of the states. In 1994, President Clinton tried to foster the creation of voluntary national standards, but was politically unable to do so — due to the efforts of Lynne Cheney and others. This time around, the Council of Chief State School Officers is leading the effort, along with Achieve — a creation of the National Governors Association. Thus, this time we have a national effort that is not tied to the federal government in Washington, DC.
Of course, it is not as though President Clinton, Governor Schwarzenegger or any of the State Superintendents write standards themselves. Rather, they are supporting the creation of standards by teams and committees of experts. These can include text book writers and publishers, teachers, researchers, professors of education, experts in the appropriate content areas and various others. Real people, with real expertise, real agendas — for better and for worse — and real histories.
Six Problems
Over the next week, I will explain the major problems I see with standards efforts, particularly high profile national standards.
Problem #1: Which Bar to Raise?
Problem #2: An Unrealistic Bar
Problem #3: Fear of Failure Rates
Problem #4: Classrooms
Problem #5: Tests Matter; Standards Do Not
Problem #6: Local Control
Standards: Why Does Anyone Bother?
Come back through the week and share what you think of each reason. In the meanwhile, do you think that there’s a strong case to be made for strong state or national standards?
Standards: Why Does Anyone Bother?
10/20/2009
For the last two weeks, I’ve been raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Why Does Anyone Bother?
Hamlet spoke of customs “More honor’d in the breach than the observance.” I would not go quite that far when speaking of standards in education, but that is primarily because standards are in large part based on what is already actually done. To the extent that they are descriptive, standards are honored. But to the extent that they are prescriptive, they are rather impotent.
So, why all this attention to standards? Why is anyone bothering, and why does anyone pay attention?
President Kennedy said of the Apollo program,
We choose to go to the moon in this decade and do the other things, not only because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.I wish that we could say the same of our high publicity efforts in education, that we do them not because they are easy. But I know that it is not true. Politicians and public leaders want to do something, and probably want to be seen as doing something, so they do what they can, even if what they can do is worthless. Putting together a group to revise standards, or toughen them up, is not that hard. Even getting fifty governors and chief state school officers to agree to adopt/adopt voluntary national standards before they are published is not very hard. It is certainly easier than actually improving educational practices in thousands of schools and millions of classrooms.
Abraham Maslow observed that if all you have is a hammer, than every problem looks like a nail. I suppose that to people at the top who know that their time is limited, top down solutions that can be rolled out quickly look like a good idea. That has got to be part of it — though the problem is bigger than that, as it is not just people at the top (e.g. governors) who believe that these kinds of efforts matter. Yet it is really hard to believe that anyone who has invested serious time in thinking about children or how their schooling actually works could think that new national standards are going to make a difference.
Standards: Demystifying, Debunking and Discrediting
Problem #1: Which Bar to Raise?
Problem #2: An Unrealistic Bar
Problem #3: Fear of Failure Rates
Problem #4: Classrooms
Problem #5: Tests Matter; Standards Do Not
Problem #6: Local Control
Debunking Standards Issue #6: Local Control
10/19/2009
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Directly contrary to the urge for state or even national standards is the long standing support for local control of schools is this country. Parents and communities want to decide what is taught in their schools, and how it is taught.
To the extent that standards are voluntary — and certainly in the absence of inspections, everything not on a test is voluntary —there are powerful forces against adopting standards. Is there really any reason to think that states will adopt national standards wholesale without making some of their own changes and tweaks? Even if only to “demonstrate” their own leadership, expertise and value, don’t we all expect many/most states to adapt any voluntary national standards, rather than adopt them wholesale?
Moreover, the long tradition of local control even works against states’ own standards. Real state efforts to control what is taught in schools are a relatively recent phenomena, and hardly have any teeth at all. All of the dynamics that make national standards so problematic also work to undermine state standards, with local officials (e.g. school board members, district leaders, school leaders, even individual teachers) exerting their own influence and power over curriculum and content.
Previous: Problem #5 — Tests Matter; Standards Do Not
Next: Why Does Anyone Bother?
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Problem #6: Local Control
Directly contrary to the urge for state or even national standards is the long standing support for local control of schools is this country. Parents and communities want to decide what is taught in their schools, and how it is taught.
- Northern Aggression? States’ rights? Slavery?
- Comprehensive or abstinence only?
- Whole language or phonics?
To the extent that standards are voluntary — and certainly in the absence of inspections, everything not on a test is voluntary —there are powerful forces against adopting standards. Is there really any reason to think that states will adopt national standards wholesale without making some of their own changes and tweaks? Even if only to “demonstrate” their own leadership, expertise and value, don’t we all expect many/most states to adapt any voluntary national standards, rather than adopt them wholesale?
Moreover, the long tradition of local control even works against states’ own standards. Real state efforts to control what is taught in schools are a relatively recent phenomena, and hardly have any teeth at all. All of the dynamics that make national standards so problematic also work to undermine state standards, with local officials (e.g. school board members, district leaders, school leaders, even individual teachers) exerting their own influence and power over curriculum and content.
Previous: Problem #5 — Tests Matter; Standards Do Not
Next: Why Does Anyone Bother?
Debunking Standards Issue #5: Tests Matter; Standards Do Not
10/15/2009
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
As much as they may hate them, teachers do respond to tests. Not always well or in good faith, but teachers and schools feel the pressure of high stakes and public reported tests.
Tests, of course, are usually supposed to be based on standards. State tests are specifically supposed to be based on state standards. State tests might, in the future, be based on the Common Core standards.
But that’s not really true, not exactly. You see, tests only include the standards that we know how to test and are capable of testing relatively cheaply.
The recent draft of the Common Core ELA standards actually begins, “A crucial factor in readiness for college and careers is students’ ability to comprehend complex texts independently.” However, psychometric requirements for reliability, combined with reasonable limits on the time and money it costs to take an exam, prevent the inclusion of complex texts if only because text length is an issue. Later, the draft says, “Students must be able to revisit and make improvements to a piece of their writing over multiple drafts when circumstances encourage or require it.” There is also a whole section for “Speaking and Listening.” Does anyone expect that that standardized tests based upon the Common Core standards will include a speaking section or will include students’ ability to revise their own work?
And so, even if tests do a good job of evaluating students on the standards that they attempt to include, they do not actually represent the larger set of standards fairly. Moreover, in the absence of real advances in testing, I do not see how changes in the standards will lead to changes in the tests. Test developers will continue to test what they know how to test, regardless of what the standards say or how they have been changed.
Previous: Problem #4 — Classrooms
Next: Problem #6 — Local Control
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Problem #5: Tests Matter; Standards Do Not
As much as they may hate them, teachers do respond to tests. Not always well or in good faith, but teachers and schools feel the pressure of high stakes and public reported tests.
Tests, of course, are usually supposed to be based on standards. State tests are specifically supposed to be based on state standards. State tests might, in the future, be based on the Common Core standards.
But that’s not really true, not exactly. You see, tests only include the standards that we know how to test and are capable of testing relatively cheaply.
The recent draft of the Common Core ELA standards actually begins, “A crucial factor in readiness for college and careers is students’ ability to comprehend complex texts independently.” However, psychometric requirements for reliability, combined with reasonable limits on the time and money it costs to take an exam, prevent the inclusion of complex texts if only because text length is an issue. Later, the draft says, “Students must be able to revisit and make improvements to a piece of their writing over multiple drafts when circumstances encourage or require it.” There is also a whole section for “Speaking and Listening.” Does anyone expect that that standardized tests based upon the Common Core standards will include a speaking section or will include students’ ability to revise their own work?
And so, even if tests do a good job of evaluating students on the standards that they attempt to include, they do not actually represent the larger set of standards fairly. Moreover, in the absence of real advances in testing, I do not see how changes in the standards will lead to changes in the tests. Test developers will continue to test what they know how to test, regardless of what the standards say or how they have been changed.
Previous: Problem #4 — Classrooms
Next: Problem #6 — Local Control
Debunking Standards Issue #4: Classrooms
10/14/2009
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
I have never heard a teacher declare that s/he was going to change what they were teaching because of something s/he saw in a standards document. Never.
Have you?
I have known teachers and other educators to look through standards and declare what they like or do not like. I have heard them say that they approve of certain changes and not approve of others. But no one has ever said that they have to stop doing something because it fell out of the standards or that they need to start something new because it is in the standards.
The fact is that teachers already know what they want to do in their classes. Whether or not you or I agree or approve of their ideas, they already have them. The ideas could have developed during their own days in school, during their preparation, from things they have read, from discussions or experiences with colleagues or any number of other ways. I suppose that, in theory, reading standards documents could shape these ideas, but I have never seen any evidence of it, and I doubt that you have, either.
Previous: Problem #3 — Fear of Failure Rates!
Next: Problem #5 — Tests Matter; Standards Do Not!
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Problem #4: Classrooms
I have never heard a teacher declare that s/he was going to change what they were teaching because of something s/he saw in a standards document. Never.
Have you?
I have known teachers and other educators to look through standards and declare what they like or do not like. I have heard them say that they approve of certain changes and not approve of others. But no one has ever said that they have to stop doing something because it fell out of the standards or that they need to start something new because it is in the standards.
The fact is that teachers already know what they want to do in their classes. Whether or not you or I agree or approve of their ideas, they already have them. The ideas could have developed during their own days in school, during their preparation, from things they have read, from discussions or experiences with colleagues or any number of other ways. I suppose that, in theory, reading standards documents could shape these ideas, but I have never seen any evidence of it, and I doubt that you have, either.
Previous: Problem #3 — Fear of Failure Rates!
Next: Problem #5 — Tests Matter; Standards Do Not!
Debunking Standards Issue #3: Fear of Failure Rates
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
I am not a fan of most of what appears on The Quick and the Ed, but last month Chad Adelman made a great point about setting high standards. He explained that when they are taken seriously and the inevitable high failure rates occur, people find or create loopholes or backdoors.
Frankly, people do not have the stomach for high failure rates. It is easy to say that we want to raise standards; that is the good news. But it is hard to endorse high failure rates; that is public bad news.
In a 2001 episode of The West Wing, two characters discussed the impact of making the standard for poverty more rigorous and realistic. The good news was that they had a better sense of the problem and would be better able to address it.
Previous: Problem #2 — An Unrealistic Bar!
Next: Problem #4 — Classrooms!
Problem #3: Fear of Failure Rates
I am not a fan of most of what appears on The Quick and the Ed, but last month Chad Adelman made a great point about setting high standards. He explained that when they are taken seriously and the inevitable high failure rates occur, people find or create loopholes or backdoors.
Frankly, people do not have the stomach for high failure rates. It is easy to say that we want to raise standards; that is the good news. But it is hard to endorse high failure rates; that is public bad news.
In a 2001 episode of The West Wing, two characters discussed the impact of making the standard for poverty more rigorous and realistic. The good news was that they had a better sense of the problem and would be better able to address it.
Toby: Let’s get back to the bad news. Four million people became poor on the President’s watch?But public failure is always a political finding, too. And people subject to politics, be they elected, appointed or just in high visible positions, have great incentive to undermine bad news or prevent the news from coming out. So, the more rigorous the standards, the less seriously others will take them, knowing that they will likely be blamed for the bad news. The idealized senior staff of The West Wing could accept “the bad news” because it was really just a more accurate description of reality. But would our real flesh & blood leaders, with all of the pressures they face today, be as well able to accept “the bad news” — and potentially the blame for it? When new more rigorous standards lead to reports of fewer expert or even proficient students, those in positions of responsibility will be blamed. Will they allow that to happen?
Sam: They didn’t become poor. They were poor already. And now we’re calling them poor.
Toby: What was wrong with the old formula?
Sam: I don’t know.
Toby: Find out.
Sam: It is possible that this is a statistical reality and not a political finding.
Previous: Problem #2 — An Unrealistic Bar!
Next: Problem #4 — Classrooms!
Debunking Standards Issue #2: An Unrealistic Bar
10/12/2009
Since last week, I have been raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Even if we did not have the kinds of gaps that we see between schools, districts and even states, there is a common problem with where to set the bar. Standards are often set by content experts who have rarely worked with below average students in their field, and perhaps not even average students. They declare what they think students ought to know or be able to do by the time that they graduate from high school, for example. Imagine what college professors/instructors of mathematics would say that high school graduates should know. And historians. And scientists.
When these standards setting committees say, “To be proficient, a student should know…,” what do they actually mean by proficient? Are these bars set for the average student? For the honors student? For the student who truly excels in that subject and will major in it in college?
I don’t see a lot of pressure for these brilliant experts — and I am perfectly willing to concede that they are brilliant experts — to consider a bar any lower than what they think ought to be possible, what they would like to see happen. But they don’t do research or investigation to see how likely or practical their goals are, for whom they might be reasonable, or what it would require for schools to raise all of their students to that level of proficiency — presumably the goal, right?
This leads to aspirational goals and standards, rather than realistically achievable standards.
Previous: Problem #1 — Which Bar to Raise?
Next: Problem #3 – Fear of Failure Rates.
Since last week, I have been raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Problem #2: An Unrealistic Bar
Even if we did not have the kinds of gaps that we see between schools, districts and even states, there is a common problem with where to set the bar. Standards are often set by content experts who have rarely worked with below average students in their field, and perhaps not even average students. They declare what they think students ought to know or be able to do by the time that they graduate from high school, for example. Imagine what college professors/instructors of mathematics would say that high school graduates should know. And historians. And scientists.
When these standards setting committees say, “To be proficient, a student should know…,” what do they actually mean by proficient? Are these bars set for the average student? For the honors student? For the student who truly excels in that subject and will major in it in college?
I don’t see a lot of pressure for these brilliant experts — and I am perfectly willing to concede that they are brilliant experts — to consider a bar any lower than what they think ought to be possible, what they would like to see happen. But they don’t do research or investigation to see how likely or practical their goals are, for whom they might be reasonable, or what it would require for schools to raise all of their students to that level of proficiency — presumably the goal, right?
This leads to aspirational goals and standards, rather than realistically achievable standards.
Previous: Problem #1 — Which Bar to Raise?
Next: Problem #3 – Fear of Failure Rates.
Debunking Standards Issue #1: Which Bar to Raise
10/9/2009
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
New standards and new standards initiatives are always about raising the bar. They are always about improving education, educational outcomes and sometimes even — shockingly — improving test scores. Standards efforts are never aimed at merely documenting what is actually done.
But this goal is actually impossible to accomplish with a single bar or single standard because we know that there are all kinds of achievement gaps in this country. Yes, there are racial/ethnic gaps and income gaps. But there are also geographic gaps. The NAEP has show us that some states simply do much much better than others. Caroline Hoxby’s latest report talks about the Harlem-Scarsdale gap. Regardless of the cause of these geographic gaps, they exist.
Given such gaps between states and within states, for whom should we raise the bar? Those who call for excellence are looking to improve the top half or quarter. Those who call for equity aim to improve the bottom half or quarter. Perhaps we can do both at the same time, but wouldn’t that call for multiple bars? States like Mississippi and California could show incredible improvement and still be behind states like Massachusetts and New Jersey. Set the bar high enough to push higher achieving states or districts, and the lower achieving area will see a demoralizing and impossible goal and be that much less likely to take it seriously. Set it at a level to be realistically inspiring for lower achieving states and higher achieving states could sit on their laurels for having already passed it. The exact same issues hold true for districts and even schools.
So, a single set of standards to raise the bar? Impossible.
Next: Problem #2 — An Unrealistic Bar.
This and next week I am raising objections to the idea that new standards — particularly new national standards — are worth the attention they get. It is ridiculous to think that they can be a meaningful lever of broad educational improvement. In fact, I do not think that they can have any significant impact at all.
Problem #1: Which Bar to Raise?
New standards and new standards initiatives are always about raising the bar. They are always about improving education, educational outcomes and sometimes even — shockingly — improving test scores. Standards efforts are never aimed at merely documenting what is actually done.
But this goal is actually impossible to accomplish with a single bar or single standard because we know that there are all kinds of achievement gaps in this country. Yes, there are racial/ethnic gaps and income gaps. But there are also geographic gaps. The NAEP has show us that some states simply do much much better than others. Caroline Hoxby’s latest report talks about the Harlem-Scarsdale gap. Regardless of the cause of these geographic gaps, they exist.
Given such gaps between states and within states, for whom should we raise the bar? Those who call for excellence are looking to improve the top half or quarter. Those who call for equity aim to improve the bottom half or quarter. Perhaps we can do both at the same time, but wouldn’t that call for multiple bars? States like Mississippi and California could show incredible improvement and still be behind states like Massachusetts and New Jersey. Set the bar high enough to push higher achieving states or districts, and the lower achieving area will see a demoralizing and impossible goal and be that much less likely to take it seriously. Set it at a level to be realistically inspiring for lower achieving states and higher achieving states could sit on their laurels for having already passed it. The exact same issues hold true for districts and even schools.
So, a single set of standards to raise the bar? Impossible.
Next: Problem #2 — An Unrealistic Bar.
What is “The Gold Standard”?
9/23/2009
Did you hear about the big report that came out this week? You know, the one that “shows” that NYC charter schools are better than traditional non-charter public schools? It has gotten a ton of attention, probably because it uses “‘the gold standard’ method[ology].” The report is not subtle about this. It is right there in the very first sentence of the executive summary, “The distinctive feature of this study is that charter schools’ effects on achievement are estimated by the best available, “gold standard” method: lotteries.” It even uses the term “gold standard” four more times throughout the report.
Everyone wants to follow The Gold Standard — or at least be able to say that they do. Of course! I mean, who wouldn’t? But I do not think that we actually have a gold standard in education research. In fact, I am quite sure that we do not, and appropriating biomedical research’s gold standard does not make it appropriate for us.
However, if we are going to borrow their standard, can we not at least get it right?
The biomedical standard uses double-blind experimental studies with random assignment. That means that some research participants get the experimental treatment and some get a placebo, and both are assigned randomly. It also means that neither the researchers nor the participants know who is getting which treatment. After all, expectations are important, and the mind can set us up for all kinds of things.
*****************
One of the latest ideas about The Gold Standard in educational research concerns charter schools.
We all want to know whether charter schools are better than traditional non-charter public schools. On one level, we certainly do want to know about individual schools. But on the policy level, we want to know about the average charter school, because we want to figure out if “charterness” helps a school be better. If it does, then we want more charter schools. If it does not, then we want fewer or none. And if we cannot be sure, we want to keep checking.
Let me say this quite clearly: Some charter schools are better than most non-charter public schools, and some are worse. And some non-charter public schools are better than most charters, and some are worse.
The Gold Standard crowd have a favorite method for comparing charter schools to non-charter public schools, one of which they are quite proud, but one that is so full of problems that I am shocked that they keep using it.
They rightly want to control for self-selection bias among charter school students. We know that children and families that apply to charter schools are different from those who do not, even if we do not know what all those differences are. This seems like the perfect time to do a randomized assignment, because that is the best method to make sure that these differences cancel out between groups. Luckily, we have some randomized assignments. Oversubscribed charter schools are virtually always supposed to accept students using a random lottery. This allows researchers to compare the outcomes of those were were randomly accepted, and those who were not.
Sounds good, right?
Well, it does sound good. But serious issues remain. Some are more obvious than others, and some are correctable by those interested in getting the correct answer, rather than the one that fits their pre-ordained conclusions.
If a student and/or his family do not get into the school of his/her/their choice, how might they react? I know from my own experience teaching that students who get their choice of schools take a bit more ownership. If they get their second choice, or last choice, or somehow do not get their choice, that’s a big hurdle for their teachers and parents to overcome. If parents do not get their choice of schools for their students, are they going to be as supportive of their child’s teachers? Of the assignments? Are they going to have the same kind of faith in their child’s school? I think that the answer is really quite obvious.
The problem with these studies is that the students and families who “lose” these lotteries are no longer like the students and families who “win” these lotteries. There simply is no basis for thinking that their views of their schools are like those of the lottery “winners.” In fact, one could quite simply argue that this method of analysis ensures that the “winning” charter school students are being compared to students who did not want to go to the schools they attend.
Obviously, that’s not an even comparison.
We also know that charter school students are not, as a group, like non-charter school students. That is how the Gold Standard crowd justifies their approach here. So: trying to control for applicant differences, but not controlling for ongoing peer effects? I don’t know if that is just lazy or actually dishonest. The importance of peer effects is so well recognized that I tend to think it is the latter, especially because techniques to control for them are so well established.
Of course, there is another way to look at this. From a personal level, if you have a child, you don’t care about controlling for peer effects. Actually, you want the effects to be left in so that you can take advantage of them. If charter schools have “better” students, that’s a reason to send your own child to a charter school. However, if this analysis is done for policy purposes, to influence policy-makers, then peer effects do matter. If you are thinking about all students, not just the select few who can get into the “better” school, you need to control for peer effects.
The goal of this lottery-based study design is to avoid self-selection bias in the data. However, those who use it do not acknowledge the additional selection problems they create.
The most important problem is that not all charter schools are oversubscribed, so not all charter schools can be included in these studies. This wouldn’t be a problem if we had good reason to believe that a random selection of charter schools were included, but that is obviously not the case. Clearly, the “better” charter schools are far, far, far more likely to be oversubscribed than the “worse” charter schools. This biases the sample rather severely towards better charter schools.
Unfortunately, the sample bias problem doesn’t stop there.
A really strong traditional non-charter public school is not going to lose a lot of students to a simply above-average charter school. In order to be oversubscribed, a significant number of students and/or families have got to believe that the charter school option is superior to the non-charter public school option, which suggests a level of dissatisfaction with the local traditional public schools. This biases the sample towards inferior non-charter schools.
The hardest thing in educational research — and perhaps research overall — is to be able to generalize one’s results to the broader population or wider world. And yet, that is usually the end goal of policy-oriented research.
These kinds of lottery-based studies only include the kinds of students and families that apply to charter schools in the first place. Even if the previous issues could be corrected, how can one know that other sorts of students and families would see the same benefits? The fact is that different populations might benefit less or more from going to a charter school. It is simply impossible to know from this kind of study. Of course, if you are only concerned about benefitting the kids of families who already opt for charter schools, then this is not a problem. But if you aim to help a broader population than that, you need a better methodology.
These generalizability concerns also apply to schools. Oversubscribed charter schools might well be better than average non-charter public schools, and I do not really question whether they are better than their local traditional alternatives. But on a policy level, we need to be concerned with charters more generally than that. If we raise or lift caps on charter schools, or approve new charter schools, we have to expect an average charter school to result, not an exceptional one. But these studies really tell us nothing about the majority of charter schools that are not oversubscribed. Nor do they tell us anything about the relative quality of non-charter public schools that lack charter school alternatives.
*****************
I understand the desire to find a Gold Standard for educational research. But simply grabbing that label because a methodology has some resemblance to biomedical research is not good enough, despite what Prof. Caroline Hoxby may claim. Moreover, the popular press really must do a better job of examining these claims critically, rather than cheerleading for them like this.
This, of course, means that researchers, journalists and the rest of us must be sure to take a more thoughtful stance than has become our habit.
Did you hear about the big report that came out this week? You know, the one that “shows” that NYC charter schools are better than traditional non-charter public schools? It has gotten a ton of attention, probably because it uses “‘the gold standard’ method[ology].” The report is not subtle about this. It is right there in the very first sentence of the executive summary, “The distinctive feature of this study is that charter schools’ effects on achievement are estimated by the best available, “gold standard” method: lotteries.” It even uses the term “gold standard” four more times throughout the report.
Everyone wants to follow The Gold Standard — or at least be able to say that they do. Of course! I mean, who wouldn’t? But I do not think that we actually have a gold standard in education research. In fact, I am quite sure that we do not, and appropriating biomedical research’s gold standard does not make it appropriate for us.
However, if we are going to borrow their standard, can we not at least get it right?
The biomedical standard uses double-blind experimental studies with random assignment. That means that some research participants get the experimental treatment and some get a placebo, and both are assigned randomly. It also means that neither the researchers nor the participants know who is getting which treatment. After all, expectations are important, and the mind can set us up for all kinds of things.
*****************
One of the latest ideas about The Gold Standard in educational research concerns charter schools.
We all want to know whether charter schools are better than traditional non-charter public schools. On one level, we certainly do want to know about individual schools. But on the policy level, we want to know about the average charter school, because we want to figure out if “charterness” helps a school be better. If it does, then we want more charter schools. If it does not, then we want fewer or none. And if we cannot be sure, we want to keep checking.
Let me say this quite clearly: Some charter schools are better than most non-charter public schools, and some are worse. And some non-charter public schools are better than most charters, and some are worse.
The Gold Standard crowd have a favorite method for comparing charter schools to non-charter public schools, one of which they are quite proud, but one that is so full of problems that I am shocked that they keep using it.
They rightly want to control for self-selection bias among charter school students. We know that children and families that apply to charter schools are different from those who do not, even if we do not know what all those differences are. This seems like the perfect time to do a randomized assignment, because that is the best method to make sure that these differences cancel out between groups. Luckily, we have some randomized assignments. Oversubscribed charter schools are virtually always supposed to accept students using a random lottery. This allows researchers to compare the outcomes of those were were randomly accepted, and those who were not.
Sounds good, right?
Well, it does sound good. But serious issues remain. Some are more obvious than others, and some are correctable by those interested in getting the correct answer, rather than the one that fits their pre-ordained conclusions.
Issue #1: No Placebo
Biomedical research does not just include randomization of treatment. It also is at least single blind. If some patients know that they are getting the new treatment, they might react differently. The mind is a powerful thing. They might be more diligent. Perhaps do their rehab exercises more often. Maybe pay more attention to diet. Who knows? And those who know that they are getting the new treatment might not lose hope.If a student and/or his family do not get into the school of his/her/their choice, how might they react? I know from my own experience teaching that students who get their choice of schools take a bit more ownership. If they get their second choice, or last choice, or somehow do not get their choice, that’s a big hurdle for their teachers and parents to overcome. If parents do not get their choice of schools for their students, are they going to be as supportive of their child’s teachers? Of the assignments? Are they going to have the same kind of faith in their child’s school? I think that the answer is really quite obvious.
The problem with these studies is that the students and families who “lose” these lotteries are no longer like the students and families who “win” these lotteries. There simply is no basis for thinking that their views of their schools are like those of the lottery “winners.” In fact, one could quite simply argue that this method of analysis ensures that the “winning” charter school students are being compared to students who did not want to go to the schools they attend.
Obviously, that’s not an even comparison.
Issue #2: Peer Effects
We all know that peer effects matter. Research, experience and common sense back this up. If you put a student in a class with a “better” group of peers, s/he will do better than s/he would have done in a class with a “worse” group of peers. The other kids all do their homework, or their parents were more likely to read to them, or they are somehow smarter, or harder working, or bring more cultural capital to school with them, or however else you think “better students” might be defined.We also know that charter school students are not, as a group, like non-charter school students. That is how the Gold Standard crowd justifies their approach here. So: trying to control for applicant differences, but not controlling for ongoing peer effects? I don’t know if that is just lazy or actually dishonest. The importance of peer effects is so well recognized that I tend to think it is the latter, especially because techniques to control for them are so well established.
Of course, there is another way to look at this. From a personal level, if you have a child, you don’t care about controlling for peer effects. Actually, you want the effects to be left in so that you can take advantage of them. If charter schools have “better” students, that’s a reason to send your own child to a charter school. However, if this analysis is done for policy purposes, to influence policy-makers, then peer effects do matter. If you are thinking about all students, not just the select few who can get into the “better” school, you need to control for peer effects.
Issue #3: Selection Bias on the School Level
The goal of this lottery-based study design is to avoid self-selection bias in the data. However, those who use it do not acknowledge the additional selection problems they create.
The most important problem is that not all charter schools are oversubscribed, so not all charter schools can be included in these studies. This wouldn’t be a problem if we had good reason to believe that a random selection of charter schools were included, but that is obviously not the case. Clearly, the “better” charter schools are far, far, far more likely to be oversubscribed than the “worse” charter schools. This biases the sample rather severely towards better charter schools.
Unfortunately, the sample bias problem doesn’t stop there.
A really strong traditional non-charter public school is not going to lose a lot of students to a simply above-average charter school. In order to be oversubscribed, a significant number of students and/or families have got to believe that the charter school option is superior to the non-charter public school option, which suggests a level of dissatisfaction with the local traditional public schools. This biases the sample towards inferior non-charter schools.
Issue #4: Generalizability
The hardest thing in educational research — and perhaps research overall — is to be able to generalize one’s results to the broader population or wider world. And yet, that is usually the end goal of policy-oriented research.
These kinds of lottery-based studies only include the kinds of students and families that apply to charter schools in the first place. Even if the previous issues could be corrected, how can one know that other sorts of students and families would see the same benefits? The fact is that different populations might benefit less or more from going to a charter school. It is simply impossible to know from this kind of study. Of course, if you are only concerned about benefitting the kids of families who already opt for charter schools, then this is not a problem. But if you aim to help a broader population than that, you need a better methodology.
These generalizability concerns also apply to schools. Oversubscribed charter schools might well be better than average non-charter public schools, and I do not really question whether they are better than their local traditional alternatives. But on a policy level, we need to be concerned with charters more generally than that. If we raise or lift caps on charter schools, or approve new charter schools, we have to expect an average charter school to result, not an exceptional one. But these studies really tell us nothing about the majority of charter schools that are not oversubscribed. Nor do they tell us anything about the relative quality of non-charter public schools that lack charter school alternatives.
*****************
I understand the desire to find a Gold Standard for educational research. But simply grabbing that label because a methodology has some resemblance to biomedical research is not good enough, despite what Prof. Caroline Hoxby may claim. Moreover, the popular press really must do a better job of examining these claims critically, rather than cheerleading for them like this.
This, of course, means that researchers, journalists and the rest of us must be sure to take a more thoughtful stance than has become our habit.
Subscribe to:
Posts (Atom)