Written Testimony of Alan Siegel
Department of Computer Science
Courant Institute of Mathematical Sciences
New York University
New York
Chairman Musick, Governing board members and colleagues,
Let be begin by expressing thanks for this opportunity to testify before you.
My name is Alan Siegel. I am a professor of computer science at New York University. I am a former Director of Industrial Relations for my Department, and am proud to have taught, years ago, remedial mathematics to inner city teenagers.
Today, I am speaking not only for myself, but also as a member of the Courant Institute of Mathematical Sciences, and as a representative of NYCHold, an organization of parents, businessmen, educators, engineers, mathematicians and scientists who share deep concerns about K-12 mathematics education nationwide.
At the outset, I think it appropriate to acknowledge that yours is a difficult task, and you deserve our thanks for your public service. I have spent a considerable amount of time at your web site, and have seen the results of what is clearly a great deal of hard work and innovation. Your site offers powerful insights into mathematics education across America, and it shows your efforts to adapt to ever changing and even conflicting expectations and responsibilities.
Mr. Chairman, I also read your congressional testimony, and understand, to some extent, the nature of your mandates. It seems as if Congress has directed the Governing Board to set the direction for the NAEP, but hasn't given the Board much authority to govern the actual process. Nobody ever said that public service comes easily.
President Bush has pointed out how school programs based on low expectations fail to serve our children, and especially those whose families are in greatest need. And he has asked us to embrace the challenge of ensuring that no child is left behind.
I would like to take a few minutes to offer my own perspective about these notions in very concrete terms that include the responsibilities of the Governing Board, and more. In particular, I would like to discuss:
Why high quality mathematics programs are crucially important.
What it means to leave no child behind.
The difficulties associated with improved expectations -- from political constraints to matters of transition engineering.
Ways the NAEP might contribute to an orderly transition toward improved learning.
First,
crucial importance.
Everyone understands that higher education in general, and advanced technical
capabilities in particular, offer access to higher paying jobs. High technology
is also crucial to the wellbeing of our economy and to our national security.
As a former Director of Industrial Relations for my department, I had the opportunity to see firsthand how shortages of technical staff forced industrial research and development projects to be cancelled and promising work to be abandoned.
As a former director of our PhD fellowship program, I managed a student support budget of about one million dollars annually. We work hard to recruit qualified American students for our federally funded fellowships, but I have to tell you that we have more supported students from mainland China than Main Street USA. This is a national trend. According to an NSF/SRS study, by 1997, more than half of the PhD students in Computer Science came from abroad. In engineering, the percentage was about 47%. Moreover, these percentages have been growing. As a consequence, many of the best paying high tech jobs will be awarded to foreigners. An unintended consequence is that some tiny percent of foreigners who are sent here to conduct technical espionage will be trained by U.S. dollars and placed with our help in sensitive positions across America. The evidence suggests that this might have already happened at our national weapons labs and elsewhere.
Mathematics is the gateway to many specialty careers. But it is much more than that.
In the Saint Elizabeth Medical Center in Kentucky, nurses have to score 80% on a math test each year. If they don't pass, they don't work. Personally, I would prefer a nurse who understands the doses, the measurements, the conversions and even how some prescriptions are adjusted by body weight and distributed over time.
Pilots have to make complicated calculations about fuel reserves, the time to meet altitude requirements, and other flight information. While they are aided by on-board computers, they have to be able to do the calculations themselves in case of a failure. They have to program their systems correctly, and have enough awareness to recognize errors. Every year or so, we hear about a mistake where, say, the ascent calculation was wrong, and a plane slammed into a mountain.
Recently, I ordered some specially designed ductwork for my house. My sheet metal contractor knew his geometry. The duct was a perfect fit, even though one end was square, and the other rectangular and shifted off center. My plumber knows his slopes. He has to. Otherwise the waste lines won't work properly. To tell you the truth, I still don't understand where ventilation lines are required. And as for carpentry, did you ever wonder if it is okay to use a four by eight beam as a substitute for a two by twelve? (It isn't.)
Math matters.
So
what does it really mean to leave no child behind?
Let me be clear. Access to a sound mathematics education should not be reserved
for the top 10 percent, and it should not be restricted to exclusive communities
and their schools of privilege.
But to leave no child behind implies that you have a vision and path for moving forward. So we must first ask what might be feasible.
The Third International Math and Science Studies offer a powerful opportunity to see what is possible, and how far we have to go.
As you know, the TIMSS reports are a real eye-opener. They show American 8th graders performing at a level comparable to the international mean. In contrast, our twelfth graders performed far below their cohorts in every industrialized country. In fact, our twelfth graders only managed to outperform Cyprus and South Africa.
As for our eighth-graders, it is instructive to compare their performance against that of Singapore.
In Singapore,
some 46% of the students scored among the top 10 percent
worldwide.
93% were in the top half.
Only 1% of their students were among the bottom 25% worldwide.
Now that is what it really means to leave no child behind. By these standards, we are out of the running.
Personally, I was very saddened to hear testimony suggesting that algebra is somehow inappropriate for a third, or so, of American high school students. Why should they be denied access to a training available to students in Singapore? It almost seemed as if you were being asked to test mathematics below the level necessary to fill out a tax return, or to select the best loan repayment program.
Of course, the failures of the past should not be revisited; but by the same token, past failures by many well intentioned textbook writers, mathematics reform programs and school administrators must not be viewed as proof that America's students cannot learn and, therefore, should be neither taught nor tested.
Back home in a significant portion of New York City, new mandates are requiring the adoption of high school mathematics programs that, according to School Chancellor Levy's own Math Commission, might not be appropriate for the college bound. In the affected districts, only two high schools have been exempted. That is about as elitist as you can get. We are also experiencing a boom in the tutoring business that seems to coincide with the introduction of these programs. But very few parents with limited education can afford outside services, and these families are the least able to provide the necessary supplementation at home.
Low expectations offer the easiest, cheapest, and most expedient quick-fix answers to the problem of inadequate education programs. But they perpetuate inequity, and have long term costs that we can ill afford.
Mr. Chairman, I spent hours studying your sample questions on the web. It is fair, I think, to characterize the majority of twelfth grade questions as being much too easy, and at a grade level that is often somewhere between grades 6 to 8, and almost never above 10. This would be scandalous, except for one thing. Your statistics confirm exactly what the TIMSS data shows. Our high school students are so poorly prepared that the majority cannot even live up to the low standards of the NAEP.
But if your NAEP assessment exams continue to use questions that are well below grade level, you will limit the ability of the American public to know if the latest programs are really working. And there cannot be any doubt: the public deserves an independent assessment authority. Otherwise we will not have the tools to tell if new programs are ensuring that no child is left behind, or are following a path of expectations so low that no child can get ahead.
So what should be done? Let me begin indirectly with a negative recommendation.
Recommendation 0. Do not change the assessment structure very much. The worst thing you can do is to repeat the transition mistakes of some localities, which tried to legislate substantially higher standards without formulating any realistic plans for achieving them. You have an obligation to maintain consistency in your exams, and can ill-afford to take steps that are unlikely to succeed, and might even retard real reform.
I believe that the American public has no notion of how seriously deficient public mathematics education really is. And I doubt very much if they are aware of how much is at risk. Even the organization of your own web site tends to hide the problem. Frankly, I sympathize with your dilemma. Bad news is never welcome, and the messengers of misfortune, history reports, are seldom rewarded for a job well done.
But if you are as concerned as I am that the wellbeing of our country and our next generation of Americans is at risk, or even if you just believe that our mathematics teaching should be an order of magnitude better, or if you believe that Americans deserve a training that is no worse than one standard deviation below the standard available in Singapore, then you might want to consider steps to strengthen public awareness.
My first suggestion is a bit radical, and for that I apologize. But I wanted to suggest something that might increase awareness and calls for action, but in a safe and orderly way. I am an academic and not a politician; frankly, I am confident that you will find better alternatives. So without further apology, here are:
Recommendation 1. As a kind of reality check, work to secure the participation of Singapore in your NAEP testing program. Better yet, offer the twelfth grade exam at several grade levels, possibly right down to the sixth grade. Chances are the scores might do more to awaken national concern than anything we can do solely within our own country. The American public is entitled to know how their students are performing in international terms.
Recommendation 2. Begin experimental sampling and data collection for a broader and deeper range of questions so that you can assess improvement in the full spectrum of primary and secondary school mathematics. Otherwise, your tests might fail to detect significant strengths and weaknesses in local and regional education programs. Similarly, a better variety of questions can provide new understandings about the kind of training and knowledge students need in order to progress.
For example, do students need to master fractions before they study algebra? It is fair to suggest that many mathematicians with middle school teaching experience believe that the ability to manipulate fractions successfully is key to mastering algebra, which uses the same kinds of operations but in more abstract and more difficult ways. A NAEP test that includes a complete battery of elementary arithmetic questions might help resolve this matter.
Recommendation 3. Current NAGB policy appears to view the matters of framework formulation, NAEP exam specification, and NAEP exam grading policies as different aspects of the same objective. This is a mistake.
The framework should probably articulate objectives and goals for content and mastery that look well beyond the next examination date. This gives states and school districts a tool for longer term curriculum planning.
Your tests should as a matter of policy cover all grade level appropriate mathematics, regardless of what some organizations might see as less important. With your freedom to use statistical sampling, there is no excuse for refusing to discover what our children are and are not learning. A twelfth grade exam specification should include, for example, trigonometry problems and questions that demand a sound grasp of algebra.
Grading policies are the appropriate vehicle for maintaining consistent scoring. You can allow all kinds of experimental questions that are not used in the scoring, or are given very little weight.
But by omitting certain kinds of high school material from your questions, the NAGB gives an implicit endorsement to their omission from state frameworks and local curricula. This is a very serious mistake. Instead, you can -- and should -- use these problems to serve notice that the content is important. They would also serve notice that the material might eventually become part of your scoring base. Moreover, the data you would get is precisely what you would need to engineer an orderly transition to better standards without having to risk the damage and backlash associated with new performance goals that cannot be met.
Incidentally, it is worth pointing out the full reason I chose to distinguish between the NAEP framework and NAEP exam specifications. According to hearing testimony, the framework -- whether you like it or not -- influences curricula and state frameworks. Consequently, you should see your framework as an independent tool for improving performance. It should reflect planned evolution for the NAEP, but should not be viewed as a perfect specification for the next exam -- it won't be anyway, and education reform should be based on a vision that exceeds the four year outlook to the next testing period.
Recommendation 4. Improve your web site to offer access to 100% of the (non-confidential) data. Then researchers can look for understandings that your staff might have overlooked, and can lend independent support to the statistical inferences you have made.
You have micro data that is available to researchers on a restricted basis. If necessary, strip out some of the personal data that might allow students to be identified. But in any case, make this information public.
(I might add that your web access to mass data is currently misconfigured. It took me 11 clicks per block of questions to download, and even then the data transfers did not include all of the information that you offer for each question.)
Recommendation 5. Get approval to include an information technology specialist, a statistical analysis researcher, some engineers, and some high tech employers on the Governing Board. These individuals are not only likely to offer new perspectives in mathematics education; chances are they will enrich your working set of feasible options as you endeavor to adapt your exams, your statistical processing, and your policies to meet the increasing demands of a more complex world.
Recommendation 6. Lobby to have an external review of your organizational structure by a management consulting firm, or possibly the O.M.B. The NAEP governing, advising and steering committee members as listed in various NAEP documents add up to more than 60 participants. Such large numbers can diminish accountability and reduce effective oversight. The NAEP Assessment Governing Board is responsible for the mathematics framework of goals, but seems to be less responsible for the creation of the actual examination. As is often the case with distributed responsibilities, there appears to be an absence of checks and balances at many critical junctures of the governance process. For example, it is not clear how the correspondence between the actual NAEP examinations and the framework is ensured.
In closing, I would like to focus on one of the published NAGB objectives, which is to seek a consensus. It is evident that the NAEP test represents, in some sense, selected content areas and question levels based on consensus. This is a very problematic approach for designing national examinations and frameworks. By adopting a stripped down framework, you endorse testing for the least common denominator and, thereby, implicitly endorse education policies so weak they might not even teach what an l.c.d. really is.
Please remember the children. Think of their futures. Recognize the paramount need to know how well we are educating them. The training they receive is no better than their knowledge and skills in high school, and the twelfth grade in particular. And we must, of course, know how they are progressing all along the way.
Bluntly put, there is simply no excuse for adopting policies that might accidentally mask weakness in our education programs. When speakers point out how unfair it might be to really know how well students have been taught algebra and other high school topics, stand ready to ask the obvious question: unfair to whom? After all, the NAEP does not and cannot give individual student scores.
The NAGB has a remarkable opportunity -- and, I believe, a civic obligation -- to tell all and thereby to contribute toward steady, substantiative reform in mathematics education. But in my opinion, you cannot achieve these objectives without a thorough reconsideration of your policies, your methodologies and your mission.
Alan Siegel
Excerpts from this testimony were given in an oral
presentation before
the National Assessment Governing Board in Washington, D.C. on 9/24/01.