Saturday, August 29, 2009

NCLB and Teaching to the Test ?

The following is a comment from my AYP Top 10 Post. I thought it deserved more attention so I brought it to the top.
"Here's a good take on one reason why I think NCLB is deeply flawed. Yes, it is capable of boosting scores (and according to some, achievement) among the low average and average learners. But the high achievers are not coming close to their potential as they tread water in a teach-to-the-test environment. Your thoughts?" Anonymous

NY Times - Smart Child Left Behind

"So what does all of this mean? It is clear that No Child Left Behind is helping low-achieving students. But it is also obvious that high-achieving students — who suffer from benign neglect under the law — have been making smaller gains, much as they did before it was enacted. Alas, this drug is producing no miracles." NY Times article


From my reading of this, NCLB has made significant gains for the "unlucky" students and had no reported negative ramifications on the "lucky". Since the lucky student improvement has continued at the previous rate. Pretty impressive to make these gains and close the gap by a grade without negatively impacting the high achievers. I saw this article as good news and am relieved.

Just a thought on knowledge gain growth rate for the high achievers. If Bobby is scoring 96% on his capability tests whereas Jimmy is scoring 45%, we would expect a much larger growth rate for Jimmy if an equal investment was made in their training. And Bobby's rate would be lower. (ie perfection is really hard...)

An interesting twist on this, if only 90% is needed to be extremely successful in society, then investing any more into Bobby would actually be a waste of public education funding. The "Outliers" book will help you understand the importance of "smart or good" enough.

Now folks, help me understand the working definition of "teach to the test environment"? And how you believe NCLB/AYP mandates a teaching method/environment?

20 comments:

Anonymous said...

Now folks, help me understand the working definition of "teach to the test environment"?

An extreme version of this occurs with SAT prep courses, where kids are taught the rules of test scoring, and techniques for identifying wrong answers. It can be extraordinarily difficult to draft a multiple choice test where the wrong answers might contain what you might call a "tell", that is some sort of suggestion that a particular choice is wrong.

Are previous tests available to teachers? If so, it would certainly make sense for teachers to study what has been tested previously, and use that as a guide as to what is taught going forward. As it is, NCLB tests only reading and math, so it also makes sense to emphasize those subjects to the exclusion of others in curriculum planning.

Where testing is concerned, there are always a lot of ways to game the system, and the narrower the focus of the test, the easier the system is to game.

John said...

So do we think that the public school administrators and teachers are prone to cheating? If they were given the test with answers before the exam. Would they pass them to the students in order to improve the results? It is an interesting concept....

Or would they focus on making sure the students know those subjects by using the most effective methods? I believe the latter, since these folks chose education as a career.

I agree NCLB & MN's AYP is driving focus on the 3R's and science. Not sure if that is such a bad thing.

As for smart test taking skills and the SAT. Those will only gain the test taker only so much improvement on their score. They had better still know their stuff if they want to score high.

Anonymous said...

"So do we think that the public school administrators and teachers are prone to cheating?"

I do think there are administrators who do "cheat". But there are lots of ways to game the system, and many would argue do not rise to the level of cheating. I wouldn't characterize the techniques SAT prep courses teach as cheating, but they do mean the tests tell us something different than what many of us think they do. My general opinion is that the only thinks tests measure with certainty is the ability to take a test. Anything we conclude beyond that is something of an extrapolation.

It's been a while since I took the SAT's, but I believe these days old tests are made available freely for use in preparation. In various schools I have attended, old tests were on file to be used for study prep.

Anonymous said...

Basically, President Bush's first Secretary of Education was pushed out of office because of cheating that occurred in schools under his supervision. It's not an uncommon phenomenon.

Anonymous said...

I don't know that it's the academic definition, but from my perspective: There's a wide range of important, age-appropriate skills that a given teacher in a given school can share with her class. Depending on the personalities, the abilities, the demographics or any number of other variables that teacher will create or adjust lesson plans to give a rich, full learning experience to the class at hand.

NCLB has narrowed that range to a rifleshot target--that same skilled teacher will know roughly what will be on the test (it doesn't change that much from year to year) and s/he'd be foolish not to plan lessons around gaining that specific bit of knowledge. Administrators encourage it (who wants to be a 'failing' school) and this one snapshot of a test has come to be the measurement of total success for a school/teacher/district.

What is left behind? Higher standards for higher achievers, more realistic standards for kids with lower abilities or those who may not do math well but have different, highly useful abilities (spatial, artistic, emotional) and those with learning disabilities. Teachers' creativity and passion. Field trips, music, art--anything that doesn't show up on NCLB is deliberately 'left behind' as the test becomes the focus of the year's learning.

A good teacher friend of mine said that being a teacher isn't being a baker--you're not baking cookies, and you can't treat every kid the same and expect to get the same product. Using one test as the one standard of measurement is so patently useless (and especially since you're not measuring any individual student's own progress--only one year's scores against the previous year's scores) that I find it astonishing that we've given so much weight to this flawed product.

But that's just me. . .

John said...

So Anonymous,

I agree that music, art, sports, etc are important, however I think they must come after a base proficiency in English reading, English writing, math and science.

As for creativity, spatial relations, field trips, special techniques for different students, student values and emotional learning, these are even more important now that we are checking the Teacher's recipe. (ie kid's capability and achievement) The check does not specify what the Teacher does to enable the unique child to learn, only that the learning sticks.

The most creative and adaptive Teachers will definitely have the best results. Please review the books from my recent posts for further information on this.

Please give us your thoughts on a potential alternative to hold the educational system accountable to ensure the following priorities are fulfilled, and almost all young citizens will be ready to join the American work force when they graduate. It is a pragmatic reality that the citizen's want this payback for their investment in public education.(ie lot of social security benefits to pay...)

G2A School Priorities"

Or, do have different priorities you would put forth ? With a fixed budget, how would you decide what to cut ?

A thought on the cookie comment. The truth is that consumers are indifferent to the baker's method and recipe. However the final product still needs to be delicious if it is to be a commercial success. It is an interesting and accurate analogy.

Thanks for your thoughts.

John said...

Jon,
Should stop using SAT's, ACT's, classroom testing, etc? Since people can learn from studying past versions...

Alternative ? No more testing....

Anonymous said...

"Should stop using SAT's, ACT's, classroom testing, etc? Since people can learn from studying past versions...

"Alternative ? No more testing...."

I am all for testing, but always with an understanding of what a given test does and what it's for.

The SAT folks used to say the prep courses didn't help. The interesting thing was that in saying that, they were simply making an assertion without any evidence to back them up, or without feeling the need to get such evidence. Independent studies proved them wrong, and now the SAT folks prepare (and sell) their own prep materials which incidentally strikes me as a conflict of interest. When old tests became available for analysis, they were found to have ambiguities and errors. Multiple choice tests are a lot messier than they look on paper.

This is not to say SAT shouldn't be given or used. I think they are of value. But as always, they are just one tool in evaluating whatever it is we are trying to evaluate.

Anonymous said...

For a comment on the Times article, check out Monday's posting at dailyhowler.com.

R-Five said...

Unrelated thoughts:

1. Somehow, I'm just not all that concerned about "gifted" and high achieving students.

2. NCLB has identified our schools' shortcomings better, but the admin cost seems high for this amount of info.

3. Since each state sets its standards, NCLB data is not nationally useful. States should do their own, drop NCLB, probably budget neutral giving its high admin costs.

4. Teaching the test is still better than nothing.

Anonymous said...

"1. Somehow, I'm just not all that concerned about "gifted" and high achieving students."

It's tempting to overlook the bright kids in class in order to meet the needs of the kids who are struggling. And NCLB encourages that. But I think we have an obligation to teach all our kids, not just those within shouting distance of passing AYP.

"2. NCLB has identified our schools' shortcomings better, but the admin cost seems high for this amount of info."

Schools do a lot of testing as indeed they should. For myself, I don't know whether the NCLB tests are more expensive than the others, but I do think a lot of the other tests give us a clearer idea of what's going on in the classroom. Most cells pass AYP, but not even I believe that everything is perfect in those that do, or that the teaching in those cells doesn't need to get better.

"3. Since each state sets its standards, NCLB data is not nationally useful. States should do their own, drop NCLB, probably budget neutral giving its high admin costs."

Well, that was a trade off to get NCLB passed, a political solution to an educational problem. There are national tests we can look at NAEP, SAT, ACT and others. But as a practical matter states can't drop NCLB because they need the money.

"4. Teaching the test is still better than nothing."

There are lots of things better than teaching to the wrong test. A testing system which encourages the neglect of our best students is deeply flawed.

Anonymous said...

I'm going to ditto Jon's sentiments in the above post.

I'm not opposed to testing at all--I think it can provide a good measuring stick of how a child is growing and learning and reaching their potential, and of how a teacher is performing when it's used over an extended time period. NCLB is one snapshot, not compared to any other.

The comparison I use is if we decide to combat obesity by declaring that EVERYONE should have a BMI of 20--tomorrow. And everyone who currently has a BMI of 20 should have a BMI of 19 next year. Meaningless, especially if each state sets its own unique BMI formula, and even if implemented properly--not achievable.

I actually don't have a problem with SAT, because as it's used, it tends to be one piece of a larger picture. Class rank, extra-curriculars, citizenship all play into the bigger student profile, as it should be.

As far as not caring about G&T students. Yeah, let them coast on their own abilities--at your own peril. Those are the kids who bring our district merit scholars, outstanding alumni, and raise the bar for everyone. Ignore them, and they'll become bored and disruptive, or else happily transfer to Wayzata, SLP, privates, etc. How does it benefit our district to have a higher concentration of lower achieving students?

And, to be entirely self-serving, they're the ones who will likely be the innovators, the leaders, and the entrepeneurs of tomorrow. I'm interested in nurturing them, as well as the average and struggling students, both on general principle and for the benefit of our society.

Interesting discussion.

John said...

I agree with R-Five that I am not worried about the gifted and talented students. My rationale is that there are so many parents focused and lobbying for them that their programs will not be lost in pursuing AYP.

Look at RAS offering RSI, IB, AP, multiple languages, free transportation, etc, etc, etc. And the outcry when anything for the lucky kids is threatened...

Now other than AYP and NCLB, who is looking out for the unlucky kids? Often their parents don't know how to lobby for their kids, are unwilling to, or are unable to... (see "Whatever it Takes")

Thoughts?

Anonymous said...

Look at RAS offering RSI, IB, AP, multiple languages, free transportation, etc, etc, etc. And the outcry when anything for the lucky kids is threatened...

------------------------------
These programs are available to all students, as far as I know.

Additionally, any special needs student has an IEP. The district is required by the federal government to meet all the needs of that student, which can range from individual 1 on 1 aids in a 'regular' classroom to small-group instruction, etc.


I strongly suspect the amount spent per student is going to be vastly in favor of the 'unlucky' students you mention. I don't begrudge them the resources to meet their needs, but I resent that it's made into a zero-sum game where we pit one group of kids against the other.

John said...

Pre-AP, IB and AP are only available to kids that are academically advanced. (pre IB, I'm not sure???)

RSI enrolls a month before all the other elementary schools, so only really involved parents know to apply. Therefore the 7% poverty, and a district task force working to improve the sign up process.

Cross district transport only applies to the above classifications.

The rough "guesstimated" costs/student I use are:
- Lucky kid in local school: ~$6,500/yr/kid. (ie majority, including my kids)
- Lucky kid transported to cross district choice program: ~$8,000/yr/kid
- Unlucky/challenging kid: ~$13,000/yr/kid
- Significant Special needs: $26,000++/yr/kid (minority)

No one has disputed the numbers yet... Now, what to do about this reality as our district has cleared the 40% poverty mark...

Raise taxes or improve productivity or eliminate waste or sacrifice a student group ??? I vote for the middle two.

Anonymous said...

Pre-AP, IB and AP are only available to kids that are academically advanced. (pre IB, I'm not sure???)
-------------------------
Pre-IB is the curriculum at Lakeview Elementary. All children participate. It is meant to boost the enrollment of *all* kids in the middle years and upper IB programs, thus making them more inclusive.

RSI is working to increase its diverse enrollment, as you mentioned. I consider it like ECFE--an incredibly valuable program that shouldn't be discontinued, but used more broadly. Though, if you really want to increase diverse enrollment from across the district, you'd better plan to continue transportation--you absolutely will not get acceptable levels of enrollment if there's no way to get low income kids to the school.

So, IB and AP are really on your hit list? Based on your suggested cuts, it looks to me like you're more than happy to sacrifice a student group.

Hypothetically--if we can get more kids from diverse/ELL/low income backgrounds into the RSI/IB/AP programs, and decrease their 'price tag' from your suggested $13k to $8k by essentially moving them up the luck meter, isn't that a gain both fiscally and in the kids' future?

Christine said...

What I have heard about NCLB is not complaints that it measures the wrong content. Teaching to the test isn't bad, if the test measures the correct things. People seem to not like their schools being labeled as failing to make AYP when it's only the poor kids that fail.

Christine said...

And while I think discussion of RSI is off the original topic, I'd just like to point out that Spanish Immersion cannot really be used in broader application.
You can't take ELL students, most special ed. students, or any student who may move outside of the district in six years' time and successfully put them in the RSI program.

John said...

Anonymous,
Please go back a few months and check out my views. I think I have been consistent.

District transportation should be paid for by those who can. And subsidized for those who can not. Just like ECFE, RAS pre-school, sports, etc. The cost for this special district transportation is ~$400,000/yr...

As for AP/IB, I think we only need one of them. Preferably pre-AP and AP because they are ala carte. Kids can be challenged in their specific area of strength(s). They do not need to be academically gifted across the board to get into a very segregated program. (full disclosure: I am biased because my kids are in pre-AP and AP... since we are PMS/Armstrong community members)

Unfortunately, moving from unlucky to lucky requires a lot more than moving them into the IB/AP/RSI programs. I am 60% of the way through "Whatever It Takes". It looks like a lot of work and true dedication. I am not sure how we are going to get there...

Christine, Thanks as always for your "on the money" comments !!

John said...

Anonymous,
I forgot to mention, I am not sure why the district decided to offer elementary IB at Lakeview. It seems they should be able to offer a good solid common curriculum at all the elementary schools that delivers results. Maybe the "IB" trademrk is a good marketing pitch. I am not sure.