Tuesday, March 31, 2009

RAS Testing Myths Forum Update

I attended the myths forum tonight and have mixed feelings regarding it. The presentation was made by 2 RAS personnel that are experts in the area of testing and analysis. They did a good job of:
  • defining acronyms and the different tests
  • explaining the general history of the "accountability" movement
  • listening and responding to questions
  • describing the real flaws of NCLB (ie includes Spec ed and non-English speaking students)
  • reviewing district "avg" performance against state and national stats
  • explaining the differences and gaps by demographics
The down sides from my viewpoint:
  • spent significant time explaining flaws of current accountability systems (ie government set too aggressive of targets)
  • used too many generalities (ie possibly apples & oranges)
  • little if any discussion of specific plans to close gaps
  • somewhat misleading or unclear graphs (ie partial y axis made differences look more significant than they are, using todays performance std distribution to estimate future performance non-proficiency rates{instead of explaining that std dist curve is supposed to shift as more kids become proficient}, etc)
  • did not detail "one school" as a case study (ie explain myths in detail and how they misled)

In summary, it was a good general learning experience with some marketing / political spin. Most importantly, it reminded me that "we need to look at the details"... Not just the headlines... (ie especially with AYP results) Both sides can find their message in the data, so be an informed citizen and look for yourselves...

MN Report Cards and Data


Sue said...

I was at the meeting also and I agree with much of what you said. I do think it was good for the speakers to point out some of the flaws in the methods of determining adequate yearly progress because these standards tend to make some RAS schools look worse than they really are. I would have liked even more information (like what are the consequences for schools that never make their AYP requirements).
And I thought the graphs were fine and not misleading. Don't be so picky.

LayLo said...

I wasn't at the meeting and just found your site - Nice.

I don't understand how these folks can continue deflecting their responsibilities. They are providing a product/service. As anyone knows, you need to define metrics to measure a product/service to be able to make any meaningful comparisons between other similar products/services.

NCLB is as good a measure as any and plenty of other schools have been successful under their criteria.

RAS needs to be vs. another.

John said...

Hi Sue,
I agree there was nothing sinister and I am being particular.. (ie man, engineer, 6Sigma convert, nerd, geek, etc)

The std distribution graphic seemed to confuse the most people. They should have had the current and planned distributions both shown, then explained the difficulties involved in increasing the school's/student's capabilities while dealing with relatively flat resources.

Showing that the legislators set the threshold to fail 70% of 11th graders was not clear or fair. It implies that the school/student performance will not improve. Which I think was the point of the targets......

Hi LayLo,
I am pro NCLB and AYP, however ESL and Special Need students need to be handled outside the metrics.Also, the improvement curve is likely too steep. With the current rules, almost all schools will fail AYP in a few years.

I am just hoping Obama does not throw out the baby(NCLB/AYP) with the bathwater.... It needs refinement, not an overhaul.