Thursday, July 15, 2010

Data Manipulation Gone Bad

So, I was purposefully going to use the data to show that RAS was doing better than the State in various classes. My hypothesis was that we were being unjustly criticized because of our relatively high poverty... Man, I had my soap box ready and everything !!!

The problem with data however is that it sometimes surprises us. See below... Thoughts? What factors am I missing?

If I get time, I may do the same thing for language, mobility or race....


2 more pics recently added for you to ponder... Too bad the tool can not give me Race and Financial at the same time. It would likely be an interesting chart. Remember you can access these at. RAS Report Card Then select Analyze Your Results, far down on the left side.


8 comments:

Anonymous said...

It would be interesting to know what expenditures per pupil are, since the educrats claim that it matters greatly. Class size might be another factor, though I'll wager it isn't.

What you are going to find, I think, is that academic results are almost entirely a function of the expectations set by administrators, teachers and parents. I doubt that such things are measured, of course, but after we eliminate all the other factors, like class size and money spent, this may be the only explanation.

J. Ewing

John said...

Well, here is what is rolling around in my head, whether it makes sense or not.

Through our district, we RAS citizens offer a great deal of variety and choice for the students. Which is a good thing because it keeps some families in the district that would have otherwise taken flight. Thereby helping to maintain additional funding to cover the fixed and admin costs. And thereby maintaining a solid "academically focused student/family" component within the very diverse district and community. (ie win/win)

I understand these benefits, however I am thinking there should be an additional academic benefit somewhere in providing these.

Some group of RAS students should be outperforming their "state level" peer group. I mean many districts do not offer these benefits to their student body. Especially in the rural communities.

So, are the kids benefitting academically in the core areas of learning. (ie reading / math) Or do we maintain "Choice Programs" primarily to market, and maintain head count. I'll keep looking.

Anonymous said...

I'm all in favor of offering students a wide range of choices, but I think some choices simply cost too much, and school districts are both unwilling and unable to look at these programs objectively. That drives up the total cost but doesn't produce any tangible benefit. Let us suppose, for example, that we offer Chinese as an elective, but must hire one teacher for that, and she teaches only one class. We are out $70,000 per year and we have 21 students taking the class. These students have been enriched, and met the state foreign language requirement, but their knowledge of English and math has not increased one iota.

What needs to happen is that the district needs to create a program-based budget, based on expenditure per pupil. In this case, we would find we were spending $3,333 per pupil, while our cost for teaching Spanish was only about 5555. Both fulfill the state language requirement, but one is too costly. Dump it, or find a way to bring its cost into line (hire a teacher that can teach both, or convert Chinese to remote learning, or web-based class).

J. Ewing

R-Five said...

My cursory look at your graphs has me a little confused. It looks like the math scores in the by grade chart are much worse than the by group chart.

Regardless, I prefer not to bash RAS too much, at least not this year when Supt Sicoli had to spend much time on administration. And I don't see huge differences with State averages. It will be difficult to do much better with all the legislative and union rules in place, problems that need fixing in St. Paul, not so much here.

We can focus on the budget however. If the scores cannot be moved because of State / union interference, there is no point paying a still premium price for it.

John said...

I guess from my perception, I am not looking for or wanting to hear bashing. I am looking for the theories and insight of the readers.

We have overwhelming data that shows RAS community students are consistently just meeting or under performing the State averages. These similar results occur in many different student classifications.

The reality is that those other districts are dealing with the same state rules, and similar Union/Budget issues. Therefore they are of questionable importance to the comparisons.

The biggest factors in my mind are:
- The make up, culture, beliefs, actions, knowledge, etc of the RAS students and families.
- The programs, actions, culture, beliefs, etc of the RAS school district and it's employees.

A third difference would be the beliefs, actions, support, etc of the RAS area citizens. However, would this really impact Reading and Math scores?

We have data, now we need to understand the root cause of the problem. (ie don't fix a symptom) One tool I use to do this at work is the 5 whys method Take a look and give it a try.

First question would be:
Why are RAS students performing at or below the State averages for their peer group?

R-Five said...

Maybe the testing methodology (MCA e.g.) doesn't travel well. Works in stable areas like Edina, not so well in Robbinsdale with its higher mobility. Would swapping the staff and/or facilities of these two districts move the results of either? Somehow I doubt it.

We should focus more on what did you learn while you were here, and not count any student entering or leaving during the academic year.

Anonymous said...

while it's true that it is unfair to blame the Robbinsdale schools for students that they have not taught for very long, I'm afraid that excuse won't wash. First of all, these scores are below the State AVERAGE. It is therefore as likely that a new student would be raising the scores as lowering them. And if the Robbinsdale schools were doing their job properly, then those mobile students that had arrived, three or four years ago would not be performing ABOVE State average and again, we shouldn't be seeing these poor scores.

I would like to go back to the "five whys" approach for a moment. Question two is "why aren't these kids learning what they need to learn to do better on these tests?" The third question is "why aren't these kids being TAUGHT what they need to do better on these tests?" Good question. Next?

J. Ewing

Anonymous said...

correction, please. "those mobile students... would NOW be performing..." Makes a big difference.