Thursday, December 8, 2011

New Harvard Study on What Makes Good Schools

This is interesting and surprising. From Business Insider online.

Two Harvard professors published a study this week that is likely to cause controversy among educators.

Will Dobbie and Roland G. Fryer studied 35 New York charter schools to see whether the traditional models of assessing a school's effectiveness, like class size, were really meaningful. Surprisingly enough, they weren't:

We show that input measures associated with a traditional resource-based model of education – class size, per pupil expenditure, the fraction of teachers with no teaching certification, and the fraction of teachers with an advanced degree – are not positively correlated with school effectiveness.

Here are the factors that were meaningful:

In stark contrast, an index of five policies suggested by forty years of qualitative research – frequent teacher feedback, data driven instruction, high-dosage tutoring, increased instructional time, and a relentless focus on academic achievement – explains almost half of the variation in school effectiveness. Moreover, we show that these variables continue to be statistically important after accounting for alternative models of schooling, and a host of other explanatory variables, and are predictive in a different sample of schools.

The study, published through the National Bureau of Economic Research, collected data including interviews with charter school principals and teachers, student surveys, lesson plans, and video observations. Dobbie and Fryer recommend implementing the five strategies named above in local public schools.

http://www.businessinsider.com/charter-school-effectiveness-will-dobbie-2011-12#ixzz1fo7cUOZt
You can download the entire paper for $5 here.

29 comments:

Cerebration said...

Hamilton Co, TN has a model school system with a strong public foundation partnership. Here is a very interesting bit of news from them. I wonder if GA Tech could manage a similar program?

At any rate - this is what some of our 'national' competition is up to:

Touring VW Academy
12/7/2011

It’s a win-win situation. High school counselors and college advisors want to know about every good opportunity for their students, and Volkswagen is looking for a rich applicant pool of students with solid math skills. So Volkswagen graciously hosted about 35 high school counselors and college advisors last week at the Volkswagen Academy, with an additional special presentation from the Wacker Institute.

VW is looking for 20 or more students per year (more) to enter their 3-year academy, which is a certified Tennessee Technology Center. Students will learn mechatronics – a combination of mechanics, electronics, computers, robotics, and automation systems – and will work to maintain and repair the robots that do so much of the work at the VW plant. Study is heavily hands-on (60-70%), with only about 30% traditional lecture; during the 3 years students will rotate in and out of actual paid time on the production floor. Upon completion, graduates will have a certificate (and possibly an associate’s degree), highly marketable skills, and will be almost guaranteed a job offer from VW, where jobs average about $25 per hour.

To be eligible for consideration, students must score a 19 on their ACT or demonstrate math competency through coursework or the COMPASS test. VW also stresses attributes like work ethic and being on time – the day starts at 7:45, which is NOT 7:46, it was made clear, and students have been dismissed because of a tendency to tardiness.

Another, more academic avenue is the 2 plus 2 program in which students would spend 2 years at Chattanooga State earning an associate’s degree in industrial technology, then potentially matriculate to Tennessee Tech to earn a 4-year degree. These graduates are also in high demand at VW, although there is no direct connection to the VW hiring pipeline.

Wacker Polysilicon North America has actually guaranteed jobs to those who complete training through the Wacker Institute at Chattanooga State. They are especially looking for chemical engineers, although they also need mechanical engineers and other technical skills. Students apply through the standard Chattanooga State application process; again, students must score 19 or above on the ACT.
These two new companies join others in the Chattanooga area that are providing really solid opportunities for students with strong math and science skills. Clearly, vocational education is no longer for those who shrug off academics – which is why it’s terribly important to make sure that ALL Hamilton County graduates have adequate skills for this new workforce. See info on TEACH/Here and on the SE Tennessee STEM Initiative to learn more about efforts that PEF and others are putting forth to improve STEM (science, technology, engineering and math) education in Hamilton County.


http://www.pefchattanooga.org/news-media/newsletter/touring-vw-academy

Anonymous said...

That the only schools studied were charter schools somewhat skews the research IMO. however, many of the ideas are sound.

I don't agree with the class size conclusion. Having not seen the entire study, I don't know what class sizes they were evaluating. The most recent vaunted Harvard study that said class size did to matter was one that compared clas sizes of 23 to 27. This was a study done at the behest of the the Florida legislature as they sought to increase class sizes that had been set by a referendum. This was not revelant in an environment of 30 or even 35 in a classroom.

Class sizes of 30+:
1. Do not allow for individualized instruction by the classroom teacher
2. Are unsafe for science labs (a subject in which American students are woefully lacking at a time that science majors are desperately needed for our future economic prosperity). NSTA studies on lab accidents clearly show this
3. Add hours to a teacher's assessment time, draining the time he/she could use to plan interesting, relevant and comprehensive lessons.
4. Are physically uncomfortable for students as movement is severely curtailed

I agree with the high dosage instruction. We need personnel who directly instruct children, not more highly paid non-teaching "coaches".

We also need more instant and easy data feedback. MIS has failed miserably in the implementation of Schoolnet/eSis. This has resulted in students "bubbling in" the answers to poorly worded benchmark tests created by non-teachers, and the teachers being forced to manually s an in the answer sheets. Data feedback has been non-existent or so tardy as to be meaningless.

Increased instructional time is also important, and teachers should be having as few interruptions to their instructional and plan time as possible. Training should consist of trainers "modeling" the lesson in front of students much of the time rather than pulling teachers away from their. Lass rooms and plan time.

The relentless focus on student achievement should be occurring on all levels. EVERY employee should be judged on what they contribute to the classroom - from coordinators, custodians, ground keeping, MIS, HVAC, etc. Teachers should be surveyed to see if the members of the classroom (the teacher and his students) are receiving the support they need. What is extremely important is that upper management that hires the teachers and sets policy, procedures and programs for the classroom must be held accountable for student performance.

Anonymous said...

The information in this study is not all really new. It's been known for years that class size (except perhaps in the lowest grades) and teacher degrees do not closely correlate to "good" schools. However, the more studies that reinforce these points, the better. Perhaps then we will see change when the data can no longer be disputed.

Also, I'd like to point out that "increased instructional time" does NOT mean longer school days. I'd hate to see the school day lengthened. The problem is that many teachers have not mastered classroom time management. Also, this does NOT mean take away recess and pack in more academics. Plenty of research and common sense says that kids need recess. Recess actually has a positive impact on student behavior and academics and should be considered an integral part of the school day.

Thanks for posting this information.

Anonymous said...

In other words, Harvard says anything that can be objectively measured doesn't matter and what they arbitrarily defined works.

Maybe high dosage tutoring can be quantified and increased instructional time (but what is "instructional time" and what isn't-there's a huge amount of room for interpretation). A "relentless focus on academic achievement?" Didn't Beverly Hall say that? Data Driven Instruction-I'm sure there's a model, but its an extraordinarily broad term. Frequent teacher feedback (from teachers to students or admin to teachers-if the latter, that's what teachers most frequently complain about-if the former, there's a limit with what they can do if they have 30 students).

There need to be more studies on what really works, but when you get jargon for the answer, one wonders if they are just trying to justify their pre-conceived notions when they started the study.

Anonymous said...

You have not taught if you don't think class size matters. Look at the NSTA research on lab safety. This is based on the number of lab accidents in varying class sizes. The rate of accidents correlated with increases in class sizes over 24:
http://www.nsta.org/about/positions/liability.aspx

DCSS administrators know these facts. That is why their official position is that science teachers are not required to conduct labs in the teaching of science.

Bhutrasgolly said...

Actually the research shows that class size makes little difference for class of less than 30 until the class size gets to be less than 15. It turns out out that a class size of 12 makes a very measurable difference.

Anonymous said...

Studies like this should not be used to paper over one very important fact: a very, very large percentage of DCSS students need intensive remediation in such basics as reading, writing, and math. Remediation of this kind can only be achieved in smaller classes.

Anonymous said...

I have always said that lower-achieving schools needed more instructional time. This require more money. I think that there should be 2 shifts of teachers. The normal 730-330 and another 1030-630 shift. Students could participate in enrichment and have actual time for free play, along with longer class times. Teachers could also be given more time to plan engaging and in depth lessons. Again, this costs money. I have no problem working more, but I want to be paid for my time. Charter schools often have extended hours, but teachers are paid as much or less than public school teachers.

atl said...

@ Bhutasgolly

30 is way too many in a classroom, especially for struggling students. Here are some studies with a much larger universe of learners over much longer periods of time.

The Star (Student Teacher Achievement Ratio) Project) is a very comprehensive and highly regarded class size study.

The STAR project was undertaken in the Tennessee and involved over 7000 students from 79 schools. For three years, from kindergarten through grade 3, students were placed either in small classes of 13-17 students; regular classes of 22-25 students; or regular classes with a teacher aide. Those in smaller classes performed significantly better on tests than those placed in regular classes. The largest gains occurred in inner-city schools.

The advantage was not only maintained in subsequent years but actually increased: in grade 4, students who had been in smaller classes were 6-9 months ahead of regular class students in reading, math, and science; by grade 8, they were a year ahead. Later, almost 44% of small class size students took college entrance exams, compared to 40% of regular class size students — the difference was greatest for African-Americans; 40.2% compared to 31.7%. 72% of small class students graduated from high school on schedule, compared to 65-6% of regular class students. They were also more likely to complete high school, to graduate with honors, to complete advanced math and English classes.

Read the entire study here:
http://www.aypf.org/publications/rmaa/pdfs/ClassSizeSTAR.pdf

And look at the Sage Project.

In Wisconsin, the Student Achievement Guarantee in Education (SAGE) program showed African American students made greater gains in the small SAGE classes than African Americans in larger classes.

"Though they started first grade with the same
academic profiles, African American students
made greater gains in the small SAGE classes
than African Americans in larger classes."

This 5 year longitudinal study followed 3,000 Kindergarten students as they compared class sizes of 12-15:1 to 21:25:1 pupil teacher ratios.

"According to the evaluators, the most significant
factor affecting individual student performance on
tests was socioeconomic status (SES), but when this
variable was accounted for, class size had the most
significant effect on student scores. All of the class
size reduction strategies used by SAGE had similar,
positive effects."

http://www.aypf.org/publications/rmaa/pdfs/ClassSizeSAGE.pdf

Anonymous said...

Way back in the 1950s-1960s era, many Catholic schools were bursting at the seams and squeezed as many as 60-70(!) students in a single class, with no discipline or learning problems.

Expectations and society in general were certainly very different back then, but it does drive home the point that there is more to education than simply class size.

Bhutrasgolly said...

To ATL, glad you agree with me-we are using the same studyis. Under 15 makes the most difference as I said.

Anonymous said...

Having taught classes of 38 and classes of 20 and classes of sizes in between, I will say that it's not class size, but room size, the children's desire to learn, the teachers desire to make learning fun, a school where students know that if they don't follow the rules, that there will be consequences, and teachers being able to give the students the grades that they earn.

I'd gladly go back and teach my 38 plus classes than the 20 to 30 student classes that I had in DCSS. My classroom size was much larger in other areas that I have taught, students came to learn, I was able to teach above the standards and keep my students growing and engaged, and I was able to give students the grades that they earned without worry as long as I could back up the grades that I gave. Also, discipline was never an issue.

If you want class sizes below 15, than send your child or teach in a private school. This is a public school. The class size is not the biggest issue, it's lack of discipline, not having high expectations for students, not holding students as accountable as teachers for learning, allowing teachers to teach above the standards and push students further, and not to teach to the test, as good teaching will result in good test scores.

Anonymous said...

@Bhutrasgolly

"Under 15 makes the most difference as I said."

That's why Title 1 Reading and Math classes are so small. Asking teachers to remediate as many as 15 students in reading and/or math out of 33 in her classroom and also serve 2 or 3 gifted is just not going to happen.

We need small group instruction for struggling students every day in reading and/or math. This is not happening in DeKalb.

Bewildered said...

Anonymous said...

Having taught classes of 38 and classes of 20 and classes of sizes in between, I will say that it's not class size, but room size, the children's desire to learn, the teachers desire to make learning fun, a school where students know that if they don't follow the rules, that there will be consequences, and teachers being able to give the students the grades that they earn.

You are "right on"!!

Anonymous said...

@ Anonymous 3:15
I taught 38 in a classroom for 6 weeks once until we got another teacher, and it was a miserable experience all around.. The sheer amount of grading and providing feedback to all of my students cut drastically into the time I used to prepare for engaging lessons. Reaching all of my students who were in the 4th grade but read below grade level (30% came in reading below grade level) was just not possible. My room was an average DeKalb County size, and I literally could not walk from the front to the back. I had to walk along the walls and squeeze through the rows. Imagine what an awful atmosphere this was for the children. Needless to say, we did not do anywhere near the hands-on activities I usually did just because there was no space. We couldn't even move the desks around to form a group.

You are right that DeKalb has serious problems with the grading system and lack of disciplinary support. However, these problems are magnified by large class sizes. There are so many areas that need to be addressed. Dr. Atkinson needs to be asking teachers what they need for their students to succeed.

Anonymous said...

Denial is really tricky. It prevents one from actually seeing the truth that is apparent to all but the one stagnated by the denial. That is what has been happening for a long time in DeKalb. The demographics; the racial make-up, the socioeconomics, the culture and the linguistics, of the county and its students have changed. That is the reality. There is no judgment here, just the stating of an obvious truth. It is this truth and all its dynamics that DeKalb must accept and realize if any real foundation for change is going to occur.
We all know that one can find a study out there that will support and evidence any position that one chooses to take. Therefore, I will not waste time pointing out nor supporting my position with any evidence other than that gained through experience and observation. It is as if DeKalb still thinks that it is educating children of their “glory days”. Anyone having a child in, or teaching in, this current environment knows this not to be the case.
DeKalb needs a dose of reality regarding the dire straits that the system really is in. We need to stop the senseless and useless finger pointing at teachers for this dire situation. It is implausible that student success is lacking because most teachers have suddenly lost the ability to teach. What most teachers have lost is the autonomy, the respect, and eventually, the motivation to do what it is we know best how to do; teach. We are mired down in mismanagement, needless paper work, senseless parental customer service, and the like. The joy of teaching has been lost in this abyss and the students continue in the rapid downward spiral of learning failure.
My hope is that Dr. Atkinson is prepared intellectually as well as emotionally to help lead and guide us out of this mess.

Cerebration said...

Maureen Downey has a posting today on this Washington Post blogpost:

When an adult took standardized tests forced on kids

It'a about a school board member who took a state standardized test and failed. This was a relatively well-educated, successful person. He started to ponder why so many students seem to be lost, and decided that it's due to the ridiculous testing and focus on the non-essential kinds of learning for the job market and even for college to advance for-profit educorps. Here's a quote:

“It makes no sense to me that a test with the potential for shaping a student’s entire future has so little apparent relevance to adult, real-world functioning. Who decided the kind of questions and their level of difficulty? Using what criteria? To whom did they have to defend their decisions? As subject-matter specialists, how qualified were they to make general judgments about the needs of this state’s children in a future they can’t possibly predict? Who set the pass-fail “cut score”? How?”

“I can’t escape the conclusion that decisions about the [state test] in particular and standardized tests in general are being made by individuals who lack perspective and aren’t really accountable.”

There you have it. A concise summary of what’s wrong with present corporately driven education change: Decisions are being made by individuals who lack perspective and aren’t really accountable.

Anonymous said...

I can remember classes of 30 students. Children sat in long rows. We did not do small group or flexible group activities. There were lots of worksheets and exercises from textbooks. Students were tracked. That means that class assignment was based on academic skills or ability. Everyone wanted to be in the smart class because they learned the most. Discipline problems were mostly related to talking in class or passing notes. If you got in trouble in school, you were also in trouble at home.

Today children of all learning levels are in the same class. A good teacher is expected to plan hands-on activities. Children work in small groups. Individualized instruction is expected. Every class will have children who are experiencing learning problems. There may be children with social or behavioral problems. Imagine having to teach a class where a child is constantly interrupting the activity. Parents are quick to blame the teacher or school situation when a child misbehaves.

Can a child learn in a class of 30 students? Absolutely. Unfortunately, large, heterogeneous classes make it very difficult for a teacher to provide the kind of instruction that many children need to be successful. It's hard on teachers. Good teachers will move on to a better school situation or leave the profession.

The current trend of sending a special teacher into the classroom to work along side the regular teacher effectively drops class size because you have two teachers working with the group. A room with 30 students and two teachers means you have 1 teacher for every 15 students.

Anonymous said...

@ Anon 12/9 7:53 p.m.

Amen! Amen! Amen!

Anonymous said...

The shifting demographics and even socioeconomics are not the main thing. It is instead the steady lowering of expectations and standards imposed by the system. This goes for instruction as well as behavior. Unfortunately, things have gotten really bad in this regard since the system leadership became dominated by African Americans. That's where the real denial is.

Anonymous said...

RE: Increased instructional time

Speaking of video observations, as a Para and TAXPAYER first and foremost, if parents were to see the LACK of UTILIZATION with respect to actual instructional time during the school day (in my classroom), parents would ask for teacher RESIGNATIONS or FIRINGS on the spot. Where are those video cameras? Can we get some installed? If I knew the administration would have my back, I'd call a spade a spade--unfortunately, that's not the case.

David Montané said...

I liked the way my science teacher in high school arranged his classes. (It was a small private school.) He let the students who were not serious about learning the hard subjects like chemistry and biology take "Physical Science", a fun class where we watched lots of science movies, the questions were True or False and he always reviewed them with us just before the test. There were at least 30 in the class and it was easy to get an A. The serious classes were much smaller, probably under 15. I'm sure for them it was easier to get an A also, because they were not distracted by, and the teacher was not distracted with, those who were not serious about in-depth learning of those subjects. They got to do lab work, where we in the larger class did not.

Anonymous said...

@anonymous 6:33pm

I teach in Dekalb.

I've often said that all voting citizens should have to spend a day in a school. Then maybe something would change.

However, you ignore DCSS's equally serious problems: the absence of varried curricula, meaningful discipline policies.

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...

Just two pieces of anecdotal "data" that I can share having read these - my grandparents moved my dad from Catholic school to public school after the 4th grade. His Catholic education was so behind they had him repeat the 4th grade..... Class size, I had a class with 12 in it a couple of years ago, just looking at EOCT scores, that was the best class I ever had. I was able to focus in on the needs of the individuals much easier than with my current course levels.

Cerebration said...

For an interesting take on technology in the classroom, read this article about the Waldorf School, a school where many high tech professionals in Silcon Valley send their children. Great lovers of technology recognize that there is a time and place for integrating technology into learning. It's most likely hardly necessary in elementary school, and increases in high school - where most of our tech budget should be utilized.

A Silicon Valley School That Doesn’t Compute

Sight Edman said...

The High Tech Heretic should be required reading for every member of the school board and all central office administrators. Maybe they can write a book report. With pen and paper.

Cerebration said...

I liken computer knowledge and the ability to use a computer and its basic software to the ability to use the telephone or television or an automobile. It's just a tool - something people use to get work done, communicate and play. Too much is made of it. If we can load textbooks on iPads or Kindles, especially at a lower cost, and allow students the ability to enjoy the journey to and from school sans a 40# book bag, then that is a good thing. Or to use the iPad to check your assignments, track your progress or communicate with your teacher would be good. But to have an iPad, just to say you have one is dumb. If it doesn't serve a helpful purpose, it's not necessary. You don't need a phone if you never call anyone.

Sight Edman said...

To some degree that analogy has merit. Probably doesn't help the classroom experience for a student to "phone a friend". Not sure we should encourage them to become "wikipediots" either. Computers are just tools, increasingly easy to use and the value of "computer skills" is highly suspect.

As for the 40# book bag, I unfortunately suffer from a high-tech career (and believe tech is a high-cost waste in the classroom) and access to math and science textbooks (via my wife who brailles them--at least those not brailled by prisoners) and quite frankly these books are simply horrible--and intentionally bloated for the appearance of gravitas. The content density of a math book is staggeringly low, riddled with PC and other useless photos, and organized to make it as difficult as possible to consume left to right, top to bottom. It is a print snapshot of a hyper-active's loosely linked web-walk. A well designed text would be significantly smaller, lighter and more useful.

Don't believe me? Pick up any 8th grade math book, open it to the first couple of pages of any chapter, where new material is presented. Now pretend you're blind. Just sticking with text, how are you going to consume this shotgun smattering of blurbs? Is there a linear story anywhere to be found? Is math really that inherently disorganized? Now the real kicker: this presentation serves the sighted no better than the blind.

Just sayin' :)