STAT Year Three Year-end Evaluation: UPDATED

Reviewed data and recent evaluations by Johns Hopkins University reveal some slight or statistically insignificant academic gains found at BCPS schools could not be attributed to STAT, since other programs/efforts to increase achievement had also been put into place, principals and outside experts indicated. And “lighthouse” or pilot school comparison figures provided by BCPS for students assigned laptops 1:1 appear to be “cherry picked” as well.

Do such lukewarm outcomes justify the exorbitant costs or support an expansion of a nearly $300 million digital initiative, for the first 6 years alone, and $60 million a year plus millions in digital curricula?

Despite Dr. Dance’s departure, Interim Superintendent Ms. Verletta White says she is committed to STAT.  As she noted in her message to the BCPS community, First-Week Thoughts, July 6, 2017:

“I do want to make clear that we are not changing course or introducing new initiatives.  Our schools are doing well, and technology is a key leverage tool for personalized learning. S.T.A.T. digital learning and Passport elementary world language instruction are just part of how we do business.”

The STAT digital learning initiative kicked off in BCPS in 2014.  Three years later, here are Johns Hopkins University’s Center for Research and Reform in Education’s (CRRE) Year report and slide-show presentation for 2016-17.  Both were presented at the Board of Education’s August 8, 2017 meeting (click on Meetings tab).  Video available here.  STAT report begins at about 1 hour, 11 minutes into the meeting.

Highlights from the 8/8/17 Meeting:

• JHU researchers: rare to see P21 skills integrated into instruction
• JHU researchers: education research is biased; in reference to successful 1:1 initiatives, it’s the “survivors” that make it into research studies (and there aren’t many of them)

UPDATED TO INCLUDE INFORMATION FROM 9/14/17 CURRICULUM COMMITTEE MEETING AND 9/26/17 BOARD OF EDUCATION MEETING

The BOE Curriculum Committee discussed STAT at its September 14 meeting; the meeting was recorded and archived.

Comments made at the meeting:

• BCPS is proud of progress achieved, but recognizes they have work to do.
• The system is moving in the expected direction at the expected pace.

STAT was also discussed at the September 26, 2017 Board of Education meeting during a REPORT ON STUDENT ACHIEVEMENT – MULTIPLE MEASURES OF PERFORMANCE (https://www.boarddocs.com/mabe/bcps/Board.nsf/Public Meetings tab, select 2017, select 9/26/17 meeting; presentation begins around Minute 1:39)

Moving beyond just MAP and PARCC, BCPS looked at a “constellation” of measures.  KEY POINT made at Minute 1:52 ~ Kindergarten readiness continues to drop; students are coming into the system less prepared (6 out of 10 students).  This is due in part to poverty.  There’s a strong relationship between poverty and student achievement.  We’re confronted with poverty — higher levels in BCPS elementary and middle schools than through the state as a whole — and we’re looking to close achievement gaps over time.

STAT-us BCPS Comment:  BCPS knows that poverty is the most important indicator, yet hundreds (and hundreds) of millions are spent on devices (and everything that goes along with them) to close gaps (and a close review of the BCPS Multiple Measures presentation shows very weak results) instead of addressing poverty and its effects?   What about community schools with wrap-around services?   What about expanding feeding programs?   What about small class sizes, increased support staff, and mentoring programs?

A chart showed that PARCC scores were higher at 10 Lighthouse schools vs. non-LH schools and state schools.  This is misleading; 3 of the 10 LH schools (Fort Garrison, Mays Chapel, and Rodgers Forge) are in economically advantaged areas.  One school, Joppa View, is in a somewhat advantaged area.  Their scores (proportion meeting and/or exceeding CCR) brought up the overall average of schools in economically disadvantaged areas (Chase, Church Lane, Edmondson Heights, Halstead, Hawthorne, and Lansdowne).

Visit schools’ websites to view report cards:

http://www.bcps.org ~ Our Schools ~ School Directory ~ Elementary Schools ~ select school ~ gray box on right has MSDE Report Card for 2016

For example:

Fort Garrison ELA 3 ~ Meeting: 56.8; Exceeding: 25

Fort Garrison ELA 4 ~ Meeting: 36.8; Exceeding: 35.3

Fort Garrison ELA 5 ~ Meeting: 52.2; Exceeding: less than or equal to 5.0

Edmondson Heights ELA 3 ~ Meeting: 6.9; Exceeding: less than or equal to 5.0

Edmondson Heights ELA 4 ~ Meeting: 16.3; Exceeding: less than or equal to 5.0

Edmondson Heights ELA 5 ~ Meeting 9.9; Exceeding: less than or equal to 5.0

Johns Hopkins University Report Highlights:

“An examination of MAP scores in Lighthouse and non-Lighthouse Grades 1 -3 showed some impact on student achievement.  Lighthouse students in Grades 1-2 exhibited improvements in reading and mathematics scores across all three years of implementation and Grade 3 increased reading and mathematics scores in all but the present year.  Further, all grades exceeded the national average mathematics and reading scores. Non-Lighthouse Grades 1-3 also exhibited improvements in reading scores across all three years and, similarly, Grades 1-2 increased mathematics scores. Grades 1-3 exceeded the national.”

“Principals and S.T.A.T. teachers perceived that enhanced teaching practices and stronger curricula were increasing mastery of CCSS.  However, they were generally hesitant to attribute the MAP gains directly or solely to S.T.A.T.  We agree with this assessment for several reasons.  First, gains in achievement were not projected by the Logic Model this early in the implementation, although we cannot rule out more rapidly occurring impacts.  Second, there are numerous programs and initiatives in BCPS, which could contribute to improved student achievement independently of S.T.A.T.”

Lighthouse (LH, pilot) Grade 3 MAP (RIT) Reading Scores:

Pre-program (2013-14): 188.52

Year 1 (2014-15): 194.14

Year 2 (2015-16): 198.37

Year 3 (2016-17): 197.49

Non-LH Grade 3 MAP (RIT) Reading Scores:

Pre-program (2014-15): 194.30

Year 1 (2015-16): 196.30

Year 2 (2016-17): 196.57

STAT-us BCPS NOTE:  While MAP gains are indicated in the report, a close reading shows them to be marginal.  The RIT (Rasch Unit) score reflects a student’s academic knowledge, skills, and abilities.  RIT scores range from 100 to 350.  Additionally, MAP (a growth measure) and PARCC (a proficiency measure) are very different.  As noted on Page 134 of the BCPS FY18 Operating Budget, in FY2016, only 50.2% of third-graders were reading on grade level. Minimal growth in RIT scores is not closing major achievement gaps.

Issues:

~ Off-task device use. “Teachers at all levels described the challenge of monitoring and managing device use during instructional hours, and their comments reflected those when asked to describe off-task/inappropriate use above.”

~ Technical issues. “Some of the technical issues expressed by middle and high school teachers centered on students’ lack of accountability with devices, such as returning to school with a depleted battery, forgetting the device at home, or breaking the devices.  Other technical issues mentioned by teachers at all grade level included slow Internet or BCPSOne not functioning.”

~ Lack of support. “Teachers at all levels conveyed feeling overwhelmed and not supported with technology integration.  Some teachers described not having enough time for planning, as noted by a Lighthouse middle school teacher: “TIME!! More planning time is definitely needed!!!”  Others mentioned the challenge of attempting to learn new approaches to instruction along with other initiatives.  A Lighthouse elementary teacher described the struggle of “incorporating the new grading system at the same time as technology,” while another noted, “My greatest challenge is just not taking on too much at one time.  Learning each new innovative ‘thing’ at a time rather than trying to do it all at once.”  Others echoed this sentiment, as a Lighthouse middle school teacher described the challenge of “deciding which resources to use and which to pass on.  There were plenty of resources available but it felt as though I was supposed to utilize as many as I could rather than focusing on/mastering a few. I eventually minimized the resources I utilized.”

Of great concern was JHU researcher Dr. Morrison’s statement, made during her presentation to the BOE, that the initiative was overwhelmingly supported by the community.  This was based on the 2017 Stakeholder Survey, which offered three vague and leading questions regarding personalized learning and technology.  These questions would mean little to community members altogether unaware of STAT and high-school students and parents not even connected to STAT (in 2016-17, the initiative was only in place in four Lighthouse (pilot) high schools).

Personalized Learning:  Parents, school-based staff, and central office staff expressed the most agreement that making learning personalized for students helps teachers meet the academic needs of all students.

Access to Technology:  Agreement was high across students, parents, school-based staff, and central office staff that access to technology increases opportunities for making learning more personalized for students.

Teacher Use of Technology:  Students, parents, school administrators, and central office staff had similarly high levels of agreement that teachers can use technology to meet the academic needs of all students.

Advertisement