EDUCATIONSECTOR REPORTS

October 2010

PUTTING DATA INTO PRACTICE:

Lessons From New York City
By Bill Tucker
ACKNOWLEDGEMENTS
Thanks to my Education Sector colleagues, in particular
Elena Silva and Kelly Bathgate, for their help in thinking
about issues related to data and learning, and to Kevin
Carey and Robin Smiles for their support in the writing
and editing of this paper. Susan Headden deserves
special thanks for her careful editing and thoughtful
feedback. Catherine Cullen provided invaluable help with
the initial research and ideas contained in this report. My
sincere appreciation also goes to the many people who
were kind enough to read and comment on an earlier draft
of this paper, including Ben Boer, Vincent Cho, Beverly
Donohue, Eric Osberg, Baron Rodriguez, Jeffrey Wayman,
and Emily Weiss. Finally, thank you to the dozens of
educators, researchers, policy analysts, and experts who
graciously offered their insights and knowledge to me
throughout the research and writing of this report.

This report was funded by the Charles Stewart Mott

Foundation. Education Sector thanks the foundation for
their support. The views expressed in the paper are those
of the author alone.

ABOUT THE AUTHOR

BILL TUCKER is managing director at Education Sector. He can
be reached at btucker@educationsector.org.

ABOUT EDUCATION SECTOR

Education Sector is an independent think tank that challenges
conventional thinking in education policy. We are a nonprofit,
nonpartisan organization committed to achieving measurable
impact in education, both by improving existing reform initiatives
and by developing new, innovative solutions to our nation’s most
pressing education problems.

© Copyright 2010 Education Sector

Education Sector encourages the free use, reproduction, and distribution
of our ideas, perspectives, and analyses. Our Creative Commons
licensing allows for the noncommercial use of all Education Sector
authored or commissioned materials. We require attribution for all use.
For more information and instructions on the commercial use of our
materials, please visit our website, www.educationsector.org.
1201 Connecticut Ave., N.W., Suite 850, Washington, D.C. 20036
ÓäÓ°xxÓ°Ón{äÊUÊwww.educationsector.org
Stephanie Ring is an educator who loves numbers. And it’s not just
because she’s a high school math teacher. She also serves as her
school’s data specialist, a savvy analyst who monitors things like
course credits, test scores, and attendance records to make sure
students are on track to graduate and getting enough attention from
teachers who instruct as many as 150 students a day. For Ring, who
teaches at Brooklyn, N.Y.’s High School of Telecommunications Arts
and Technology, data is valuable only when she can act on it—not
when a student is a senior and hopelessly far behind, but when he’s
a freshman and just starting to fall short of the credits he’ll need to
graduate on time. Data in hand, Ring and her colleagues can then
UHÀQHLQVWUXFWLRQDOSUDFWLFHVDQGSUHVFULEHLQWHUYHQWLRQVDGGUHVVLQJ
the problem soon after it starts. For Ring, data drives action. “What
are we doing for these kids?” it prompts her to ask. “Did we talk to
them? Did we move them?”

U.S. Secretary of Education Arne Duncan calls data purposes of accountability, a 2009 report from the
“the driving force [behind education] reform.”1 With U.S. Department of Education found that “even in
stronger data, the reasoning goes, policymakers districts with a reputation for leadership in using
can expose problems, identify effective teachers, data, electronic data systems are barely influencing
more smartly allocate resources, and build political classroom-level decision-making.”4 Many systems
will for reform. In the classroom, better data can aren’t designed to provide data to teachers, let alone
help educators identify learning gaps and behavior students. All too often, the preoccupation with data
patterns so they can tailor instruction to individual collection has overshadowed the ways in which data
students. is—or isn’t—used.5

Such is the promise of data that, nationwide over the Progress, however, is being made, and few districts
past decade, school districts and states have spent have embraced the use of data like New York City,
more than $1 billion to build and implement data the nation’s largest district with 1.1 million students
systems to track and analyze statistical information and 90,000 educators. Data is a vital component of
about students and their learning.2 Data systems were New York City’s aggressive strategy to hold educators
a key criterion of the Obama administration’s Race to accountable for student performance and to make
the Top competition, and by 2011, all 50 states will sure they have all the tools and support they need
have systems to track individual student progress to succeed.6 The district’s “inquiry teams,” groups
from year to year.3 Many school districts have built of teachers who collaborate to help students based
data systems as well. on shared information, rely heavily on data produced
by an $80 million information storehouse called the
But while there have been impressive advances in Achievement Reporting and Innovation System, or
the collection and management of data used for

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 1

ARIS, a repository of statistical information about school system—and how it can be employed in
students. smaller districts, as well. Seen from the perspective
of both its obstacles and successes, New York’s
With a couple of mouse clicks, classroom teachers experience holds valuable lessons for all school
can now get such data as interim test scores, subject districts about how to succeed with the critical
grades, attendance records, and English language second component of the drive for data—using the
learner status on a single computer screen. Thanks information to improve student performance.
to ARIS, a high school instructor who may have a
student for just one period a day can now see how
that student is progressing across all courses, and The Promise and Perils of Data
can identify students at risk of academic failure.
Teachers are now also able to spot long-term learning Data is used to inform decisions in almost every field
trends, even for students who have moved often of endeavor, from health care to sports, from criminal
among schools and who have only just arrived in justice to finance. Film studios use data to decide
their class. After initial resistance, ARIS has won the what movies to produce.8 Police departments use it to
predict and prevent crime. And credit card companies
instantly compare transactions with historical patterns
Above all, the district has to spot potential fraud.

realized that building a data In health care, providers are beginning to use data
from electronic records to reduce errors, cut costs,
system is only the first step— and improve patient outcomes. Effective use of
what educators do with the data electronic health records is one reason why the
Veterans Health Administration now surpasses other
is the critical second. health systems on standardized quality measures
even though its patients are for the most part older,
sicker, and poorer. Not only do these records allow
cautious support of the local principals union, and for coordinated care by multiple physicians, they also
more than 65 percent of the district’s teachers now enable more rapid identification of risk factors, making
participate in inquiry teams.7 it possible, for instance, to detect kidney disease in
veterans often before symptoms emerge.9
But ARIS has been fraught with problems, as well.
Developers have confronted a tangle of antiquated Likewise, the best teachers have always used
systems that can’t talk to each other—information information about their students to help them improve
silos that prevented any one person from getting a instruction—and they know that more and better
complete picture of a student. And they continue to information can lead to even better results. Yet,
struggle with making the data timely and accurate unlike for almost all other professionals who perform
and giving educators the time and training they need complex, demanding work, the information tools
to use it well. In the process, they have learned that available to teachers have been remarkably limited.
technology holds little value unless it is flexible, Most teachers still work isolated in their classrooms,
relevant, and provides the fine-grained information with only their own eyes and rudimentary assessment
that teachers really need. Above all, the district has tools to guide them. For the most part, they aren’t
realized that building a data system is only the first benefiting from sophisticated information-gathering
step—what educators do with the data is the critical tools, from their colleagues’ knowledge, or from
second. Building the conditions and demand for data- analyses of thousands of similar situations—the very
based analysis is often more difficult than collecting kind of information that physicians, police officers,
the data itself. and even sports executives use on a daily basis.
Concludes a recent article from SRI International’s
While New York’s size and the scope of its initiative Center for Technology and Learning: “Teachers do
make it unique, the district provides a rich and timely not have the data-rich, performance-support, and
illustration of how data is being used in an urban information-feedback work environment that virtually

2 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

all other high-performance professionals … have at entire institutions. But physicians require different
their disposal.”10 types of information, such as measurements of vital
signs and results of blood tests, to diagnose and
That is not to say that educational data doesn’t treat individual patients.17 Likewise, in education,
exist. On the contrary, it has been centrally collected state and district officials want data that shows broad
since at least 1867 when the first federal Education trends so they can assess a school’s or a district’s
Department was charged with “collecting such overall effectiveness. (This is accountability data.)
statistics and facts as shall show the condition and Teachers want additional information, such as results
progress of education in the several States and from classroom assessments that may track weekly
Territories, and of diffusing such information.”11 But progress.
until recently, data almost always flowed one way:
up. It went from school to district, from district to Health reformers have also demonstrated that
state, and from state to the federal government. It electronic data systems will not improve performance
was retrospective in nature and designed almost on their own. Although they are essential, improved
exclusively to show compliance with state and federal technology and better data are just the infrastructure
regulations. Because the information was reported in for more substantive changes in the daily practices
aggregate, state agencies could provide policymakers of providers. For example, officials at Central
with descriptive snapshots of student populations, but Pennsylvania’s Geisinger Health System, a model in
they couldn’t give them data about individual students its efforts to improve quality and control costs, quickly
and how those students changed over time. realized that the ability to share data across a variety
of systems and contexts was not enough to improve
The 2001 federal No Child Left Behind Act, which
imposed new reporting requirements on states,
sharpened the focus on data. The law required states
to collect test data for individual students each year, The best teachers have always
meaning states could no longer rely on aggregated used information about their
achievement information reported by local districts.
(Student-level data can now be sorted by school, students to help them improve
race, and a variety of other subgroups). At the same
time, researchers and policymakers urged states to
instruction.
develop longitudinal data systems that would allow
them to track individual students over time, across care. What was needed was a cultural change among
schools, through college, and into the work force. In its employees—a shift from working in isolation on
2002, only 15 states had a longitudinal data system.12 single tasks to working together on tasks that are
By 2008, 48 states did.13 aligned.18
But these systems often fail to give educators
the information they need. While the amount of
educational data collected continues to grow—Texas New York City’s Data Strategy
school districts alone respond to 104 data collections Individual teachers can benefit from access to
by the Texas Education Agency each year—the quality analytical tools like ARIS. But New York City’s
and utility of much of it remains questionable.14 Many goal—to build evidence-based school cultures—is
systems have become de facto data morgues,15 used even more ambitious and requires both the tools
more often to perform autopsies of failed programs and collaborative processes to use the information
than to help educators and policymakers improve to improve student performance. Shael Suransky,
existing ones.16 deputy chancellor for the Division of Performance and
Accountability, identifies the two key components of
Other fields have tackled similar problems. Hospitals,
the district’s strategy: regular collection and analysis
for example, collect data on patient outcomes and
of assessment data—much of which can be found in
mortality rates so policymakers, administrators, and
ARIS—and the use of teacher inquiry teams.19
consumers can use it to make judgments about

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 3

Figure 1. The Collaborative Inquiry Process

Setup 1 2 3 4
School self-assessment Principal’s ownership Teacher team formation: Structured support for
and goal-setting and launch vast majority of teachers on teams teacher team facilitators
with regular meeting times

Examine student Examine teacher

work/data work (including
classroom visits) Reflect on teacher teams’
results and consider for Analyze school
schoolwide change (teachers capacity and plan for
Revise and repeat assume leadership role) schoolwide change
inquiry cycle
Instructional Engage external
resources Schoolwide
inquiry cycle inquiry cycle
(multiple teams)
Monitor student
progress with Define instructional
common assessments strategy and set goals

Take action to
Take action: implement
build teacher and
instructional strategy
school capacity

Share and celebrate work

Source: NYC Department of Education.

The inquiry teams focus intensively on small groups of children’s progress.21 Teachers can also use the data
students. A team of teachers and administrators looks to customize progress reports. Educators looking for
at schoolwide data to identify a focus group—say, patterns of performance are no longer limited to what
sixth-graders with scores of two or less on last year’s they see in their own classrooms. They can easily
state English Language Arts test, which is graded on group and track students receiving specific help or
a scale of one to four. The team then studies both the identify students with a particular learning challenge.
students’ work and the data, and reviews instructional
approaches: “How are these students being taught?” New Dorp High School in Staten Island provides a
they ask each other. “How are they being grouped?” promising example of how all these components
After answering these questions, the team develops come together. Teachers at New Dorp formed an
a theory about why the students are not succeeding inquiry team to determine why student writing
and begins to implement changes.20 Periodic or was deficient across the board. In another era, the
interim assessments, such as those from Columbia teachers might have assigned some extra work
University’s Teachers College Reading and Writing and hoped for the best. But these teachers had the
Project, which measure recognition of letters and their tools to take a more analytic approach. The teachers
corresponding sounds, are meant to help teachers reviewed data from ARIS, such as scores on state
gauge progress along the way. (See Figure 1.) Regents tests, along with classroom-based student
assessments, essays, journals, and report cards.
ARIS provides these teams of educators with a They confirmed that students were struggling in all
common set of data, including state and interim areas. But the insight was too general to lead to
assessment results, attendance records, and course specific actions. “Writing is too big to fix everything,”
grades. (See Figure 2.) The teams use this information says assistant principal Dina Zoleo.22 So the team
to collaborate on instructional interventions, not just drilled further down, identifying a small group of the
for single students but for whole groups of students. lowest-performing freshmen who routinely used poor
The data helps teachers identify students’ strengths grammar and could not differentiate among words like
and learning gaps and inform parents of their their, there, and they’re.

4 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

Figure 2. How the ARIS system works

ARIS home page:

Student detail screen:

Sample report:

Source: NYC Department of Education.

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 5

Significantly, the New Dorp team included teachers The technical challenges proved much greater than
from all subject areas. They observed each other’s anticipated. None of the underlying data on students
classes to see how, and whether, critical writing resides in a single database: Attendance data lives
skills were being taught. And what they found was in one system, grades in another, and test scores in
illuminating: They discovered that grammar was others. Replacing all of these systems would have
rarely addressed across the curriculum, and that been almost impossible and certainly unaffordable.
even in English classes, learning literary terms, like So, rather than collect all this information on its
the difference between a metaphor and a simile, own, ARIS was developed to “sit on top” of all these
occupied more instructional time than lessons in such systems, requiring just a single user name and
matters as syntax, punctuation, and verb agreement. password for access. But the systems used by New
The school responded by injecting grammar across York—and many other districts—were not designed
the curriculum, including it as part of the grade on for this purpose, and many come from competing
assignments such as history essays. The students vendors. Each required costly software upgrades to
examined by the inquiry team were canaries in the allow ARIS to understand its data.
school’s coal mine. “The theory is that if you can
move those students,” Zoleo says, “then we can A series of rapid fixes followed. Wireless Generation,
move groups of students overwhelmingly.”23 an education technology company that served as
a subcontractor to IBM on the project, took over
the development of ARIS, and an improved version
A Rocky Start was launched in November 2008. A site that allows
parents to track their child’s progress was launched
As encouraging as its efforts have been, New Dorp is the following May.27 Soon critics were changing their
the exception when it comes to using data to improve tune. In August 2009, a poll by the New York City
instruction. Elsewhere in the district, getting this sort Office of the Public Advocate reported that most of
of information into the right hands, let alone getting the principals surveyed supported ARIS, even if a
teachers to use it to improve learning, has been large majority still challenged the costs.28 Antoinette
anything but easy. Isable, a representative for the principals union, the
Council of School Supervisors and Administrators,
From the beginning, ARIS has fallen short of the told the New York Times: “While we were skeptical in
grandiose promises made about it—that it would the beginning, we have come to understand much of
transform instruction, that it would provide all of the [ARIS’s] long-term value and importance.”29
information teachers need, that it would allow parents
to get involved in their children’s educations as never At first, the ARIS training program focused on
before. And it has been bedeviled by countless teaching the teachers—principals included—how
delays.24 Designers had several systems to integrate, to use the tools. There was less emphasis on how
and whatever errors these systems held made their and why to apply them. During the 2008–09 school
way into ARIS. The district also had to replace its year, each school selected at least two educators
initial vendor, IBM, with another contractor. The to be trained. The district also offered instruction
delays served to only further inflame existing political for principals so they could build awareness and
opposition to the accountability strategy of schools support for the tool. But officials soon learned that it
Chancellor Joel Klein. Citing budget cuts and layoffs, wasn’t enough just to supply the data and schedule
many teachers and parents felt the money could have the training sessions. Just as important, says Emily
been better spent elsewhere. Weiss, chief of staff for the Division of Performance
and Accountability, the district also had to “build the
In ARIS’s first year and a half, the results were demand for data.” And to do that, they had to make it
decidedly underwhelming. A February 2008 article in relevant.
the New York Post called the project a “disaster” and
noted that almost half the budget had already been There are two important prerequisites to building
spent.25 And in October 2008, the New York Times demand for data, Weiss says. First, educators must
reported that the system was still largely inoperable.26 understand what the data is and what it means—they
must become “data and assessment literate”—before

6 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

they can understand how to use it for instruction. January, it may not load onto ARIS until February. So
Second, the operational structure of the school must teachers have to go into separate databases to get
accommodate teacher collaboration based on data. the latest scores.33
That means teachers must be encouraged to share
data and talk about what they think it means. “We Teachers also complain that ARIS is not as detailed as
need to move to conversations among educators,” they would like it be. Properly designed assessments,
says Weiss, “not just individual data analysis.”30 for instance, can provide teachers with remarkably
specific information about student performance; they
That sort of collaboration—“shared accountability can show how well a student seemed to grasp a math
across multiple educators for the same students,” problem, for instance, or how fluently he was able
Weiss calls it—represents a significant change in the to read certain letter pairings. Recently, the system
closed-door culture of many schools. Historically, was updated to allow educators to see not only
notes Suransky, teachers haven’t felt comfortable overall student assessment scores, but also individual
discussing student performance—regardless of student performance on specific components of the
whether those students were failing or succeeding. state assessments.
Yet, Suransky says, instruction dramatically improves
when student learning is openly discussed. This kind But teachers, notes Carmina D’Angelo-Schiavi of
of atmosphere also encourages teachers to leverage P.S. 85 in Queens, “want to be able to see the whole
their individual strengths and leadership skills.31 “I child.” They want to understand their students in the
can’t over-emphasize the inquiry teams,” says Klein. context of their demographic makeup, their aptitudes
“Pointy-headed people in the central office can’t in various subjects, and their prior learning history.
figure this all out … we want the teachers to come And they want to be able to get all that information
together.”32 Accordingly, the city’s Department of in one place. For instance, along with things like
Education has shifted the emphasis of its data training standardized test scores, D’Angelo-Schiavi would
from the technical—how to log in to ARIS, how to like simultaneous access to measures like Schools
generate reports—to the practical, using data as a Attuned, an approach she uses to assess the learning
component of applied professional development. strengths of her second-graders.34
Significantly, Weiss says, the training now starts from Like the Schools Attuned material that D’Angelo-
a specific question that teachers want answered. Schiavi finds so helpful, the most relevant data
is that which is closely aligned with the teacher’s
curriculum—quizzes, homework assignments,
Information That Moves feedback on writing, and the like. But, because it
Each school has a different perspective on which varies so widely across classrooms, this micro-level
data is important. And there is a torrent of potential data is usually not included in the accountability
information to be gathered and analyzed at the programs that states use to compare school
classroom level, from homework scores to behavioral performance. This is despite the fact that micro-
incidents, which every school collects in its own level data is critical to improving day-to-day student
way. As for ARIS data, teachers say that it can performance, says Beverly Donohue, vice president of
become quickly outdated, preventing them from policy and research at New Visions for Public Schools,
acting in time to help students who are struggling. a school support organization responsible for working
Teachers tell Anne LaTarte, director of instructional with 76 of the district’s public schools. The issue,
and data tools for the city’s Education Department, Donohue says, “is not accountability versus micro-
that although ARIS gives them good information, “it level student data.”35 Both types of data are valuable,
doesn’t move frequently enough.” Since much of the but serve different purposes.
data in ARIS is generated at the end of the academic Suransky agrees, saying that the biggest drawback to
year, it becomes less relevant as the following year ARIS is its inability to provide data on these frequent,
goes on. And even though ARIS contains data from classroom-based assessments. The city’s Education
periodic assessments, the information doesn’t flow Department is responding with ARIS Local, an
instantaneously across databases. If a test is given in initiative that allows individual schools and teachers

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 7

 Learning From ‘Data Exhaust’
Type the misspelled word “educaton” into Google. The even contemplate the wealth of data made available
search engine instantly anticipates that you really meant by technology-enabled assessment and instruction.
to search for “education” and redirects you accordingly. Benjamin Boer, educational data expert and director
In fact, it is almost entirely as a by-product of this data of policy for Advance Illinois, an education advocacy
created by billions of mistyped searches that Google has organization, cautions that “the data systems needed
created what may be the world’s best spell-checker.1 are transformatively different than the data systems we
have.”5
As computer power has increased, the bits of data
generated by ordinary online activities—from buying Still, harvesting unexpected data from everyday activity—
clothes to paying subway fares—have become what Wireless Generation CEO Larry Berger likens to the
increasingly valuable and useful. Smart organizations agricultural practice of “drip irrigation”—may help show
are using this behavioral data—known as “data exhaust” how very granular information can inform instruction.
because it is often a by-product of other transactions—to Tools such as online grade books, says Beverly Donohue
test and refine their products and services. with New Visions for Public Schools, “can help teachers
do some element of their work better and faster and
Electronic health data—such as the information the U.S.
generate data as a by-product.”6 Any data already being
Veterans Health Administration has from 8 million patients
collected in a digital form can be analyzed in conjunction
—allows researchers to augment formal clinical trials by
with other data to generate new insights. And from all
generating and testing hypotheses around treatments.2
this data—and careful analysis—unexpected tools, like
These stores of data create natural experiments, says
Google’s spell-checker, may also emerge.
Chris Anderson, Wired Magazine’s editor-in-chief, who
speculates that they could someday make theoretical
models obsolete: “This is a world where massive amounts Notes
of data and applied mathematics replace every other tool 1. “Special Report on Managing Information,” The Economist,
that might be brought to bear.”3 February 27, 2010.

In education, the data that students leave behind—when 2. Lynn M. Etheredge, “A Rapid Learning Health System” Health
Affairs 26, no. 2 (2007).
engaging with an online math tutorial, for instance—could
generate useful data exhaust.4 Just as Google constantly 3. Chris Anderson, “The End of Theory: The Data Deluge Makes
improves its spell-checker, the math tutorial could use the Scientific Method Obsolete, ” Wired Magazine, Issue
16.07, June 2008, http://www.wired.com/print/science/
real-time data—such as how long it took students to
discoveries/magazine/16-07/pb_theory
respond to tasks, what actions enabled students to
persist through a set of questions, when did students 4. For a discussion of how technology-enabled assessments
can generate this type of data, please see Bill Tucker,
ask for help, what sorts of errors did students make—to Beyond the Bubble: Technology and the Future of Student
learn exactly which problem sets, tutorials, and processes Assessment (Washington, DC: Education Sector, February
led to better outcomes. Teachers, in turn, could provide 2009) http://www.educationsector.org/publications/beyond-
more tailored instruction, leading to improved curriculum bubble-technology-and-future-student-assessment
and instructional approaches. They could create self- 5. For more discussion of and possible solutions to these
correcting feedback loops—the educational equivalent of challenges, see Benjamin Boer, Assessment and Technology:
Google’s spell-checker. An Opportunity to Improve Data Design to Inform
Instruction (Chicago, IL: Advance Illinois, 2010) http://www.
Instructional programs, such as those that use response advanceillinois.org/filebin/pdf/Reference_Assessment.pdf
clickers, remote devices that capture immediate student 6. Beverly Donohue, e-mail message to author, September 9,
feedback, are beginning to use exhaust data in this 2010.
way. But most state and district systems aren’t built to

to add their own data points, such as classroom between scores on classroom history tests and scores
test scores, to ARIS so they can compare it to other on state tests in history. The pilot of ARIS Local will
demographic and standardized test data. This also incorporate assessment data from open-ended,
specific data won’t necessarily be comparable across essay-type assignments.
schools, but the collection and reporting of it will be
in a consistent format. LaTarte would also like to see The frequent, front-line use of data by educators is
data displayed in a way that would allow teachers critical not only to improving student learning, it’s
to make comparisons they couldn’t before, such as also essential to preventing and correcting the errors
that plague so many data initiatives. In traditional

8 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

compliance-oriented systems, in which data is going to school, and 75 percent of teachers thought it
collected, locked in a warehouse, and spit out without could transform the way that instruction in New York
review, errors can easily creep in. By contrast, if is tailored to individual students.37 (See Learning From
the people closest to the data—students, parents, ‘Data Exhaust’ on page 8.)
and teachers—are actually using the data, they will
consistently update it and improve its accuracy,
essentially creating a continuous process of checks Beyond Schools
and balances.
Increasingly, data is not just a tool for principals and
One barrier to getting the detailed information that teachers. It can empower students and families, as
teachers want is purely practical: the time and effort well. While Stephanie Ring intervenes with students
it takes to collect all of the information. Although who have drifted off course, her school also engages
new digital instructional tools, such as interactive
math programs or simulated biology dissections, The frequent, front-line use
have the potential to make data entry automatic,
many teachers still have to enter data tediously by of data by educators is critical
hand. Developers of classroom tools have noticed.
Larry Berger, CEO of Wireless Generation, considers
not only to improving student
saving teachers time to be a crucial element of any learning, it’s also essential to
successful data product. To be effective, he says, a
tool “needs to give teachers back more time in one preventing and correcting
month than it takes to train [them] to use it.” For
example, digital tools can capture important data on a
the errors that plague so many
student’s use of grammar, providing instant feedback data initiatives.
that gives the student more time to practice writing
and the teacher more time for thoughtful feedback.
Once educators overcome the challenge of digitizing students and their families to chart a new path on their
information, Berger says, they often embrace the data own. Using a personalized College Readiness Tracker,
tools. a sort of GPS for on-time graduation developed by
New Visions for Public Schools, these students can
New York’s School of One, a pilot sixth-grade math see exactly where they are—what classes and state
program that calibrates instruction to a students’ Regents courses they have passed—and where they
progress, offers one glimpse into this digital future. need to go. (See Figure 3.) This sort of predictive
The school’s ambitious goal is to create an adaptable, data is powerful because it allows students, and their
minute-by-minute learning experience, challenging families, to understand how their current actions
students just enough to keep them engaged and affect their future goals. It’s also continuous and
moving at the right pace. Each night, based on the immediate, as with a GPS system that constantly
results of that day’s lessons and diagnostic tests, a monitors progress, anticipates traffic patterns based
computer algorithm automatically creates a detailed on historical data, and alerts a driver instantly after
lesson plan for the next day. If students fall short in a missed turn—not 50 miles later. With the help of
grasping a certain concept, for example, the algorithm these tools, family engagement changes from a one-
will devote more time to that goal. If students learn time, end-of-year event to an ongoing, substantive
better using some methods as opposed to others, it conversation.38
adjusts accordingly. The lesson plan is e-mailed to
teachers, who revise it as necessary.36 Early results As do teachers, parents need to be convinced that
are promising: An evaluation of the 2009 pilot by the time with data is time well spent—that they will gain
Education Development Center’s Center for Children enough to be able to really help their children. ARIS’s
and Technology found that students gained 28 Parent Link, which provides parents with information
percentage points from pre-test to post-test. Perhaps about their children’s achievement, offers instructions
more promising, 79 percent of students said they liked in nine languages. (See Figure 4.) But, again, just

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 9

providing access is not enough; educators have to system that requires students to swipe magnetic
encourage parents to use it. At P.S. 85, Principal cards when they enter, for example, a Boys and
Ann Gordon-Chang makes an extra effort to ensure Girls Club. The school system can then add that
that parents know about and understand how to information to its own data warehouse and provide
use the portal. She staffs the school’s computer lab data about student achievement to after-school
with multilingual teachers outside of school hours educators.42
for special parent workshops. She provides food
and opens the school on Saturdays. Already, says Just seven miles from the Department of Education
Gordon-Chang, parents appreciate that the data gives headquarters, data plays a prominent role at the
them a more complete and objective picture of their Harlem Children’s Zone (HCZ), the model for the
children than do teacher perceptions alone.39 Obama administration’s Promise Neighborhoods
initiative. The organization’s goal is to ensure the
Parent Link employed more strategies in a pilot effort academic success of 15,000 Harlem children by
to boost use by low-income families in 24 schools. putting them in “an enriching environment of college-
Successful strategies include using parent and oriented peers and supportive adult staff.” It touts a
student volunteers; in one school, students train their “comprehensive, data-driven approach.”43 Indeed,
parents and have them sign notes confirming they HCZ chief operating officer George Khaldun tells
have logged in to Parent Link. Teachers have been his staff that “if it’s not in the database, it didn’t
trained in how to talk about data with parents. Another happen.”44 But HCZ staff doesn’t have access
school opened its library early for ARIS workshops, to ARIS, with its wealth of information about the
and at another, a parent coordinator e-mailed parents very youth the organization is trying to help. Nor
who had not logged in.40 One school, located across do public school educators have access to HCZ
from a homeless shelter, even created a resource
room with a washer, dryer, and Internet access.41 Figure 3. College Readiness Tracker
Parents, moreover, are not the only adults who work
with children outside of school. Other counselors and
mentors need access to information from schools,
and vice versa. By analyzing student achievement
data along with information about community
programs (such as the influence of a mentoring
program on school attendance) experts can better
evaluate the effectiveness of those services and
target them appropriately. Sharing information
across school-home-community boundaries means
establishing positive, trusting relationships—
relationships that don’t always exist. It also requires
overcoming privacy barriers, both real and imagined.

The federal Family Educational Rights and Privacy

Act prevents schools and teachers from sharing
personally identifiable information about students with
anyone outside the institution except parents. Narrow
interpretations of the law have led many schools to
restrict any information sharing at all. But parents
can consent to release information, and several
communities have developed ways to integrate
data across a variety of programs. For example, in
Jefferson County (Louisville), Ky., school officials get
up-front parental consent for students participating
in after-school programs through KidTrax, a tracking Source: New Visions for Public Schools.

10 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

Figure 4. Parent Link Website

Source: NYC Department of Education.

data on programs, interventions, and other student An Iterative Approach

information.45
While it’s still too early to judge the ultimate impact
Sophie Lippincott, former director of knowledge of New York’s data initiatives, there are positive
sharing in the Division of Performance and signs that they are taking hold among educators and
Accountability, sees the clear value of sharing ARIS parents. When inquiry teams were first introduced
information with community-based organizations, and by the city’s Department of Education, they were a
she has been trying to begin a program to do so. “It’s highly structured add-on to existing work; schools
obviously in our favor to have partner organizations assembled a special team, gave members additional
using ARIS,” she says. The district has trained two compensation, and devoted new resources to the
organizations that are “gung-ho . . . and ready to enterprise. But, increasingly, in schools like New
go,” she says. But, here again, organizational silos Dorp, using data is embedded into existing teaching
are proving difficult to break. The developers of ARIS teams. Sixty-five percent of New York City educators
did not contemplate out-of-school use; because are now participating in collaborative inquiry, up
user authentication is based on the Education from 10 percent in the program’s first year. School
Department’s human resources databases, it is quality reviews, in which experienced educators
difficult for non-school employees to gain access.46 conduct evaluations of school practices, also report
(The district has recently developed a temporary significant growth in data use among city schools.47
solution that enables schools to grant access to On an April 2010 survey, 77 percent of principals
certain community partners.) (See Using Data Across rated ARIS as “helpful” or “very helpful” in improving
Children’s Lives, on page 12.) student outcomes, up from 47 percent two years

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 11

 Using Data Across Children’s Lives
Students in New York City schools interact with a number parents to improve attendance, for example, and
of public agencies and participate in a variety of out- checking up on homework assignments if they see
of-school activities. All are likely to influence academic students going off course. They can also recognize and
achievement. Yet educational data systems are mostly praise student success.2
blind to students’ lives outside the classroom. At the
Ɣ In California, the Youth Data Archive, run by the John
same time, public agencies and community groups know
W. Gardner Center for Youth and Their Communities
little about the academic situations of the youth they
at Stanford University, uses data from schools and
serve.
community organizations to research questions,
As the school district is doing with ARIS, the city is such as the association between Boys and Girls Club
working to consolidate disparate bits of data to get a attendance and the development of proficiency among
more complete view of the citizens it serves. The city English language learners.3
wants to coordinate services across nine city agencies,
Ɣ Nationwide, Intelligence for Social Policy (ISP)
including those dealing with public health, homelessness,
promotes the development and use of integrated data
and juvenile justice. On average, a single family is involved
systems. Now working with nine jurisdictions, ISP grew
with five different agencies, says Linda Gibbs, deputy
out of Philadelphia’s Kids Integrated Data System,
mayor for Health and Human Services, “but they didn’t
which houses data that helped shape the city’s dropout
know about each others’ presence in the household.”1
prevention initiatives. Dennis Culhane, one of ISP’s two
Social services data is not yet integrated into ARIS. Nor is
principal investigators, says that four of the ISP sites
ARIS data available to the social workers who use a new
include some educational data and that “all wish they
system known as HHS-Connect.
did.” Integrated data, he says, is especially important
Because of these gaps, educators and community leaders across developmental transitions. For instance, during
lack data to help them understand how these systems early childhood, he says, the “baton gets dropped all
interact and to help them make decisions and coordinate the time.”4
their work. It means they can’t take advantage of powerful
Still, these examples are nascent. And, without careful
tools to detect patterns or risk factors across interventions
attention to the design of data initiatives, the potential to
—patterns that might be impossible to discern from
coordinate actions across the variety of organizations and
school data alone. This cross-agency information is
adults supporting youth learning will go unmet.
particularly important to serving at-risk youth, such as
children in foster care, who are most likely to use multiple
public services. Notes
But in New York and elsewhere, schools and social 1. Zachary Tumin, “New York’s HHS-Connect: IT Crosses
service agencies are slowly improving their capacity to Boundaries in a Shared-Mission World,” posted on Governing.
com, August 24, 2009, http://www.governing.com/blogs/bfc/
share data toward useful ends. Some examples: New-Yorks-HHS-Connect-IT.html
Ɣ In St. Louis, the main objectives of Big Brothers 2. Jesse Bogan, “Big Brothers Big Sisters Breaks New
and Big Sisters of Eastern Missouri are to improve Ground by Monitoring Student Progress” Stltoday.com,
attendance, behavior, and classroom success (“ABC”). August 4, 2010, http://www.stltoday.com/news/local/metro/
But the organization had little information about what article_6c2e5fc1-4686-5f23-ab4a-58b59a991665.html
actually happened in their mentees’ classrooms. So, 3. For more information see: http://gardnercenter.stanford.edu/
after getting privacy waivers from parents, officials current_initiatives/youth_archive.html
secured access to school data about students’ 4. Dennis Culhane, in discussion with author, August 3, 2010;
attendance, tardiness, behavior, and grades. Now For more information on Intelligence for Social Policy see
youth workers and mentors can step in, working with www.isppenn.org

earlier.48 Educator and parent usage of ARIS and Department caution that they lack valid comparisons
its components also continues to rise: A total of to gauge whether these numbers are high or low. And
62,000 unique users logged in to the ARIS educator they are just now rolling out analytical tools to track
tools from July 2009 to March 2010. As of August usage patterns to help them determine how and why
2009, 340,000 different parent accounts had been educators and parents use ARIS.
accessed at least once, most often during parent/
teacher conferences.49 Officials with the Education But to what extent has the drive for data reached
down to the students it intends to help? The evidence

12 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

to date suggests it has far to go. As LaTarte notes, In the future, New York will work to transform ARIS
the ultimate goal of data analysis is not just to identify from a large, centrally controlled system to one that
gaps in knowledge so that instructors can re-teach. provides core data with a flexible platform that others
It is to help teachers understand why a student can build on—“more like an iPhone than a single
didn’t get something in the first place. LaTarte says application,” Suransky says.52
that although teachers are getting steadily better at
analyzing data, data analysis “is not yet leading to One such platform is already under way. This year,
fundamental change in teacher practice or decision- New Visions is launching the Datacation platform
making.” Both “educator capacity and the system across its network of 76 schools. Customized to
itself,” she says, lack the means so far to transform feature an automated version of the New Visions
learning.50 data tools, including the College Readiness Tracker,
the platform will allow high schools to visualize,
An initial Teachers College study on the with charts, graphs, and animation, data extracted
implementation of inquiry teams echoed LaTarte’s
conclusion, noting that while the teams were
successful in using data to identify student needs, they Sixty-five percent of New
were less skilled in developing strategies to address
those needs.51 Knowing where a student stands is York City educators are now
just the beginning; teachers also need things like participating in collaborative
instructional materials and guidance from peers to act
on that information. And, ideally, those resources and inquiry, up from 10 percent in
interventions are connected to the issues identified in
the data so that they are specifically relevant.
the program’s first year.
Likewise, among New York educators, tools that from the same databases that ARIS draws on. And
once seemed powerful are quickly becoming an online grade book will combine teachers’ daily
outdated as teachers seek to solve challenges that data with information about state tests and credit
they didn’t even recognize before. At New Visions, accumulation information, allowing for combined
Susan Fairchild, director of data analysis and applied analysis.53
research, is creating tools to help educators become
what she calls “learning scientists.” She says that the
next generation of early warning indicators to be used Extraordinary Persistence
by Stephanie Ring and her colleagues will combine
multiple types of data into tools that will not only The challenge for designers of data systems is no
identify at-risk students more quickly, but will also do longer entirely technical; new tools for collecting,
so in finer detail. For example, just as the Department storing, sharing, and analyzing massive amounts
of Veterans Affairs has learned to identify risk factors of information are readily available. And while the
for kidney disease even in the absence of visible initial investments in data capabilities, such as the
symptoms, data may reveal heretofore unknown states’ development of unique student and teacher
combinations of indicators that signal future problems identifiers, have been essential, the next phase—
for freshman who otherwise appear on track. generating truly useful data at the student level—will
not happen without a learner-centered approach. And
Reflecting on ARIS’s development, Suransky says that approach must be deeply embedded into the
that one of the biggest lessons has been the need to work flow of, incentives for, and actual day-to-day
design technology tools in an iterative fashion, rather practices of educators.
than all at once. He says he wishes that the school
district had worked more closely with educators in The most successful data initiatives—especially those
the field. He also advises other districts to create seeking to transform complex and costly industries—
prototypes of a number of tools, testing each with require extraordinary persistence and a deep
teachers and principals before taking them into full- commitment of resources over time.54 Data is most
scale development. powerful when it’s used not just to automate systems,

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 13

but to provide a catalyst for transforming how those qualitative information to give a full picture of learning
systems work. Research shows that even in the and performance throughout a student’s academic
most sophisticated companies and organizations, career.
the effective use of data is an iterative process—a
continuous cycle of use and discovery. But the
research also shows that once people start using data 2. Information must flow across
effectively, they become more sophisticated; they institutions
demand data that is increasingly timely, relevant, and
Students are increasingly mobile, and not just
specific.55
across schools, districts, and states. They also span
School districts and state governments have made a number of different learning opportunities both
the difficult decisions to build institutional data during and after the school day. But a teacher can’t
systems, they have committed the financial resources, use information about a transfer student when it’s
and they are working out the technical bugs. Now trapped in another district’s data system. A learner-
comes the hard part—using better information about centered system would operate across institutions;
teaching and learning to boost outcomes for every it would integrate important information from a wide
student. variety of schools, programs, and interventions into
a complete and accurate depiction of a student’s
progress. It would allow all adults who work with
Recommendations: students to better communicate with each other and
to understand more about the students. And it would
Design Principles for Smarter enable schools, districts, states, and the federal
Data Systems government to improve the accuracy, timeliness, and
efficacy of data-gathering. Internet-based platforms
The country is now entering the second phase of a that enable this seamless exchange of information—
costly and concerted push to use data to improve rather than costly projects to integrate systems on
educational outcomes. The challenge is no longer an ad hoc basis—are critical to learner-centered
whether to build institutional data systems, but how approaches.
to use the data that the best of these systems provide
to make a difference in the classroom. Thus, a focus
on actual educator use of data must drive the next 3. Usefulness and usability must
decade of investment. And, considered in this regard,
as an essential ingredient of good teaching, the
drive adoption
nation’s investments in data appear to be seriously To be successful, data initiatives must have an impact
lacking. Five principles should inform these initiatives: on the working relationships, incentives, and day-
to-day practices of educators, without forcing them
into uniform or standardized solutions. And unless
1. Systems must change from systems are designed to be of obvious value to these
being institution-centered to educators—to give them insights into students—then
their use will be limited. Data tools shouldn’t require
learner-centered
extensive training or public relations campaigns to
Designing learner-centered systems means moving
from compliance-focused data, such as that
required by federal programs, toward information Data is most powerful when
that students, families, and teachers can actually
employ to shape day-to-day instruction. Like a GPS
it’s used not just to automate
navigation system, learner-centered systems would systems, but to provide a
track individual student progress and guide the
way toward a learning goal. These systems should catalyst for transforming how
combine hard data—both timely and accurate—with those systems work.

14 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

convince educators of their value; they should be Notes
so intuitive and easy to use that educators will take
advantage of them as a matter of course. Frequent 1. Maya T. Prabhu, “Forum Calls for Better Use of Data in
Education,” eSchoolNews 12, no. 4 (April 2009): 14,
usage also ensures accuracy, leading to better http://www.eschoolnews.com/media/eschoolnews/
decision-making across the board. eSchoolNewsApril09.pdf
2. Author estimate based on federal and state expenditures.

4. Systems must be common, 3. 2009 Data Quality Campaign Annual Progress Report,
November 2009, http://www.dataqualitycampaign.org/files/
yet open DQC_11-19.pdf

Data initiatives in the health field, such as the use of 4. U.S. Department of Education, “Implementing Data-
Informed Decision Making in Schools — Teacher Access,
electronic medical records to improve care, provide Supports, and Use,” 2009, http://www.gesci.org/assets/
the education business with an important lesson: files/Knowledge%20Centre/Implementing%20Data%20
Good governance is essential to ensure that common Informed%20Decision%20Making%20in%20Schools-
Teacher%20Access,%20Supports%20and%20Use.doc
policies, technical standards, privacy protections, and
usage protocols across institutions are enacted and 5. For research on promising use and success factors, please see
Mary Ann Lachat and Stephen Smith, “Practices That Support
monitored. This facilitates use and allows educators
Data Use in Urban High Schools,” Journal of Education for
to easily exchange information. But that does not Students Placed at Risk 10. no. 3 (2005): 333–349; and Jeffrey
mean they should be monolithic. Such systems are C. Wayman and Sam Stringfield, “Technology-Supported
neither effective nor easily adopted. Rather, districts, Involvement of Entire Faculties in Examination of Student Data
for Instructional Improvement,” American Journal of Education
schools, and other programs must be able to tailor 112 (August 2006).
systems for their particular needs. “Smart” phones are
6. For more information on New York’s Children First strategy,
examples of this sort of customization. They adhere please see the New York City Department of Education
to recognized protocols so they can operate across website, 05/14/2010, http://schools.nyc.gov/Accountability/
common networks, but the software accommodates resources/childrenfirst/default.htm (May 14, 2010).
countless unique applications, or “apps,” with which 7. Shael Suransky, in discussion with author, July 9, 2010.
users can expand use of the device. State longitudinal 8. Ian Ayres, Super Crunchers: Why Thinking-by-Numbers Is the
data systems, which now exist separately from district New Way to Be Smart (New York: Bantam, 2007).
systems, could be designed in a similar way; they 9. Joel Kupersmith, Joseph Francis, Eve Kerr, Sarah Krein,
could provide the core data for every district, but let Leonard Pogach, Robert M. Kolodner, and Jonathan B. Perlin,
districts tailor the information or go beyond the core “Advancing Evidence-Based Care for Diabetes: Lessons From
the Veterans Health Administration,” Health Affairs 26, no. 2
as they choose. (2007): w156–w168, (published online January 26, 2007).
10. Valerie M. Crawford, Mark S. Schlager, William R. Penuel,
and Yukie Toyama, “Supporting the Art of Teaching in a
5. The data must fit the goal Data-Rich, High-Performance Learning Environment” in
Linking Data and Learning, E. B. Mandinach and M. Honey,
The best data allows users to not only assess eds. (New York: Teachers College Press, in press) http://
performance—of students, educators, or ctl.sri.com/publications/downloads/Crawford_etal_
administrations—but also to understand the TechSupportArtTeaching.pdf
processes that could improve learning. Its message 11. Thomas D. Snyder, ed., 120 Years of American Education:
should be not just “You’re failing.” It should be “Here’s A Statistical Portrait (Washington, DC: National Center for
Education Statistics, 1993) http://nces.ed.gov/pubs93/93442.
where you are. Here’s where you need to go. And here
pdf
are the suggested actions for getting you there.” High
12. Chrys Dougherty, A Policymaker’s Guide to the Value
quality data can empower students, while teaching
of Longitudinal Student Data (Denver, CO: Education
educators and institutions much about their practice. Commission of the States, September 2002) http://www.
The most successful performance management dataqualitycampaign.org/files/Publications-Policymakers_
initiatives—those that actually change practice and Guide_to_Longitudinal_Student_Data_090102.pdf
get results—will engage educators, reward use, and 13. See the Data Quality Campaign for annual reports on state
put student success and responsibility at the center. data capabilities, www.dataqualitycampaign.org
14. TEA Data Collection, Analysis and Reporting Systems
Investigation (TDCARSI) Issues and Recommendations (Austin,

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 15

 TX: Texas Education Agency, January 27, 2009) http://ritter. 27. See GothamSchools for a series of articles about ARIS: http://
tea.state.tx.us/tea/IBM_TDCARSI_Recommendation.pdf gothamschools.org/tag/aris/
15. Larry Berger, in discussion with author, Feb. 23, 2009. 28. Philissa Cramer, “Principals Are Optimistic About ARIS But
Kinks Continue,” GothamSchools, August 29, 2009, http://
16. For additional background information on different types of
gothamschools.org/2009/08/20/principals-are-optimistic-
data for different types of uses, please see Glynn D. Ligon, about-aris-but-kinks-continue/
Why Eva Baker Doesn’t Seem to Understand Accountability
(Austin, TX: ESP Solutions Group, 2007) http://www. 29. Javier C. Hernandez, “Data System is Now Supported by Most
espsolutionsgroup.com/espweb/library.html Principals, Survey Says,” New York Times, August 20, 2009.
17. Valerie M. Crawford, Mark S. Schlager, William R. Penuel, and 30. Emily Weiss, in discussion with author, April 26, 2010.
Yukie Toyama, “Supporting the Art of Teaching in a Data-Rich,
31. Shael Suransky, in discussion with author, July 9, 2010. For
High-Performance Learning Environment” in Linking Data and
Learning. further discussion of the importance of safe collaboration
space, see also “Teacher Team Inquiry: Sustaining Instructional
18. “Stimulating Health Information Technology,” event sponsored Improvement at New Dorp High School.”
by Health Affairs on March 10 2009, in Washington, D.C.
32. “Technology That Spurs Data-Driven Achievement,” panel
19. Shael Suransky, in discussion with author, July 9, 2010. discussion at Education Week Data Event, April 19, 2010.
20. Marian Robinson, Patricia Kannapel, Joan Gujarati, Hakim 33. Anne LaTarte, in discussion with author, April 26, 2010.
Williams, and Andrea Oettinger, A Formative Study of the
34. Carmina D’Angelo-Schiavi, in discussion with research
Implementation of the Inquiry Team Process in New York City
Public Schools: 2007–08 Findings (New York: Consortium for assistant Catherine Cullen, March 5, 2009. For more
Policy Research in Education, Teachers College, Columbia information on Schools Attuned, see http://www.
University, September 2008); Shael Suransky, in discussion allkindsofminds.org/
with author, July 9, 2010. 35. Susan Fairchild and Beverly Donohue, in discussion with
21. Shael Suransky, in discussion with author, July 9, 2010,
author, June 28, 2010.
36. Ta-Nehisi Coates, “The Littlest Schoolhouse,” The Atlantic,
22. “Teacher Team Inquiry: Sustaining Instructional Improvement
at New Dorp High School,” Case Study, New York City July/August 2010, http://www.theatlantic.com/magazine/
Department of Education, Staten Island, NY, http://slcp. archive/2010/07/the-littlest-schoolhouse/8132/1/
ed.gov/success-stories/teacher-team-inquiry-sustaining- 37. School of One Brochure, New York City Department of
instructional-improvement-at-new-dorp/ (accessed July 9, Education, http://schools.nyc.gov/NR/rdonlyres/9435AD08-
2010). 90F3-42AA-838C-6372C3B5D2E6/0/SchoolofOneBrochure_
23. Ibid.
FINAL.pdf (accessed July 16, 2010).
38. Barbara Taveras, Caissa Douwes, Karen Johnson, with Diana
24. ARIS was announced to much hoopla in March 2007. Schools
Chancellor Joel Klein said it would “enable educators to tailor Lee and Margaret Caspe, New Visions for Public Schools:
instruction to their students’ needs, and parents will able to get Using Data to Engage Families, (Cambridge, MA: Harvard
involved in their children’s educations like never before.” The Family Research Project, May 2010), http://www.hfrp.org/
district said that parents would get ARIS-generated reports on family-involvement/publications-resources/new-visions-for-
performance beginning that fall, and would have online access public-schools-using-data-to-engage-families
to the system next year. See Mary Hayes Weier, “Can an $80 39. Ann Gordon-Chang, in discussion with Catherine Cullen,
Million IBM Deal Save New York City’s Schools?” Information March 5, 2009.
Week, March 6, 2007, http://www.informationweek.
com/news/infrastructure/management/showArticle. 40. Heather Weiss, founder and director of the Harvard Family
jhtml?articleID=197800547 Research Project, guest post on The Quick and the Ed blog,
http://www.quickanded.com/2010/03/five-big-ideas-for-
25. Yoav Gonen, “Schools Computer an $80M ‘Disaster,’” NY data-rigor-without-mortis.html
Post, February 27, 2008, http://www.nypost.com/p/news/
regional/item_GskUYOLHM49DrQElopcvfL;jsessionid=9500 41. Jennifer Saltzstein, program director, ARIS Parent Link,
F75B516E639299DA229AC46BA888 Webinar presentation, Data Driven: Making Student and School
Data Accessible and Meaningful to Families, August 10, 2010,
26. Elissa Gootman, “As Schools Face Cuts, Delays on http://www.hfrp.org/publications-resources/browse-our-
Data System Bring More Frustration,” New York Times, publications/webinar-brief-data-driven-making-student-and-
October 23, 2008, http://www.nytimes.com/2008/10/24/ school-data-accessible-and-meaningful-to-families
education/24aris.html?pagewanted=1&_r=1&emc=eta1;
see also Elizabeth Green, “High Hopes for New ARIS Data 42. Marty Bell, Jefferson County Schools, in discussion with
Warehouse After Stumbles,” GothamSchools, October 24, Catherine Cullen, March 2009
2008, http://gothamschools.org/2008/10/24/high-hopes-for- 43. From Harlem Children’s Zone website, http://www.hcz.org/
new-aris-data-warehouse-after-stumbles/; Elizabeth Green, about-us/the-hcz-project (accessed May 24.2010).
“New Visions Tells Principals It ‘Overstated’ Problems With
ARIS,” GothamSchools, Dec. 11, 2008, http://gothamschools. 44. George Khaldun, in discussion with author, March 6, 2009.
org/2008/12/11/new-visions-tells-principals-it-overstated-
problems-with-aris/

16 EDUCATION SECTOR REPORTS: Putting Data Into Practice www.educationsector.org

45. While ARIS is available to all New York City district public
schools, it’s not available to charter schools, such as those
operated by the Harlem Children’s Zone.
46. Sophie Lippencott, in discussion with author, May 5, 2010.

47. Shael Suransky, in discussion with author, July 9, 2010.

48. See Principal Satisfaction Survey, New York City Department of

Education, April 2010, page 31, http://schools.nyc.gov/NR/
rdonlyres/CD79D1EC-9B48-47B8-96F9-C2BC1AC0A4C9/0/
PrincipalSurvey_SPRING2010_FINAL.PDF
49. Tools for better data on the frequency of use are currently
under development, see Jennifer Saltzstein, Webinar
presentation, Data Driven: Making Student and School Data
Accessible and Meaningful to Families, August 10, 2010.
50. Anne LaTarte, in discussion with author, May 26, 2010.

51. Formative Study of the Implementation of the Inquiry Team

Process in New York City Public Schools, http://www.cpre.
org/index.php?option=com_content&task=view&id=258&Ite
mid=76
52. Shael Suransky, in discussion with author, July 9, 2010.

53. Additional New York City schools are also experimenting with
homegrown data systems. See Anna Phillips, “Frustrated
With City’s Data System, Teachers Build Their Own,”
GothamSchools, September 15, 2010, http://gothamschools.
org/2010/09/15/frustrated-with-citys-data-system-teachers-
build-their-own/
54. For example, health provider Kaiser Permanente proved that
better use of electronic medical data could significantly reduce
costs and improve care. But change has been slow, expensive,
and wrenching. According to BusinessWeek, Kaiser has spent
$4 billion on the data effort and “encountered disgruntled
doctors, system outages, and a temporary decrease in
productivity as physicians get accustomed to the new
system.” See Rachael King, “How Kaiser Went Paperless,”
BusinessWeek, April 7, 2009 and Anna-Lisa Silvestre, Valerie
M. Sue, and Jill Y. Allen, “If You Build It, Will They Come?
The Kaiser Permanente Model of Online Health Care,” Health
Affairs, March/April 2009.
55. Thomas Davenport, Don Cohen, and Al Jacobson, “Competing
on Analytics” Working Knowledge Research Report, (Babson
Park, MA: Babson Executive Education, May 2005).

www.educationsector.org EDUCATION SECTOR REPORTS: Putting Data Into Practice 17