Research on retaining students in computer science is clear: pedagogical and curricular interventions that are experienced by all students are the most effective. Yet understanding what brings faculty to modify their teaching practices is scant. This study presents findings from a study based on interviews with 66 computer science professors and instructors in the U.S. Faculty described various sources of student feedback that prompt them to change their teaching practices. Though not originally an interview topic, every instructor mentioned ways in which student feedback instigated change. Faculty described several sources of feedback that influence teaching choices, with the goal of trying to improve their teaching. These include formalized evaluations; feedback inferred from student performance; direct requests from students; and students' nonverbal behavior. As sources of evidence for making decisions about teaching practices, each of these is problematic. We describe what makes them problematic and offer solutions described by faculty interviewed.