Machine learning (ML) models are often included in high-risk algorithmic decision making software, making them a particularly important facet of ethics education for those creating them to make sure they are unbiased and fair to all users. Natural Language Processing (NLP) specifcally functions on text, a human produced artifact, making it even more prone to inheriting fawed biases. However, teaching about ethics in machine learning courses is lacking. To address this issue, we created three interventions in an NLP course to introduce students to biases in existing machine learning models. We employed a combination of hands-on programming activity, lecture, programming quiz, and a project that discuss machine learning fairness at diferent levels and for different populations including gender bias and disability bias. Each of the teaching interventions included a refection question on students’ learning about bias. After the completion of the course, we interviewed six students to further understand the impact of the interventions. The answers to the refection questions and the interviews were qualitatively analyzed. We found that integrating fairness topics throughout the NLP course with repeated discussions led to an overall positive shift in students’ attitudes and awareness towards ML fairness and biases.