Blogs (19) >>
Thu 16 Mar 2023 10:45 - 12:00 at Online Authors' Corner - Online Authors' Corner 1

The teaching and assessment of introductory programming involves writing code that solves a problem described by text. OpenAI Codex is a trained machine learning model that aims to do exactly that. Previous research found that Codex performs well on many programming problems, often generating correct and readable Python code. While Codex requires payment, GitHub Copilot, an IDE plugin that uses Codex, is freely available to students. This raises pedagogic and academic integrity concerns. Educators need to know what Copilot is capable of, in order to adapt their teaching to the age of AI-powered IDE plugins. Previous research investigated the most performant of the four Codex models and evaluated it quantitatively: how many tests does each generated code suggestion pass and how many problems have at least one correct suggestion that passes all tests. Here I evaluate Copilot instead, to see if and how it differs from Codex, and look qualitatively at the generated suggestions, to understand the limitations of Copilot. I also report on the experience of using Copilot for other activities asked of students in programming courses: explaining code, generating tests and fixing bugs. The paper concludes with a discussion of the implications of the observed capabilities for the teaching of programming.

Thu 16 Mar

Displayed time zone: Eastern Time (US & Canada) change

10:45 - 12:00
Online Authors' Corner 1Papers at Online Authors' Corner

Opportunity for attendees to connect with authors for interactive Q&A and discussion

10:45
75m
Paper
Reducing Procrastination Without Sacrificing Students' Autonomy Through Optional Weekly Presentations of Student-Generated ContentOnline
Papers
Iman YeckehZaare University of Michigan - School of Information, Sean Chen University of Illinois at Urbana-Champaign, Tirdad Barghi NTP (Ltd.)
DOI
10:45
75m
Paper
Characterizing Women’s Alternative Pathways to a Computing Career Using Content AnalysisOnline
Papers
Jia Zhu Florida International University, Stephanie Lunn Florida International University, Monique Ross The Ohio State University
DOI
10:45
75m
Paper
Securely Autograding Cybersecurity Exercises Using Web Accessible Jupyter NotebooksOnline
Papers
Mac Malone University of North Carolina at Chapel Hill, Yicheng Wang , Fabian Monrose Georgia Institute of Technology
DOI
10:45
75m
Paper
Using GitHub Copilot to Solve Simple Programming ProblemsOnlineGlobal
Papers
Michel Wermelinger The Open University
DOI
10:45
75m
Paper
Logistics, Affordances, and Evaluation of Build Programming: A Code Reading Instructional StrategyOnline
Papers
Amanpreet Kapoor University of Florida, USA, Tianwei Xie University of Florida, Leon Kwan University of Florida, Christina Gardner-McCune Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL, USA
DOI
10:45
75m
Paper
Cyber Security in English Secondary Education Curricula: A Preliminary StudyOnlineGlobal
Papers
Ollie Stepney University of Gloucestershire, Jordan Allison University of Gloucestershire
DOI
10:45
75m
Paper
Programming Is Hard - Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code GenerationCCIn-PersonPCI Best PaperGlobal
Papers
Brett Becker University College Dublin, Paul Denny The University of Auckland, James Finnie-Ansley The University of Auckland, Andrew Luxton-Reilly The University of Auckland, James Prather Abilene Christian University, Eddie Antonio Santos University College Dublin
DOI
10:45
75m
Paper
Modeling Determinants of Undergraduate Computing Students’ Participation in InternshipsOnline
Papers
Megan Wolf University of Florida, Amanpreet Kapoor University of Florida, USA, Charlie Hobson University of Florida, Christina Gardner-McCune Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL, USA
DOI
10:45
75m
Paper
The Engaging Computer Science Education Laboratory: A Mixed-Methods-Based Design of an Innovative Classroom for Informatics Teacher EducationOnlineGlobal
Papers
Andreas Dengel Goethe-University of Frankfurt, Tim Läufer Goethe University Frankfurt, Rupert Gehrlein Goethe-University of Frankfurt
DOI
10:45
75m
Paper
Who Wins? A Comparison of Accessibility Simulation Games vs. Classroom ModulesOnline
Papers
Devorah Kletenik Brooklyn College, City University of New York, Rachel F. Adler Northeastern Illinois University
DOI